Pro Analytics
Methodology
How the regression and machine-learning models behind PoliYT's recommendations, factor breakdowns, and per-video scores are built, tested, and reported — including where they're weakest.
Overview
PoliYT fits two kinds of model, nightly, against every tracked video: a ridge regression (interpretable, coefficient-by-coefficient) and a gradient-boosted model (XGBoost) (more accurate, less directly interpretable). Both describe patterns already present in the videos we've collected. Neither one runs an experiment on your channel. That distinction — description of a corpus vs. a controlled test of your content — shapes every number on this page and every recommendation card on your politician page.
What we predict
Early versions of the model predicted "current views" — but a video's current view count depends heavily on how long ago it was checked, which made politicians and videos hard to compare fairly. The model now predicts views at day 28 after publish, a fixed measurement point for every video (accepting any snapshot in the 28–35 day window). Day-28 views track day-90 views closely in our data — they're not a perfect final number, but they're a stable, comparable one, and don't force us to wait three months to say anything.
Videos younger than 28 days don't get a model score yet. Instead, they get an early-feedback "pacing" read: how their views so far compare to this channel's own historical videos of the same age and format — an honest "ahead of / behind / in line with your usual pace" signal, not a prediction.
The ridge model
The primary model is a ridge regression on log(views), with content features (title, thumbnail, transcript, and video-content signals), channel size, and a politician random effect. Standard errors are two-way cluster-robust — clustered on both the individual politician and the calendar week a video published in. Single-axis clustering (politician only) understates uncertainty, because videos published by different politicians in the same week are correlated too: shared news cycles, coordinated messaging, and the same recommendation-algorithm conditions all push same-week videos together. Two-way clustering means some effects that looked "significant" under the old method no longer are — that's the correction working, not a regression.
Recommendations and factor breakdowns are generated from the Democrat-only model variant, not the all-politician one. A Republican politician's page shows the same historical charts, ad-spend transparency, and content analysis as anyone else's — but no personalized recommendations, because the model those recommendations are drawn from is fit specifically on Democratic-channel patterns and we don't think it's honest to imply otherwise.
Honest accuracy
"Model accuracy" is reported four different ways, because a single R² number hides more than it reveals:
| Measure | What it answers |
|---|---|
| In-sample R² | How well the model fits the data it was trained on. Always the highest number, and the least useful one — it doesn't tell you anything about new videos. |
| Random holdout R² | Accuracy on videos the model didn't train on, chosen at random. Still optimistic, because it can share a channel and time period with training data. |
| Temporal holdout R² (headline number) | Accuracy predicting more recent videos from older ones — the situation the model is actually used in nightly. |
| Leave-one-channel-out R² | Accuracy on a politician the model has never seen at all. This is deliberately the hardest test and the lowest number — it tells you how much of the model's power is "this specific channel" vs. general content patterns. |
The Research page leads with temporal holdout accuracy, because that's the test that matches how the model is actually used. We don't publish leave-one-channel-out accuracy as a marketing number, because every paying subscriber's channel is already in the training data — a "how well does this generalize to politicians we've never seen" statistic doesn't describe their situation, and leading with a deliberately pessimistic number there would be its own kind of dishonesty. It's still computed every run, still used internally as a canary, and available on request.
Statistical significance
With dozens of features tested simultaneously, some will look "significant" by chance alone. Published factors are filtered through a Benjamini-Hochberg false-discovery-rate correction — a standard multiple-comparisons fix that controls the share of published findings expected to be false positives, rather than treating each feature's p-value in isolation. This means the published list of "significant factors" is shorter than an uncorrected list would be. That's intentional.
Rare features (present in only a handful of videos) are separately flagged as underpowered when the sample is too small to trust the estimate, regardless of what the p-value says.
The XGBoost model
Alongside the ridge model, we run a gradient-boosted model over the same features. It's more accurate on held-out data, but its coefficients aren't directly readable the way ridge coefficients are — so we use SHAP values (a standard method for attributing a gradient-boosted model's prediction back to its input features) to explain individual predictions and per-politician recommendations.
Two honesty checks specifically target this model, because gradient-boosted models are easy to over-trust: out-of-fold scoring (a video's predicted-vs-actual badge is computed from a model fold that never saw that video during training — scoring in-sample flipped roughly 3 in 10 badges when we checked) and independent-fold confidence votes on per-politician, per-feature claims (a claim only ships if multiple independent cross-validation folds agree on its direction — most raw per-politician, per-feature claims don't clear this bar and are suppressed rather than shown with false confidence).
Some politicians' predictions come from the ridge model, others from XGBoost — decided empirically, per politician, by comparing which model was more accurate on that politician's own held-out videos. A model only overrides the default when it beats the alternative by a meaningful margin on enough test videos; close calls default to the more transparent ridge model.
Within-channel model: what you can actually move
The single biggest driver of a video's view count is how big the channel already is — true, and nearly useless as advice. The within-channel model is built to answer a narrower, more useful question: net of your own channel's typical audience size, which content choices correlate with doing better or worse than your own usual video? It does this by centering each video's outcome against that politician's own median performance before fitting, which removes the channel-size signal entirely and leaves only within-channel variation to explain.
This is a smaller, harder number than the headline model accuracy — deliberately. It's the honest answer to "what can I actually control," not "what predicts a video's views overall" (audience size dominates that question, and audience size isn't a lever you pull per-video). The "What differentiates your high-performing videos" section on your politician page comes from this model.
Peer comparisons
Peer benchmarks group politicians by party × chamber (e.g. Democratic senators, Republican House members) and report percentile distributions — 25th, 50th, and 75th percentile — for views, engagement, posting frequency, and video length within that peer group, over rolling 30/90/365-day windows. The 50th percentile (median) describes a typical peer's current usage; the 75th percentile is a useful "what the more successful quarter of your peer group is doing" reference point. Neither implies causation between any single practice and the views peers in that bracket happen to get.
Paid-video exclusion
Videos identified as paid promotion — matched against Google Ads Transparency records, or flagged by behavioral anomaly scoring when no ad match exists yet — are excluded from the "organic" view counts the models train on. Ad spend can inflate a video's views by orders of magnitude for reasons that have nothing to do with its content, title, or thumbnail; leaving those videos in would teach the model that "having a large ad budget" is a content strategy. Confirmed-paid videos still show up in a politician's Ad Spend Transparency section — they're excluded from the model's training data, not hidden from the site.
Why we avoid "expected lift"
Every model on this page is correlational: it describes what tends to go together across the videos we've tracked. None of them ran an experiment — no A/B test, no random assignment of which politicians tried which content choices. That means a recommendation card that says "videos with this attribute average +12% views in our dataset" is a true, useful, honestly-scoped statement. A recommendation that said "this will increase your views by 12%" would not be — it implies a causal guarantee the data can't support, because politicians who already do well on other dimensions may also happen to use that attribute more. You'll see phrasing like "worth testing on a handful of your own videos and measuring" instead of a promised return, on purpose, everywhere on this site.
Limitations
- Correlational, not causal — see above. Nothing here was produced by an experiment.
- Recommendations are Democrat-only by design (see "The ridge model" above).
- Leave-one-channel-out accuracy is meaningfully lower than temporal-holdout accuracy — the model leans on channel-specific history more than a "pure content" story would suggest.
- Rare features carry wide, sometimes underpowered, uncertainty — flagged rather than hidden.
- Numbers move week to week as more data accumulates and the models are refit nightly (weekly for the full regression refit) — don't compare this week's coefficient to last month's as if the ground didn't shift.
- This is a tracking and analytics product, not an endorsement system — see About & Data Sources for the broader data-coverage caveats.