How to Check Statistics in News Stories
A reader’s checklist for denominators, baselines, uncertainty, absolute risk and cherry-picked time windows.
Find the denominator
“Thousands affected” means little without the population or total number of events.
Check absolute and relative change
A doubling from one case to two is a 100% increase but an absolute increase of one. Context changes what a headline means.
Inspect the time window
Selecting a convenient start date can turn normal volatility into a dramatic trend.
Separate estimate from count
Models, projections and survey estimates have uncertainty that administrative counts may not.
Look for revisions
Economic, health and election data can be revised after initial release.
Read the source table
The underlying table often contains footnotes that materially limit the headline claim.
Percentage-point vs. percent-change calculator
For a quick sanity check, compare the claimed change with the underlying totals and another reputable dataset covering the same period. Large discrepancies do not automatically prove error, but they tell you where definitions, geography or timing need closer inspection.
Administrative counts, surveys, modelled estimates and rates are not interchangeable. Survey results have sampling uncertainty and question wording. Administrative data can change when reporting rules or coverage change. Model outputs depend on assumptions. Before treating a number as a fact about the world, identify what was actually measured and what had to be inferred.
Check whether the statistic represents people, events or estimates
Then find the baseline. A single month can be compared with the previous month, the same month last year, a pre-policy period or a long-run average. Each comparison answers a different question. The article should tell you why its chosen baseline is the relevant one.
Write the statistic as a small equation. Identify the numerator, denominator, unit and period. “Cases doubled” means something different when the count moved from 2 to 4 than when it moved from 20,000 to 40,000. Percentage change can describe movement, while percentage points describe the gap between two rates; confusing the two can make an ordinary change sound much larger.
Rebuild the number before judging the headline
Series can also break when agencies revise definitions, survey methods or data collection. A sudden jump near a methodological change deserves a footnote before it deserves a causal story. Read the dataset notes and look for revisions, provisional values and breaks in series.
Rates can rise because the numerator changed, the denominator changed, or both. Crime per 100,000 people, hospitalisations per infected person and spending as a share of GDP each depend on a denominator that may move independently. Check whether a comparison quietly switches from counts to rates or from one population definition to another.