Almost every misleading statistic you meet is true. Outright fabrication is rare and easily caught, so selection and framing do the real work — while remaining defensible. That is why "check the facts" is not enough. The facts usually check out. What needs checking is what was selected, what was left out, and what the number was divided by.
Here are the red flags worth memorising, each with a question that tends to dissolve the trick.
Watch what was selected
Cherry-picked baselines and windows
Any noisy series contains stretches that trend up and stretches that trend down. Choose the start date and you choose the story. "Sales are up 30% since March" invites the question: why March? If March was an unusual low, almost any later month looks like a triumph.
Ask: why this period, this subgroup, this comparison?
Survivorship bias
When you analyse only what remained and ignore what disappeared, the survivors flatter the picture. "Our graduates earn well" quietly omits everyone who dropped out. Fund performance tables look strong because closed funds vanish from the record. The classic case is the Second World War aircraft studied for where to add armour: engineers first wanted to reinforce the damage on returning planes, until the statistician Abraham Wald pointed out the data covered only planes that came back. The undamaged areas were where a hit was fatal — so those planes never returned to be counted.
Ask: what left the dataset, and would it have looked different?
Watch the denominator
Missing denominators
"Forty incidents this year" — out of how many? An absolute count with no base is uninterpretable. A rising count often just reflects a bigger population, better detection, or a changed definition, not a real change in rate.
Ask: out of what, and did the definition change?
Percentage of a percentage
"Support rose 5%" can mean 40% to 45% (five percentage points) or 40% to 42% (5% of 40%). Both are written identically and mean very different things. A percentage point and a percent change are distinct quantities, and conflating them is the most common numerical sleight in reporting.
Relative risk without the baseline
"This doubles your risk" sounds alarming until you learn the risk went from 1 in 10,000 to 2 in 10,000. Relative risk — the ratio between two chances — is uninterpretable without the absolute risk it sits on. Whenever you meet a percentage increase in danger, hunt for the plain, absolute number underneath.
Ask: doubled from what?
Watch the framing
Misleading averages
A single "average" hides the shape of the data. When a distribution is skewed, the mean gets dragged toward the extremes while the median stays put. Quoting the mean salary at a firm with a few enormous earners tells a very different story from the median. Neither is wrong; the choice is the message.
Correlation dressed as causation
Two things moving together is a pattern, not an explanation. Ice-cream sales and drowning deaths rise together, but hot weather drives both — a confounding variable doing the work while the two visible numbers take the credit. Before accepting "X causes Y," rule out a confounder, reverse causation (the arrow pointing the other way), and plain coincidence, which enough comparisons guarantee.
Ask: what else could produce this pattern?
Chart manipulations
Accurate numbers still mislead through the picture. A truncated axis magnifies a trivial wiggle into a cliff. Two independent vertical axes can be scaled so unrelated lines appear to move in lockstep. Using area to show a single value means doubling the number quadruples the visual footprint. The numbers can all be right while the impression is wrong.
Unrepresentative samples
A sample only speaks for the population it was actually drawn from. An online poll captures people who chose to click (selection bias); a survey where only the angry reply captures a mood, not a majority (non-response bias). "Nine in ten users prefer it" means little if the ten were hand-picked.
Ask: who was measured, and who could not be?
The three questions
Most misleading numbers fall to just three: out of what, compared with what, and what is missing? Note too that not every framing choice is manipulation — truncating an axis to show real variation is often necessary, and relative risk is right when comparing risk factors. Honest framing survives being asked "why this choice?"; manipulation does not. Assuming bad faith by default is its own error.
Keep learning: explore the Statistics for Everyday Life lessons on sampling, risk, and misleading statistics to practise these questions on real claims.