Forecasting and Prediction

Forecasts govern decisions we cannot avoid: whether to carry an umbrella, stock a warehouse, price an insurance policy, or brace for an election result. Yet we often judge them by a standard they never claimed to meet, treating likely as certain and then declaring the forecaster wrong when the unlikely happens. This misreading has real costs. It makes us distrust honest probabilistic forecasts and prefer confident ones that are actually worse. Understanding what a forecast is, and is not, lets you use uncertain information well, hold forecasters to a fair standard, and make better calls in exactly the conditions where certainty was never on offer.

A forecast is a probability

A good forecast assigns odds to outcomes; it does not pronounce what will happen. "70% chance of rain" is a claim about likelihood, and a single dry day cannot refute it, because thirty-percent events happen roughly three times in ten. The right test is calibration across many forecasts: of all the times a forecaster says 70%, it should rain about 70% of the time. Judge the long-run scorecard, not the single outcome.

Probabilistic versus point forecasts

A point forecast names one number: sales will be 4,200, the candidate gets 51%. A probabilistic forecast gives a range and the odds across it: most likely 3,800 to 4,600, centred near 4,200. The point version feels precise and hides all its uncertainty; the probabilistic version is honest that the future is a spread of possible outcomes, not a single fixed one. When someone quotes a confident single number, ask what range sits behind it.

What good forecasters do

Research finds that the best forecasters share habits more than raw brilliance. They think in degrees rather than yes-or-no, updating their probabilities as evidence arrives instead of digging in. They break large questions into smaller, answerable pieces. They keep score, learn from misses, and, crucially, avoid overconfidence, leaving room for outcomes they did not expect. Frequent, modest revision beats bold, fixed prediction.

Imagine a weather service that says "70% chance of rain" on a hundred days. If it is well calibrated, it rains on about seventy of them and stays dry on about thirty. On any one dry day you might feel misled, but the dry days are part of the forecast, not evidence against it. Now suppose a rival always declares rain or no rain with total confidence and is right sixty days out of a hundred. It sounds decisive, yet it is less useful: it never tells you how sure to be, so you cannot size your decisions. Judged over the full run, the probabilistic forecast carries more information.

Probabilities are not an excuse that makes every forecast unfalsifiable. A forecaster who says "60% chance" every single day, regardless of conditions, cannot be caught out on any one day either, but the long-run record exposes them. If it rains on only 20% of their supposed 60% days, they are badly miscalibrated, and no appeal to "it was only a probability" saves them. Probabilistic forecasts are testable; they are simply tested over many predictions rather than one. The escape hatch closes as soon as you start counting.

Modern election and weather forecasting is openly probabilistic. The statistician Nate Silver and his FiveThirtyEight model, for instance, expressed the 2016 US presidential race as odds rather than a verdict, giving Donald Trump roughly a thirty-percent chance of winning on election day, higher than many rival models. When Trump won, many called the forecast wrong, but a thirty-percent event occurring is not a refutation; it is what thirty percent means. The deeper lesson is to read such numbers as probabilities, not predictions.

The psychologist Philip Tetlock supplied much of the evidence base. His long-running studies found that many confident expert pundits forecast world events barely better than chance. But in the Good Judgment Project, part of a US-government-sponsored forecasting tournament in the 2010s, Tetlock's team identified ordinary people, dubbed superforecasters, who reliably beat others. Their edge came not from expertise or secret information but from method: thinking in fine-grained probabilities, updating often in small steps, and staying open to being wrong.

• "A 70% forecast that doesn't happen was wrong." No single outcome tests a probability; calibration over many does.
• "A confident single number is a better forecast." It hides its uncertainty and cannot tell you how much to trust it.
• "Probabilities make forecasts unfalsifiable." They are falsified over long runs by poor calibration.
• "Good forecasters are simply smarter experts." The evidence points to habits, such as updating, humility, and thinking in odds, over raw expertise.

Sort a season of "70% chance" forecasts against what actually happened, and judge whether the forecaster was genuinely well calibrated or merely lucky-sounding.

Think Like a Maester: A forecast is a distribution of futures, not a prophecy of one. Judge it by the scorecard of many calls, and trust the forecaster who says probably over the one who says certainly.

1. Why is "70% chance of rain" not proven wrong by a single dry day?
2. What does it mean for a forecaster to be well calibrated?
3. How does a probabilistic forecast differ from a point forecast?
4. What habits distinguish the best forecasters in Tetlock's research?
5. Why isn't a confident single-number forecast necessarily the most useful one?

A good forecast states the odds, not the outcome, and describes the future as a range rather than a point. It cannot be judged by one result, only by calibration across many, where the 70% days really do come true about seven times in ten. The best forecasters, as Tetlock's work suggests, win through method rather than genius: they think in probabilities, update often, and hold their confidence loosely. Read forecasts as probabilities and you will both use them better and judge them more fairly.

• Randomness, Streaks, and Regression
• Statistical Significance