Frequentist and Bayesian Thinking

Behind almost every statistical method sits a quiet philosophical choice: what does probability actually mean? Two answers have competed for centuries. To a frequentist, a probability is a long-run frequency, the fraction of times an event happens if you could repeat the situation endlessly. To a Bayesian, a probability is a degree of belief, a number expressing how confident you are, which you revise as evidence arrives.

This is not idle philosophy. The two views lead to different methods, different ways of reporting results, and different answers to everyday questions. Knowing which lens is being used helps you read studies, forecasts, and risk estimates without being misled, and helps you choose the right approach for your own questions.

Two meanings of probability

Say there is a 30 percent chance of rain tomorrow. A strict frequentist reading is awkward: tomorrow happens only once, so what is the long-run frequency of this particular day? A Bayesian reading is natural: 30 percent is the forecaster's degree of belief given the evidence. Now say a fair coin lands heads with probability 0.5. Here the frequentist reading is natural: over many tosses, about half land heads. Each interpretation fits some situations more comfortably than others.

The frequentist view

Frequentist methods, developed by figures such as Ronald Fisher and, later, Jerzy Neyman and Egon Pearson, treat the unknown quantity as fixed and the data as random. Probability describes how the data would behave over many hypothetical repetitions. This is the world of p-values, confidence intervals, and significance tests you met earlier. Its strength is that it does not require you to state a prior belief. Its weakness is that its statements are easy to misread: a p-value is not the probability that a hypothesis is true.

The Bayesian view and updating

Bayesian methods rest on a rule published in the eighteenth century, discussed in the case study below. You begin with a prior, your belief before seeing the data, then use the evidence to compute a posterior, your updated belief. The rule weighs how likely the evidence would be under each competing explanation. As more data arrives, a well-chosen prior matters less and the data dominates. The appeal is that it answers the question people usually want, given what I have seen, how probable is this, and that it updates smoothly as new evidence comes in.

Two bowls of cookies sit before you. Bowl 1 holds 30 vanilla and 10 chocolate; Bowl 2 holds 20 vanilla and 20 chocolate. You pick a bowl at random and draw a vanilla cookie. How probable is it that you reached into Bowl 1?

Start with a prior: each bowl is equally likely, 0.5. Vanilla is more common in Bowl 1: the chance of drawing vanilla is 30 of 40, or 0.75, from Bowl 1 and 20 of 40, or 0.5, from Bowl 2. Bayes's rule combines these: the posterior for Bowl 1 is 0.75 divided by (0.75 plus 0.5), which equals 0.6. The vanilla cookie nudged your belief from 50 percent up to 60 percent that it was Bowl 1. That single, transparent step, prior then evidence then posterior, is Bayesian reasoning in miniature.

Neither school wins everywhere, which is the real point. Imagine a factory stamping out millions of identical parts. Asking what fraction are defective in the long run is a textbook frequentist question, and long-run frequency is exactly the right language. Now imagine a one-off event: will this specific new bridge design fail in its first year? There is no long run of identical bridges to count, so a pure frequency has no clear meaning; expressing your belief and updating it as inspections come in is more natural. A poorly chosen prior can also mislead a Bayesian: start almost certain of the wrong thing and it takes a lot of data to recover. The lens should fit the question, not the other way round.

The core rule comes from Thomas Bayes, an English minister, whose essay Towards Solving a Problem in the Doctrine of Chances was read to the Royal Society in 1763, two years after his death, by his friend Richard Price. Pierre-Simon Laplace independently developed and greatly extended the idea soon after. For much of the twentieth century Bayesian methods were sidelined in favour of frequentist statistics, then revived as computers made them practical.

Two modern uses show the payoff. Early spam filters used a naive Bayes approach, popularised by Paul Graham's 2002 essay A Plan for Spam and studied earlier by researchers such as Sahami and colleagues in 1998, updating the probability that a message is spam word by word. And Bayesian search theory has helped find lost vessels: the US Navy used it to help locate the submarine USS Scorpion in 1968, and in 2011 the consultancy Metron applied Bayesian analysis to build a probability map that guided searchers to the wreckage of Air France Flight 447 on the Atlantic sea floor, after two years of earlier searches had failed. In each case a prior map of belief was updated by evidence until the target was found.

• Bayesian means subjective and therefore unscientific. Priors are stated openly and are overwhelmed by enough data.
• A p-value is the probability that the hypothesis is true. It is not; it describes the data assuming a hypothesis, a frequentist idea.
• You must be loyal to one school forever. Working statisticians use whichever approach fits the problem.
• A prior is just a wild guess. Priors often come from real earlier data or well-established knowledge.

You are given a prior and a piece of evidence: a search zone with an initial probability map, then a sonar sweep that comes back empty in one sector. Redistribute the probability across the remaining sectors and say where you would look next, explaining how the evidence changed your belief.

Think Like a Maester: Frequentist and Bayesian thinking are two lenses on the same world, one asking how often, the other how sure, and the skilled analyst knows when to reach for each.

1. In one sentence each, define probability as a frequentist and as a Bayesian would.
2. In the cookie example, why does drawing a vanilla cookie raise your belief in Bowl 1 to 60 percent?
3. What are the three ingredients of a Bayesian update?
4. Give one situation that suits a frequentist view and one that suits a Bayesian view.
5. Who first stated the rule behind Bayesian reasoning, and how did it reach the public?

Two great schools interpret probability differently. Frequentists treat it as a long-run frequency and build p-values, confidence intervals, and significance tests around fixed unknowns and random data. Bayesians treat it as a degree of belief, starting from a prior and updating to a posterior as evidence arrives, using a rule first set out by Thomas Bayes and published in 1763. The cookie example shows the update in miniature; spam filters and Bayesian search-and-rescue, including the map that guided searchers to Air France 447's wreckage in 2011, show its modern power. Neither school is universally right. The mature view is to understand both and match the lens to the question.

• Conditional Probability and Base Rates
• Landmark Studies That Changed the World