Probability Distributions

A single average tells you where a process tends to land; a distribution tells you everything it can do and how often. Most consequential questions — how bad could a loss be, how rare is this result, should we be surprised — are questions about the shape of a distribution, not its centre. The costliest mistakes in finance, engineering, and planning come from quietly assuming one particular shape, the bell curve, and then being blindsided when reality turns out to have a fatter tail than the curve allowed. Learning to ask what shape you are dealing with is one of the highest-leverage statistical habits there is.

A distribution is the whole picture

A probability distribution lists every outcome a quantity can take and how likely each one is. Roll a fair die and the distribution is flat: six outcomes, each equally likely. Measure adult heights and the distribution is a hump — most people near the middle, fewer at the extremes. The distribution is the complete answer; an average is only one summary drawn from it.

The normal curve and the 68/95 rule

The most famous shape is the normal, or bell-curve, distribution: symmetric, single-humped, thinning smoothly toward both tails. It tends to appear whenever an outcome is the sum of many small independent influences — measurement errors, heights, exam totals. Its convenience is a rule of thumb worth carrying: roughly 68% of values fall within one standard deviation of the mean, and about 95% within two. So a value three or four standard deviations out should be genuinely rare — under a normal curve.

When normal is the wrong shape

Many things are not normal. Incomes, city sizes, book sales, and insurance losses are right-skewed and heavy-tailed: a long, thick tail of large values that a bell curve would treat as all but impossible. Under a normal curve a 'ten-sigma' event essentially never happens; in heavy-tailed data such extremes arrive often enough to dominate the total. Assuming normality here does not just mislead — it systematically hides the very events that matter most.

Suppose adult male heights are roughly normal, mean 175 cm, standard deviation 7 cm. The 68/95 rule says about 68% of men fall between 168 and 182 cm, and about 95% between 161 and 189 cm. A man of 205 cm sits more than four standard deviations out — under a normal curve, rarer than one in ten thousand, which matches reality: such heights exist but are very unusual. Here the bell curve earns its keep, because height genuinely is the sum of many small factors and has no runaway tail.

Now apply the same reasoning to wealth. Suppose average wealth in a room is $80,000. Treat it as normal and you would expect almost nobody beyond a few hundred thousand. Then one person worth $50 billion walks in. Under a normal curve that is a near-impossibility, dozens of standard deviations out. In the real, heavy-tailed distribution of wealth it is merely rare — and when it happens, the single value swamps everyone else combined. The normal model does not just misestimate the tail; it denies the tail exists.

Francis Galton, a 19th-century British polymath, did much to popularise the normal distribution. He built a device now called the Galton board: balls dropping through rows of pins pile up into a bell shape, a physical demonstration of how many small random deflections sum to a normal curve. Galton wrote of the pattern in near-reverent terms. The lasting caution — clearer in later work on economics and extreme events — is that the bell curve was over-applied, assumed to describe phenomena such as financial returns that are in fact heavy-tailed. The details of who claimed what are worth checking, but the shape of the error is not in doubt: the curve is real and useful, and treating it as universal is the mistake.

• "Most data is normally distributed." Many important quantities — income, wealth, city sizes, losses — are not.
• "The 68/95 rule always applies." It applies to the normal curve; for heavy-tailed data it badly understates the extremes.
• "A rare event proves the model was right to call it rare." Under the wrong shape, 'rare' events are actually common.
• "The average captures the distribution." The average is one point; the shape and the tails carry the risk.

Given a real quantity, decide whether its distribution is roughly normal or heavy-tailed, and predict where the surprises will come from.

Think Like a Maester: Before trusting any 'this is extremely unlikely', ask what shape was assumed. Under a bell curve almost nothing extreme can happen — which is exactly why bell curves get people hurt.

1. What does a probability distribution tell you that an average does not?
2. State the 68/95 rule and the shape it applies to.
3. Name two quantities that are heavy-tailed rather than normal.
4. Why does assuming normality underestimate rare extremes?
5. What did Galton's board demonstrate, and what is the danger in over-applying the bell curve?

A distribution is the full map of what a quantity can do and how often; the average is only its centre. The normal curve, popularised by Galton, describes sums of many small effects and follows the handy 68/95 rule — but many of the quantities that matter most are skewed and heavy-tailed, and assuming a bell curve there quietly erases the rare, large events that end up dominating the outcome.

• Averages and Distributions
• Expected Value and Decisions