Choosing the Right Method

This module has assembled a toolkit: t-tests, chi-square, regression, ANOVA, effect sizes and intervals, and robust and nonparametric methods. The final skill is not another test but knowing which to reach for — and remembering that the tool is only ever half the job. Interpretation and honesty are the other half.

Start with the question and the data

Choosing a method flows from two questions. First, what are you asking? Comparing group averages, testing a relationship between categories, predicting a number, comparing many groups? Second, what kind of data do you have? Numerical or categorical, how many groups, and what shape (symmetric or skewed, outliers or not)? The answers point to the tool.

A simple decision guide

Roughly: comparing two groups' averages → t-test (or a rank-based test if skewed). Categorical association → chi-square. Predicting or explaining a number from others → regression. Comparing several groups → ANOVA (then post-hoc). And for any of these, report effect sizes and confidence intervals, not just p-values, and switch to robust/nonparametric versions when assumptions fail.

The tool is half the job

Even the perfect test is worthless if the study was biased, the sample unrepresentative, or the result over-interpreted. Statistics does not replace thinking about design, confounders, and meaning. The best analysts pick a reasonable method and then interpret it honestly, acknowledging uncertainty.

Someone asks whether a new tutoring programme raises test scores. Question: compare two groups' averages. Data: numerical scores, roughly symmetric, decent sample. Choice: a t-test, reported with an effect size and confidence interval. If the scores were badly skewed, they'd switch to a rank-based test. If there were three programmes instead of two, they'd use ANOVA. The method follows from the question and the data — not from habit.

The wrong instinct is to memorise 'which test' as a lookup table and stop thinking. Two analysts can pick the same correct test and reach opposite real-world conclusions because one ignored a confounder or a tiny effect size. And no test rescues a fundamentally flawed study. Method selection matters, but it is the beginning of good analysis, not the end.

The tools in this module were not designed as a menu; they grew from a connected tradition of people solving real problems — Gosset's small samples at Guinness, Pearson's categorical counts, Galton's heights, Fisher's field experiments, Wilcoxon's messy data, and later reformers urging effect sizes over bare p-values. Seen together, they form a coherent way of thinking: state your question precisely, respect the shape of your data, quantify both the size of an effect and your uncertainty about it, and never mistake a test's output for the truth. That mindset — not any single formula — is the real toolkit. It is why a statistically literate person can read a study in an unfamiliar field and still ask the right questions: what was compared, with what data, how big was the effect, how sure are we, and what might have been missed?

• "Pick the test first." Start with your question and your data; the test follows.
• "The right test guarantees a right conclusion." Design, confounders, and interpretation still decide.
• "There is exactly one correct method for every case." Often several are defensible; judgment matters.
• "Statistics is about formulas." It is about asking and answering questions honestly under uncertainty.

Invent a simple question (e.g., 'do two teams differ in average sales?'). State the question type and data type, choose a method from this module, and name one thing you'd check before trusting the result.

Think Like a Maester: Let the question and the data choose the tool — and never let the tool do your thinking for you.

1. What two questions guide method selection?
2. Which method fits comparing two group averages? Categorical association? Several groups?
3. Why report effect sizes and intervals regardless of the test?
4. Why is choosing the right test only half of good analysis?
5. What single mindset unites the whole statistical toolkit?

Choosing a statistical method starts with two questions: what are you asking, and what data do you have? Comparing two averages points to a t-test, categorical association to chi-square, prediction to regression, many groups to ANOVA — always reported with effect sizes and intervals, and swapped for robust or nonparametric versions when assumptions fail. But the tool is half the job: honest interpretation, attention to design and confounders, and comfort with uncertainty are what turn a test result into real understanding.

• The Statistical Mind
• Statistics for Everyday Life