The big idea
🧾
Publication is a filter. Replication is the test.
"Peer-reviewed" settles less than it is used to settle in argument.
Scientific Thinking · Lesson 4
Peer Review and Replication
beginner⏱ 17 min · 7 cards
Compare
What each does
Peer review
Catches weak method
Tap.
tap for example ↻
Replication
Tests the finding
Tap.
tap for example ↻
Reviewers rarely see raw data and cannot detect competent fabrication.
Try it
Weight the Evidence
“A meta-analysis pooling twenty independent randomized trials.”
How much weight does each source carry?
Rank sources by how much they should move you.
Key term
The file drawer
Why does the literature overstate effects?
tap to reveal ↻
Publication bias
Positive novel results get submitted and accepted; null results sit unpublished. Twenty groups test a useless treatment, one gets a fluke, and that is the paper you read.
Nuance
2015
Think like a Maester
When psychology repeated 100 studies, roughly a third to a half replicated. The field responded with pre-registration and data sharing — self-correction running in public.
Quick check
When is a truncated y-axis legitimate?
You did it
Nice work
Mark this lesson complete to track your progress.