Power is the probability your study can find what it's looking for — and most negative studies never had it
A single-center study compares wound complications after two closure techniques: 20 patients per arm. Complication rates are 25% vs 10% — a difference most surgeons would care about. The p-value is 0.41. The authors conclude the techniques are "equivalent."
They are not equivalent. With 20 patients per arm, this study had roughly a 1-in-4 chance of detecting that difference even if it were completely real. The study didn't show no difference — it was never capable of showing the difference. That capability has a name: statistical power.
Power is the probability that your study detects an effect of a given size, if that effect truly exists. It is set by four quantities locked in a trade-off: sample size, effect size, alpha, and outcome variability. Fix any three and the fourth is determined. This lesson teaches you the machine; the rest of the module teaches you how to operate it.
Every hypothesis test can fail in exactly two directions. Power lives in one of them.
When you test a hypothesis, reality is one of two things (there is a real difference, or there isn't) and your test says one of two things (significant, or not). That gives a 2×2 table with two correct cells and two errors:
| Real difference exists | No real difference | |
|---|---|---|
| Test says "significant" | True positive Probability = power (1 − β) | Type I error (false alarm) Probability = α |
| Test says "not significant" | Type II error (miss) Probability = β | True negative Probability = 1 − α |
Declaring a difference that isn't real. By convention we cap this at α = 0.05: if there is truly no difference, we accept a 5% chance of a false alarm. This is the "p < 0.05" threshold you already know — it was chosen to control this error.
Failing to detect a difference that is real. Power = 1 − β: the probability of catching a true effect. Convention aims for 80% power (β = 0.20) or 90% (β = 0.10). Notice the asymmetry we tolerate: a 5% false-alarm rate but a 20% miss rate. Missing real effects is treated as the cheaper error — which is worth remembering when the effect you'd miss is a real reduction in mortality.
Key framing: a p-value tells you about the false-alarm side. It tells you nothing about whether the study could have caught a real effect. That is power's job, and it is decided before the data arrive.
A hypothesis test is a smoke detector. α is how often it goes off when you're just making toast (false alarm). Power is how often it goes off during an actual fire (true detection). A detector that never false-alarms but sleeps through half of real fires is not a good detector — and a study with tightly controlled α but 30% power is not a good study. The 40-patient study in the intro was a smoke detector with a dead battery: silence told you nothing about fire.
Power is not a mystery number. It is the output of four inputs — and you control some of them.
More patients → less random noise in each group's estimate → easier to distinguish a real difference from chance. The only lever you can turn without changing the question, which is why "how many patients do I need?" is the standard phrasing of the whole problem.
Precision improves with the square root of n: to halve the noise you must quadruple the sample. This is why the last 10% of power is so expensive.
The size of the difference you are trying to detect. Big effects (mortality 30% vs 10%) are easy to see with small samples; small effects (30% vs 27%) need enormous ones. You don't control the true effect — but you must choose the effect size the study is designed to detect. That choice is the soul of the calculation, and it gets its own lesson next.
How strong the evidence must be before you call it significant. A stricter threshold (0.01 instead of 0.05) means fewer false alarms but requires more patients for the same power. Effectively fixed at 0.05 two-sided by convention — reviewers will notice if you quietly relax it.
One-sided vs two-sided: a one-sided test puts all 5% of the false-alarm budget in one direction, making significance easier to reach. Legitimate only when the opposite direction is truly of no interest — which is rare. A one-sided test appearing without justification is a red flag you already know from Module 1.
The noisier the outcome, the harder a signal is to see. For continuous outcomes this is the standard deviation (detecting a 5-minute difference in operative time is easy if SD = 10 min, hopeless if SD = 60). For binary outcomes the variability is set by the event rate itself — rates near 50% are the noisiest. You reduce this lever with better measurement and more homogeneous populations, not with more patients.
Lock any three of these and the fourth is fixed. This is why a power calculation is really a negotiation: given α = 0.05 and the effect I care about, how many patients buy me 80% power?
The intro study needed ~100 patients per arm, not 20. Its "negative" result was baked in before the first patient enrolled. When you read a negative study, your first question is not "was the p-value big?" but "what power did this study have to find an effect worth caring about?"
Four sentences you will hear in journal club — and what's wrong with each.
"p = 0.41, so there's no difference between the techniques."
Absence of evidence is not evidence of absence. A non-significant p-value means the study failed to detect a difference — which happens for real differences all the time when power is low. "No difference detected" and "no difference exists" are different claims; only an adequately powered study (or an equivalence trial designed for the purpose) can support the second one.
"Our result was significant, so the study was obviously well-powered."
Significance does not certify power. Underpowered studies do sometimes reach p < 0.05 — but disproportionately when the observed effect is inflated by chance. A significant result from a small study is more likely to be an overestimate of the true effect (the "winner's curse"), which is exactly why dramatic small-study findings shrink on replication.
"With α = 0.05, there's a 5% chance our significant finding is false."
Alpha is a false-alarm rate, not the probability your finding is false. α = 0.05 means: if there is truly no effect, 5% of studies would still cross the threshold. The chance that a given significant finding is false also depends on power and on how plausible the hypothesis was to begin with. Low-powered studies testing long-shot hypotheses produce significant results that are false far more than 5% of the time.
"We used a one-sided test because we expected the new technique to be better."
Expecting a direction doesn't justify ignoring the other one. A one-sided test assumes that harm in the other direction is either impossible or of no interest — almost never true when patients are involved. Since one-sided testing halves the evidence bar, it makes borderline results look significant. Unless the protocol justifies it up front, read a one-sided α as an admission that the two-sided result was p > 0.05.
When you hear "no significant difference," reflexively ask three questions: what effect size was the study powered to find, what power did it actually have, and does the confidence interval exclude effects a clinician would care about? Lesson 4 turns this into a full framework.
Type I vs Type II, the four levers, and reading negative studies.
Module 6 - Lesson 1 complete
Next lesson: Where Does Delta Come From? — the effect size is the one input someone has to choose, and that choice is where most sample size calculations quietly go wrong.