One fictional manuscript, every trick in the module — you're the reviewer
The (fictional) VELOCITY trial has landed in your journal club: a single-center RCT of drug-coated balloon (DCB) versus plain balloon angioplasty for femoropopliteal disease. Primary endpoint: 12-month primary patency. It concludes that DCB "provides no patency benefit."
Every number in this paper is arithmetically consistent — nothing is fabricated, nothing miscalculated. And the conclusion is still wrong. Your job is to find out why, using everything from Lessons 1–4: the delta audit, the calculator, attrition math, CI interpretation, and the post-hoc power fallacy.
This is the skill the module was building toward: reading a power paragraph the way you read an operative note — knowing what should be there, noticing what's missing, and never mistaking fluent prose for sound method.
Here is VELOCITY's sample size section, verbatim. Read it like a reviewer — then we'll take it apart.
Based on a retrospective series from our institution demonstrating 12-month primary patency of 55% with plain balloon angioplasty and 80% with DCB, we determined that 52 patients per arm would provide 80% power to detect this difference at a two-sided alpha of 0.05. Anticipating 15% loss to follow-up, we enrolled 124 patients (62 per arm) with 1:1 randomization.
Run 55% vs 80% through Lesson 3's calculator: ~52 per arm at 80% power. Correct. The attrition inflation is even done properly — dividing by 0.85 (52/0.85 = 61.2 → 62), not multiplying by 1.15. This is a competently executed calculation. Which is exactly why arithmetic-checking alone is never enough.
A 25-point absolute patency difference, taken from the authors' own retrospective series. Lesson 2's alarm bells should all be ringing: retrospective single-center comparisons are confounded by selection (who got DCB and why?), and effects that motivate trials are selected for being impressive. Published RCTs of this comparison report patency differences closer to 10–15 points. The trial is powered for an effect roughly twice what the literature supports.
Nowhere does the paper say what difference would matter. Would a 10-point patency improvement change practice? Most interventionalists would say yes — and detecting 65% vs 75% requires roughly 326 per arm, six times what VELOCITY enrolled. The study is a test of "is DCB spectacularly better?" quietly presented as a test of "is DCB better?"
Now the results and discussion — where design debts come due.
Follow-up was completed by 93 patients (46 plain balloon, 47 DCB; 25% attrition). Twelve-month primary patency was 68% with plain balloon versus 75% with DCB (difference 7 points, 95% CI −11 to +25; p = 0.45). Of nine prespecified secondary endpoints, freedom from target lesion revascularization favored DCB (p = 0.04).
We found no significant difference in primary patency. A post-hoc power analysis using the observed 7-point difference yielded 11% power, suggesting our study may have been underpowered for small effects; nevertheless, our findings indicate that DCB provides no meaningful patency benefit, while significantly reducing reintervention.
They planned for 15% loss and suffered 25%, leaving ~46 per arm — below even the 52 the optimistic calculation demanded. Against the realistic 10-point MCID, the completed study had roughly 18% power. This trial was over before the first balloon inflated.
The CI runs from −11 to +25 points: compatible with meaningful harm, nothing, or two and a half times the MCID in benefit. Lesson 4's framework calls this an uninformative negative — yet the discussion converts it to "no meaningful patency benefit." Note also the observed 7-point difference is close to the 10-point MCID; framed honestly, this is a signal awaiting an adequately powered trial, not a null result.
"Observed power of 11%" computed from the observed difference — the exact circular maneuver from Lesson 4. It restates p = 0.45 and adds nothing. The legitimate version would have been power against a prespecified MCID, disclosed as a limitation.
One significant result among nine secondaries (a ~37% chance of at least one false positive even if nothing works), in an underpowered trial where any "hit" is inflated by the winner's curse — promoted to the discussion's closing claim. The abstract will say "DCB reduces reintervention." Journal club should not.
Seven questions, in order. They take two minutes and catch nearly everything.
Applied to VELOCITY: items 4 and (half of) 5 pass; items 1, 2, 6, and 7 fail outright. A paper can be arithmetically flawless and still be a machine for laundering "we couldn't tell" into "there's no benefit." The checklist is how you catch it in real time, in journal club, with the authors' fluent prose working against you.
Eight questions on VELOCITY — and on the audit skills you'll reuse forever.
Module 6 - Sample Size and Power complete
Next time you're handed a "negative" study in journal club — or asked "how many patients do we need?" on rounds — you have the whole toolkit. Pair this module with Module 3 (Understanding Your Output) for interpretation and Module 1 (Reading a Paper) for the rest of the audit.