How to formally compare survival curves instead of squinting at them
Two Kaplan-Meier curves are plotted on the same figure. One is higher than the other for most of the follow-up period. The presenter says the difference is "clearly significant."
Is it? You cannot tell from looking. You need a statistical test that accounts for censoring, sample size, and the timing of events across the entire follow-up period.
The log-rank test is the standard method for comparing two or more survival curves. It answers one question: are the survival experiences of these groups statistically different?
The intuition, without drowning in formulas.
At every time point where an event occurs, the test asks: given the number of patients at risk in each group, how many events would we expect in each group if there were truly no difference?
It then compares the total observed events to the total expected events across all time points. If the observed and expected numbers diverge substantially, the test is significant.
A chi-squared statistic with one degree of freedom (for two groups) and a corresponding p-value. The test does not produce a hazard ratio -- that requires Cox regression.
The log-rank test is not always the right choice. Know the alternatives.
Use when: Hazards are roughly proportional over time
Equal weight to all time points. Best for constant hazard ratios. The default in most surgical papers.
Use when: Early differences matter more
Gives more weight to earlier time points where more patients are at risk. Better when early events drive the clinical difference (e.g., perioperative mortality).
Use when: You want a compromise
Intermediate weighting between log-rank and Wilcoxon. Weights by the square root of the number at risk.
Use when: You need to adjust for a confounder
Performs the log-rank test within strata of a confounding variable, then combines. Cannot adjust for continuous variables -- use Cox for that.
If two survival curves cross, the standard log-rank test can return a non-significant p-value even when there is a real difference -- because the early advantage of one group is canceled out by the late advantage of the other.
In vascular surgery, this happens when comparing endovascular vs open approaches: endovascular often has better perioperative outcomes but worse long-term durability. The log-rank test averages these effects and may show "no difference."
The log-rank test tells you WHETHER the curves differ. It does not tell you HOW MUCH they differ. For that, you need the hazard ratio from Cox regression (Lesson 4).
Errors you will see in submitted manuscripts and published papers.
"The log-rank test showed HR = 0.72 (p = 0.03)."
The log-rank test does not produce a hazard ratio. It produces a chi-squared statistic and a p-value. If they report an HR, they ran Cox regression (or are confusing their tests). This is a surprisingly common error in methods sections.
"We compared 5 treatment groups using the log-rank test (p = 0.01)."
A significant omnibus log-rank test with 5 groups tells you at least two groups differ, but not which ones. You need pairwise comparisons with multiple testing correction (Bonferroni, Holm, etc.) to identify specific differences. Without correction, you are inflating your false positive rate.
"Log-rank was non-significant (p = 0.08), so the treatments are equivalent."
Non-significance does not prove equivalence. The study may simply be underpowered. A p-value of 0.08 with 50 patients per group means something very different than p = 0.08 with 5,000 per group. Always consider power and effect size.
"We used the log-rank test to compare outcomes after propensity score matching."
This may be incorrect. After propensity score matching, the standard errors from the log-rank test do not account for the matched design. You should use a stratified log-rank test (stratified by matched pair) or a paired analysis. Using the unstratified test after matching ignores the dependence structure you created.
When you see a log-rank p-value in a paper, ask three questions: (1) Do the curves cross? If yes, the test may be underpowered. (2) Is there a number-at-risk table? You need it to assess the data behind the test. (3) Did they also report the hazard ratio? If not, you have significance without magnitude.
Choose the best interpretation for each scenario.
Module 4 - Lesson 3 complete
Next up: Cox Proportional Hazards -- the workhorse model that gives you adjusted hazard ratios and handles multiple covariates.