* Total Points: 0
Back to Lessons When ML Adds Value (and When It Doesn't) 0 pts Module 1 · Lesson 5
Introduction

When ML Adds Value (and When It Doesn't)

ML is not a magic upgrade. It is a different tool with different requirements.

The Setup

You have seen it in the literature: a study replaces logistic regression with a random forest and claims a "novel machine learning approach." The sample is 150 patients and 5 variables. The AUC improves from 0.76 to 0.78.

Was ML the right tool here? Probably not. But how do you know when it is?

Two proposals, two different answers

Consider two research proposals from the same vascular surgery department:

Proposal A: 45,000 VQI patients, 60 variables, predicting 1-year amputation after infrainguinal bypass. Complex variable interactions expected.
Proposal B: 120 single-center patients, 5 preoperative variables, testing whether a new anticoagulation protocol reduces graft thrombosis in an RCT.

Proposal A is a strong ML candidate. Proposal B should use traditional statistics. This lesson teaches you why.

The goal is not to use ML everywhere. The goal is to know when ML adds value and when it does not -- and to have the vocabulary to explain why.

When ML Adds Value

ML excels when the problem has specific characteristics that traditional statistics cannot handle well. Tap each card for details.

High-Dimensional Data
Many predictor variables relative to what traditional models can handle efficiently.
ML Wins
Why ML helps
Logistic regression with 60 variables becomes unwieldy -- you worry about multicollinearity, you need to pre-select variables, and interactions are difficult to specify manually. Tree-based ML methods (random forest, XGBoost) handle high-dimensional data naturally, automatically detecting interactions and handling correlated predictors.
The threshold
There is no magic number, but once you have more than ~15-20 candidate predictors and a large enough sample, ML methods start to show their advantage. Below that, logistic regression is usually just as good and more interpretable.
Vascular example: A VQI study with 45,000 infrainguinal bypass patients and 60 preoperative variables (demographics, comorbidities, anatomy, wound characteristics, hemodynamics, prior procedures). An XGBoost model can consider all variables and their interactions simultaneously -- something impractical with traditional regression.
Tap to expand
Complex, Nonlinear Relationships
When the relationship between predictors and outcome is not a straight line.
ML Wins
Why ML helps
Traditional regression assumes linear (or pre-specified) relationships. But clinical reality is messy: the effect of age on outcome might change direction at 75, the impact of diabetes might depend on whether the patient is also on dialysis, and three-way interactions between wound severity, ischemia grade, and infection may matter. ML methods find these patterns automatically.
The caveat
You need enough data for the model to learn these complex patterns. With a small sample, ML will find "interactions" that are actually noise -- this is overfitting. The complexity ML can reliably learn is bounded by your sample size.
Vascular example: Predicting wound healing in CLTI patients. The interaction between WIfI stage, revascularization completeness, and tissue oxygenation is highly nonlinear. A gradient boosting model can capture threshold effects (e.g., healing probability drops sharply below a certain perfusion level) without you having to specify the functional form in advance.
Tap to expand
Non-Tabular Data
Images, text, waveforms -- data that does not fit in a spreadsheet.
ML Required
Why ML is necessary
Traditional statistics works on structured, tabular data -- rows of patients and columns of variables. But CT angiograms, wound photographs, operative notes, and arterial waveforms are not tabular. Deep learning (CNNs for images, NLP models for text) can process these data types directly. There is no traditional statistics equivalent for learning from raw pixels or free text.
Vascular examples: CNN classifying wound severity from smartphone photos. NLP extracting conduit type from unstructured operative notes. Deep learning segmenting aortic anatomy from CT angiograms. These tasks are impossible with logistic regression -- ML is the only option.
Tap to expand
!

The "Just Use Logistic Regression" Heuristic

For structured tabular data with fewer than ~15 variables and a clear outcome, logistic regression is almost always sufficient. ML rarely provides meaningful improvement in this setting and adds complexity. Start simple. Only reach for ML when you have a specific reason it would help.

When Traditional Statistics Win

ML is not always better. Here are the scenarios where reaching for ML is the wrong call.

Small Sample Sizes
ML needs data to learn patterns. With small samples, it learns noise.
Stats Wins
Why ML struggles
ML models have many parameters to estimate. Without sufficient data, they memorize the training set instead of learning generalizable patterns -- this is overfitting. The events-per-variable rule that limits logistic regression (roughly 10-20 events per predictor) is even more constraining for ML, which typically needs more data to learn its more complex patterns.
Rules of thumb
For tabular data with ML, you generally want at least several hundred observations with a reasonable event rate. Deep learning (images, text) typically needs thousands. If your single-center study has 80-200 patients and a 10% event rate, logistic regression with careful variable selection is almost certainly the better choice.
Vascular example: A single-center study of 150 thoracoabdominal aortic aneurysm repairs with 12 postoperative deaths. A random forest with 30 variables will overfit this data badly. A logistic regression with 3-4 carefully chosen predictors is the right approach.
Tap to expand
Causal Questions
ML predicts outcomes. It does not tell you what causes them.
Stats Wins
The fundamental distinction
ML is designed for prediction -- "given these inputs, what is the likely output?" Traditional statistics (especially in the context of RCTs) is designed for causal inference -- "does this treatment cause a better outcome?" These are fundamentally different questions. An ML model can predict surgical outcomes beautifully without telling you which variables are causally related to the outcome.
When this matters
If your question is "Does drug A reduce graft thrombosis compared to drug B?" you need a designed experiment (RCT) and traditional statistical testing. An ML model trained on observational data might show that patients on drug A have better outcomes, but it cannot separate the treatment effect from confounding. For treatment effect questions, use the statistical tools designed for causal inference.
Vascular example: An RCT comparing rivaroxaban vs. aspirin for graft patency after infrainguinal bypass. The primary analysis is a log-rank test and Cox regression. ML has no role here -- the question is causal ("does rivaroxaban improve patency?"), not predictive ("which patients will have graft failure?").
Tap to expand
Interpretability Requirements
When stakeholders need to understand why the model made a decision.
Stats Wins
The transparency trade-off
Logistic regression gives you odds ratios -- intuitive, communicable, directly interpretable. "Each 1-cm increase in aneurysm diameter increases the odds of rupture by 40%." Try explaining what an XGBoost model learned to a patient, a hospital administrator, or a malpractice attorney. ML interpretability tools (SHAP, LIME) help but add complexity and are post-hoc approximations, not exact explanations.
When this tips the balance
Clinical guidelines, quality metrics, risk adjustment, shared decision-making tools, and medico-legal contexts all favor interpretable models. If the primary goal is understanding which variables drive an outcome (not just predicting the outcome), traditional regression is usually the better choice.
Vascular example: A hospital wants to understand the key drivers of 30-day readmission after vascular surgery to design targeted interventions. They need to know which factors matter and by how much. Logistic regression with odds ratios provides this directly. An ML model might predict readmission better, but does not clearly tell you where to intervene.
Tap to expand

The Decision Framework

Factor Favors ML Favors Traditional Stats
Sample size Thousands+ Hundreds or fewer
Variables Many (20+), complex interactions Few (5-10), known relationships
Question type Prediction Causal inference
Data type Images, text, mixed Structured tabular
Interpretability Less critical Essential

The "Start Simple" Principle

Even when ML is appropriate, always build a logistic regression baseline first. If your traditional model has an AUC of 0.82 and your random forest has 0.83, the complexity of ML is probably not worth it. ML should earn its place by providing a meaningful improvement, not just a statistically significant one. Many published surgical ML papers show trivial improvements over logistic regression -- and those improvements often disappear with external validation.

The best answer to "should I use ML?" is almost always "start with logistic regression and see if it's good enough." If it is, you are done. If it is not, now you have a baseline to beat.

Exercise: ML or Traditional Stats?

Read the research proposal. Decide: is ML the right tool, or should they use traditional statistics?

Proposal 1 of 4

Lesson Complete!

0
Total Points Earned
Proposals (0/4 correct) +0 pts
Lesson Completed +100 pts
✓

When ML Adds Value (and When It Doesn't)

Module 1 - Lesson 5 complete

Key Takeaways

  • ML excels with high-dimensional data: Many variables, complex interactions, large samples. This is where tree-based methods and neural networks outperform traditional regression.
  • ML is required for non-tabular data: Images, text, and waveforms cannot be analyzed with logistic regression. Deep learning is the only option for these data types.
  • Traditional stats wins with small samples: ML overfits small datasets. With fewer than a few hundred patients and limited events, logistic regression is more reliable.
  • Causal questions need traditional methods: ML predicts outcomes but does not establish causation. RCTs and designed experiments require traditional statistical testing.
  • Always start simple: Build a logistic regression baseline first. ML should earn its place by providing a meaningful improvement, not just a marginal one.

You now have a complete foundation in ML literacy. You know what ML is (Lesson 1), how it learns (Lesson 2), what it outputs (Lesson 3), what goes into it (Lesson 4), and when to use it (Lesson 5). You are ready to learn how to evaluate whether an ML model is actually any good.