* Total Points: 0
Back to Lessons Model Building Without Overfitting 0 pts Module 5 · Lesson 6
Introduction

Model Building Without Overfitting

Why your 15-variable model is garbage

The Temptation

You have a dataset. You have a research question. And you have a list of every variable you collected: age, sex, BMI, diabetes, smoking, eGFR, ejection fraction, ASA class, frailty index, prior surgery, insurance status. Why not put them all in the model and let the regression sort out what matters?

Because the model does not know the difference between signal and noise. Give it enough freedom and it will happily fit the random quirks of your particular 200 patients -- quirks that will not exist in the next 200. That is overfitting: the model memorizes the noise in your sample and then fails on new patients.

Overfitting is the single most common modeling sin in the surgical literature. The fix is not a fancier algorithm -- it is discipline: fewer predictors, chosen in advance, and honest validation before you believe a single number the model reports.

Events Per Variable

The first sanity check for any logistic or Cox model: do you have enough events to support the number of predictors?

Count Events, Not Patients

For logistic regression and Cox regression, the quantity that limits model complexity is the number of events -- the rarer of the two outcomes -- not the total sample size. A study of 1,000 patients with only 20 deaths has the statistical power of a study with 20 events, no matter how impressive the headline N looks.

The traditional rule of thumb: at least 10 events per candidate predictor (EPV ≥ 10). Below this, coefficients become unstable, confidence intervals are unreliable, and apparent performance is wildly optimistic.

A Predictor Is Not Always One Degree of Freedom

"Number of predictors" really means degrees of freedom estimated, and some predictors cost more than one:

Continuous (linear): 1 degree of freedom.

Categorical with k levels: k − 1 degrees of freedom. A 4-level ASA class costs 3.

Spline / non-linear term: several degrees of freedom (often 3-4) for a single variable.

Interaction term: an additional degree of freedom (more if either term is categorical).

Count degrees of freedom, not the number of lines in your variable list.

StudyPatientsEventsPredictors (df)EPV
Overfit example20030122.5
Max defensible20030310.0
Comfortable20030215.0
EPV = 30 / 12 = 2.5
200 patients, 30 complications, 12 predictors. Far below the EPV ≥ 10 threshold -- this model is garbage. The coefficients reflect noise specific to these 30 patients.
Max defensible predictors = events / 10 = 30 / 10 = 3. To fit 12 predictors honestly you would need roughly 120 events, not 30.

For linear regression, the analog uses the total sample size: roughly 10-15 observations per predictor. With 60 patients you can defensibly fit about 4-6 predictors in a linear model, not because of events (there are none) but because each parameter still needs data to estimate it stably.

Selection and Validation

How you pick variables and how you measure performance determine whether the model means anything.

Why Stepwise and p<0.20 Screening Are Bad

Stepwise selection (forward, backward, or "screen univariably at p<0.20, then enter the survivors") lets the data choose the model. This is seductive and wrong.

It inflates type-I error (you test the same data many times), produces unstable variable sets (a few resampled patients change which variables "make it"), biases coefficients away from zero, and yields optimistic performance estimates. The resulting "independent predictors" are partly artifacts of your sample.

Prefer A Priori Clinical Selection

Specify the predictors before looking at outcomes, based on clinical knowledge and prior literature. Include known confounders regardless of their univariable p-value.

A pre-specified model that respects the EPV budget is more reproducible and more honest than any data-driven search. If you must reduce variables, do it on clinical grounds or with penalization, not p-values.

Penalized Regression (LASSO / Ridge)

A principled alternative to stepwise selection. Penalized methods shrink coefficients toward zero to control overfitting.

LASSO can shrink some coefficients exactly to zero (a built-in, less unstable form of selection); ridge shrinks all coefficients but keeps every variable. Both trade a little bias for a large reduction in variance, which usually improves performance on new patients. The penalty strength is tuned by cross-validation.

The Optimism Problem

Performance measured on the same data used to build the model -- the apparent performance -- is systematically too good. The model was fit to those exact patients, so it scores itself on its own homework.

A C-statistic of 0.89 or an R² of 0.7 on the training data proves nothing about new patients. You must estimate optimism using internal validation (bootstrap or cross-validation) or, better, external validation on a separate cohort. Bootstrap-based shrinkage and recalibration pull the optimistic estimates back toward reality before you report them.

A high C-statistic or R² on the training sample is the most over-reported, least meaningful number in prediction papers. Always ask: optimism-corrected or externally validated? If neither, the number is apparent performance and should be treated with suspicion.

Overfitting Pitfalls

Mistakes that produce publishable but unreproducible models.

"Our multivariable model adjusted for 15 covariates." The study had 30 events.

EPV = 30 / 15 = 2.0. This is not multivariable adjustment -- it is noise fitting. The coefficients are unstable, the confidence intervals are misleadingly narrow, and the "adjusted" estimates will not replicate. The maximum defensible number of predictors here is 3 (events / 10).

"We used stepwise selection to identify the independent predictors of complication."

Stepwise selection does not identify independent predictors. It identifies the variables that happened to look good in this sample. The procedure inflates type-I error, biases coefficients, and is unstable -- resample the data and a different set of "independent predictors" emerges. Pre-specified, clinically chosen variables or penalized regression are the defensible alternatives.

"The model showed excellent discrimination (AUC 0.84)." No validation is mentioned.

That is apparent AUC. Reported without internal or external validation, it is the model grading itself on the data it was built from -- guaranteed to be optimistic. The optimism-corrected AUC (via bootstrap) is often substantially lower. Without it, 0.84 means very little.

"We built a prediction model and confirmed its accuracy on the same dataset."

Building and validating on the same data is not validation. It cannot detect overfitting because the model has already seen every patient. Honest options: split-sample (weaker), cross-validation, bootstrap optimism correction, or -- the gold standard -- an external cohort the model never touched during development.

Dichotomizing a continuous predictor "to simplify" (e.g., age > 65 vs ≤ 65) throws away information and statistical power. A 64-year-old and a 90-year-old become identical; the model loses the gradient of risk. It rarely simplifies anything real and usually weakens the model. Keep continuous variables continuous -- and model non-linearity with splines if needed.

Exercise: Model Building and Overfitting

Evaluate model complexity, selection, and validation.

Question 1 of 8

Module Complete!

0
Total Points Earned
Exercise (0/8 correct) +0 pts
Lesson Completed +100 pts

Model Building Without Overfitting

Module 5 - Lesson 6 complete

Key Takeaways

  • Count events, not patients: For logistic and Cox models, EPV ≥ 10. Max predictors = events / 10.
  • Predictors cost degrees of freedom: A k-level category costs k−1; a spline costs several. Budget accordingly.
  • Avoid stepwise and p<0.20 screening: Data-driven selection inflates type-I error and is unstable. Pre-specify variables.
  • Penalized regression (LASSO/ridge): A principled way to control overfitting when you have many candidate predictors.
  • Apparent performance is optimistic: A high training C-statistic or R² proves nothing. Demand internal or external validation.
  • Do not dichotomize continuous predictors: It discards information and power. Keep them continuous, model non-linearity with splines.

You have completed Module 5: Regression Without Crying. You can now build, critique, and defend regression models -- choosing the right model, interpreting coefficients honestly, and recognizing overfitting before it embarrasses you in peer review.