How to tell whether your regression is describing reality -- or quietly lying to you
Regression software always returns coefficients, standard errors, and p-values. It does not return a warning that says "the assumptions behind these numbers are violated." A model can produce a clean-looking table with three significant predictors while every one of those estimates is biased, the confidence intervals are too narrow, and a single patient is driving the headline result.
Diagnostics are the only way to catch this. They are how you check whether the machinery that produced your hazard ratio or odds ratio was actually allowed to run.
Reviewers and co-authors almost never ask whether you checked the residuals, the VIFs, or the influential points. That is exactly why you should -- the difference between a defensible model and a misleading one usually lives in the diagnostics nobody reads.
The classic mnemonic is LINE: Linearity, Independence, Normality of residuals, and Equal variance. Each one has a specific diagnostic and a specific fix.
The relationship between each continuous predictor and the outcome must actually be a straight line. If creatinine raises risk steeply above a threshold but is flat below it, forcing a single linear slope through it will distort the estimate and may hide the real effect entirely.
How to check: Plot residuals versus fitted values. A clear curve (U-shape or arch) signals non-linearity. Fix it with a transformation (log), a polynomial term, or restricted cubic splines rather than forcing a straight line.
Each observation must contribute independent information. Surgical data routinely violates this: multiple lesions per patient, multiple joints per patient, repeated measures over time, or patients clustered within surgeons or centers. Treating correlated observations as independent fabricates information you do not have.
How to check: This is a design question more than a plot -- ask whether any unit appears more than once or shares a cluster. When it does, use mixed-effects models (random intercepts) or GEE with cluster-robust standard errors.
The spread of the residuals should be roughly constant across the range of fitted values. If the residuals fan out as the predicted value grows -- larger errors for larger outcomes -- the variance is not constant, and the standard errors (and therefore the p-values and CIs) are wrong.
How to check: Look at the residual-versus-fitted plot for a funnel or cone shape. Fix it with robust (heteroscedasticity-consistent) standard errors, a variance-stabilizing transformation, or weighted least squares.
It is the residuals that should be approximately normal -- not the raw outcome and not the raw predictors. This is the single most misunderstood assumption in the clinical literature. With large samples it barely matters because of the central limit theorem; it matters mainly for inference in small samples.
How to check: A Q-Q plot of the residuals -- points should fall along the diagonal line. Heavy tails or strong skew warrant a transformation or a more robust method. Do not run a normality test on the raw variable and conclude anything about your model.
Write it on the wall: normality applies to the residuals, not to the data. A skewed outcome with normal residuals is fine; a normal-looking outcome with curved residuals is not.
Even when the LINE assumptions hold, two structural problems can quietly wreck your coefficients.
When two or more predictors are strongly correlated (think BMI and weight, or systolic and mean arterial pressure), the model cannot tell their effects apart. The result is inflated standard errors, coefficients that swing wildly when you add or drop a variable, and signs that flip in ways that make no clinical sense. Crucially, collinearity does not bias predictions -- the model still predicts the outcome fine -- but it destroys your ability to interpret any individual coefficient.
How to check: The Variance Inflation Factor (VIF). A VIF above 5 is a concern and above 10 is usually a serious problem. Fix it by dropping or combining redundant predictors, or by using a composite/score. Do not interpret a single collinear coefficient in isolation.
Three related but distinct ideas: leverage is an observation with an unusual predictor pattern (far out on the x-axis); an outlier is an observation with an unusual outcome (a large residual); influence is the dangerous combination -- a point that is both, so that removing it single-handedly moves a coefficient or a p-value. In small surgical datasets one such patient can manufacture an entire "finding."
How to check: Cook's distance flags points with large overall influence; dfbeta shows how much a specific coefficient changes when each point is removed. Always re-run the model without the flagged points to see whether the conclusion survives.
Logistic regression does not assume normality or homoscedasticity of residuals -- those concepts do not apply to a binary outcome.
It does still require independence of observations and the absence of damaging multicollinearity, and it adds its own version of the linearity assumption: linearity of the logit. For each continuous predictor, the log-odds of the outcome must change linearly across its range. Check this with the Box-Tidwell test or by modeling the predictor with splines and inspecting the shape.
So multicollinearity and influential-point diagnostics carry over to logistic (and Cox) models; the residual-normality and constant-variance checks do not.
Each of these passes peer review regularly. Each one is a model lying to you.
"We confirmed normality by running a Shapiro-Wilk test on operative time and it was non-significant, so linear regression was appropriate."
Wrong target. Linear regression assumes the residuals are approximately normal, not the raw outcome. Testing the distribution of operative time itself tells you nothing about whether the model's errors are normal. A skewed outcome can produce perfectly normal residuals once predictors are accounted for, and a symmetric outcome can produce badly non-normal residuals. Check a Q-Q plot of the residuals instead.
"We analyzed 480 lesions from 150 patients with ordinary logistic regression and found a highly significant predictor (p = 0.003)."
Pseudoreplication. Multiple lesions from the same patient are not independent observations. Treating 480 correlated lesions as 480 independent data points inflates the effective sample size, shrinks the standard errors, and produces confidence intervals that are far too narrow and p-values that are far too small. The fix is a mixed-effects model with a random intercept for patient, or GEE with cluster-robust standard errors.
"After adjustment, higher BMI was protective (OR 0.82, p = 0.04). We hypothesize an obesity paradox." (The model also included weight and body surface area.)
Collinearity, not biology. BMI, weight, and body surface area are mathematically tangled together. With VIFs well above 10, the model cannot separate their effects, and coefficients routinely flip sign and become "significant" in directions that are artifacts of the collinearity. Before invoking a clinical paradox, check the VIFs, drop the redundant predictors, and see whether the sign and significance survive.
"In our series of 28 patients, prolonged ischemia time independently predicted failure (p = 0.04)."
One patient may be the whole result. In a dataset this small, a single influential observation -- a patient with both an extreme ischemia time and an unusual outcome -- can produce a "significant" coefficient on its own. Cook's distance and dfbeta will flag it; re-running the model without that patient often makes the effect vanish. A finding that depends on one data point is not a finding.
When you review a regression: confirm normality was judged on residuals (not raw data), confirm clustered data was modeled as clustered, check the VIFs before believing any single coefficient, and demand a sensitivity analysis with influential points removed. If any of these is missing, the estimates are suspect.
Identify the violated assumption, the right diagnostic, and the correct interpretation.
Module 5 - Lesson 5 complete
Next lesson: Model Building Without Overfitting -- why your 15-variable model is garbage, what events-per-variable really buys you, and how to build a model that survives outside your own dataset.