One, two, or more?
The number of groups you're comparing determines which statistical test to use. T-tests are for two groups. ANOVA is for three or more. Use the wrong one and your analysis is invalid.
This is one of the simplest concepts in test selection, but getting it wrong is surprisingly common.
The question is straightforward: How many groups are you comparing?
Every comparison involves one, two, or more than two groups.
Comparing to a known value or theoretical standard
Comparing two treatments, conditions, or populations
Comparing multiple treatments, sites, or categories
Count your groups before choosing your test. It's that simple.
Here's how the number of groups maps to statistical tests.
| Groups | Continuous Outcome | Categorical Outcome |
|---|---|---|
| 1 group | One-sample t-test | One-sample proportion test |
| 2 groups (independent) | Independent t-test / Mann-Whitney | Chi-square / Fisher's exact |
| 2 groups (paired) | Paired t-test / Wilcoxon | McNemar's test |
| 3+ groups (independent) | One-way ANOVA / Kruskal-Wallis | Chi-square |
| 3+ groups (paired/repeated) | Repeated measures ANOVA / Friedman | Cochran's Q |
If you have 3+ groups and you run multiple t-tests (A vs B, A vs C, B vs C), you inflate your false positive rate. With 3 groups and 3 comparisons at α = 0.05, your actual false positive rate is about 14%, not 5%.
Solution: Use ANOVA first to test if ANY groups differ. Only then do post-hoc tests (Tukey, Bonferroni) to find WHICH groups differ.
Sometimes counting groups isn't as obvious as it seems.
"I'm comparing 3 time points in the same patients (baseline, 6 months, 12 months)."
Three groups, but paired (repeated measures). Same patients measured three times. Use repeated measures ANOVA, not regular ANOVA. The observations aren't independent.
"I have treatment vs control, but I also want to compare by sex (male vs female)."
This is a 2×2 factorial design. You have 4 cells (treatment-male, treatment-female, control-male, control-female), but the analysis depends on your question. Two-way ANOVA lets you test treatment effect, sex effect, and their interaction.
"I'm comparing ASA class I, II, III, IV, and V. Is that 5 groups?"
Yes, but consider the variable type. ASA class is ordinal, not nominal. You might use Kruskal-Wallis or ordinal regression rather than regular ANOVA. Also, you may need to collapse groups if some have very few patients (ASA V is often rare).
"I want to compare Hospital A vs Hospital B vs Hospital C vs Hospital D..."
That's 4+ groups. Use ANOVA. But consider whether "hospital" should be a fixed effect (you care about these specific hospitals) or random effect (hospitals are a sample from a larger population). The latter needs mixed models.
"My outcome is binary. Does the number of groups still matter?"
Yes. For 2 groups with binary outcome: chi-square or Fisher's exact. For 3+ groups: chi-square still works, or logistic regression with the grouping variable as a categorical predictor.
Count the levels of your grouping variable. If your predictor is "surgical approach" with values {open, laparoscopic, robotic}, that's 3 groups. If it's "treatment" with values {drug, placebo}, that's 2 groups.
For each scenario, identify how many groups are being compared.
How to count groups and select the appropriate test family. Combined with variable type and paired vs independent, you now have most of what you need to choose the right test.
Next lesson: The Decision Tree. We'll put it all together into a step-by-step flowchart for choosing the right statistical test.