Introduction

You have three doses, three treatment groups, or three sampling sites. A t-test between every pair looks tempting, but it is the wrong move — each extra comparison inflates your chance of a false positive. The standard tools for three or more independent groups are one-way ANOVA and the Kruskal-Wallis H test.

This guide explains when to use which, what assumptions actually matter, and how to report the result so a reviewer cannot object.

What each test asks

Both tests start from the same null hypothesis: the groups are drawn from populations with the same central tendency. They differ in what they measure and what they assume about the data.

One-way ANOVA       ->  Are the group MEANS equal?
                        Tests variance between groups
                        against variance within groups.

Kruskal-Wallis H    ->  Are the group RANK distributions equal?
                        Non-parametric alternative to ANOVA.

If your data are roughly symmetric and you care about means, ANOVA is more powerful. If your data are skewed, bounded, ordinal, or have outliers, Kruskal-Wallis is the safer default.

The F-statistic in plain language

ANOVA compares two sources of variation:

F = variation between group means / variation inside the groups

A large F means the group means are spread out far more than the noise inside each group. A small F means the observed differences could easily be random scatter. The p-value comes from the F distribution with the appropriate degrees of freedom.

ANOVA itself does not tell you which groups differ — only that at least one group is different. That is where post-hoc tests come in.

Assumptions, stated plainly

One-way ANOVA

Observations independent within and between groups
Residuals approximately normal in each group — or large enough n for the central limit theorem
Homogeneity of variance: group variances should be roughly similar (use Welch ANOVA or a Brown-Forsythe adjustment if they are not)

Kruskal-Wallis H

Observations independent
Ordinal or continuous outcome
Groups have similar distribution shapes if you want to interpret a significant result as a difference in medians

As with the t-test, do not use a normality test as a gatekeeper. Decide from the nature of the measurement, sample size, and a visual check.

The decision path

1. Are the observations independent?
   No  -> use a repeated-measures model
         (repeated-measures ANOVA / Friedman test)
   Yes -> continue

2. Are there 3 or more groups?
   Yes -> continue
   No  -> use the two-group guide
          (t-test / Mann-Whitney)

3. n per group >= ~15-20 and data roughly symmetric
   with no heavy tails or strong skew?
   Yes -> One-way ANOVA
   No  -> continue

4. Data clearly skewed, bounded, ordinal, or with outliers?
   Yes -> Kruskal-Wallis H
   No  -> One-way ANOVA

When group variances differ but the data are otherwise normal-ish, prefer Welch's one-way ANOVA over classic ANOVA. It is the multiple-group equivalent of the Welch t-test.

Why you cannot just run multiple t-tests

Suppose you run three pairwise t-tests at alpha = 0.05. The probability of at least one false positive is not 0.05 — it is roughly:

1 - (1 - 0.05)^3 = 0.14

With six comparisons it rises to about 0.26. ANOVA and Kruskal-Wallis keep the family-wise error rate under control by testing all groups in one model, and post-hoc tests apply further corrections.

Post-hoc tests: where the real answer lives

A significant omnibus test only tells you "something is different." Use post-hoc comparisons to identify which pairs differ.

After ANOVA           ->  Tukey HSD (equal variances)
                           Games-Howell (unequal variances)

After Kruskal-Wallis  ->  Dunn's test with Bonferroni or
                           Benjamini-Hochberg correction

Report adjusted p-values from the post-hoc test, not the raw p-values from each pairwise comparison.

Report effect size, not just significance

A small p-value with a trivial effect is not impressive. Always report an effect measure:

ANOVA         ->  eta-squared (n2) or partial eta-squared
                  omega-squared (w2) for unbiased estimate

Kruskal-Wallis -> epsilon-squared (e2)

Pairwise      ->  mean difference + 95% CI (ANOVA)
                  median difference / common-language effect
                  size (Kruskal-Wallis)

A reviewer-proof sentence looks like this:

Cadmium concentration differed across the three sites (Kruskal-Wallis H = 12.4, df = 2, p = 0.002; epsilon-squared = 0.31). Post-hoc Dunn tests showed Site C was higher than both Site A (padj = 0.003) and Site B (padj = 0.041).

Common mistakes worth avoiding

Running t-tests on every pair after ANOVA is significant. Use a proper post-hoc test instead.
Forgetting equal-variance checks. If variances differ, Tukey HSD is anti-conservative. Use Games-Howell or Welch ANOVA.
Treating a non-significant omnibus test as proof of equivalence. It is not.
Reporting unadjusted pairwise p-values after a significant Kruskal-Wallis test. The comparisons were suggested by the data; the p-values need correction.
Running ANOVA on ordinal Likert data. Kruskal-Wallis is usually the better tool.

Repeated-measures note

If the same wells, animals, or subjects are measured under all conditions, independence is violated. Use:

Parametric     -> repeated-measures ANOVA
Non-parametric -> Friedman test

Labreador's Group Comparison module currently handles independent groups. For repeated measures, export your reduced values and use a dedicated repeated-measures tool.

Doing this in Labreador

The Group Comparison module runs the whole workflow automatically: it inspects distribution shape and variance homogeneity, selects ANOVA or Kruskal-Wallis, applies the appropriate post-hoc test with multiple-comparison correction, and reports effect sizes and adjusted p-values. Significance brackets are drawn directly on the plot. Everything runs locally in your browser, so raw data never leaves your device.

Related guides

Citation

Helczman, M. (2026). Labreador - Bioassay Analysis Platform (Version 1.6.0) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.21676846