Introduction

Almost every bench experiment ends with the same question: are these two groups different? Treated vs control, exposed vs reference site, patient vs healthy donor. And almost every time, the same doubt follows — should this be a t-test or a Mann-Whitney U test?

The honest answer is that the choice depends on three things: the shape of your data, your sample size, and what hypothesis you actually want to test. This guide gives you a decision path you can defend in review.

What each test actually tests

They are not two flavours of the same question.

Student / Welch t-test   ->  difference in MEANS
Mann-Whitney U test      ->  probability that a value from group A
                             exceeds a value from group B
                             (stochastic dominance; often summarised
                             as a shift in medians)

That distinction matters. If your outcome is strongly skewed, the mean may not be the quantity you care about at all — a median or a dominance statement can be the more meaningful claim.

Assumptions, stated plainly

Welch t-test

Observations independent
Residuals approximately normal — or n large enough for the central limit theorem
Unequal variances allowed (this is why Welch is the sane default, not Student)

Mann-Whitney U

Observations independent
Ordinal or continuous outcome
Sensitive to differences in distribution shape, not just location — with very unequal spread, a "significant" result does not automatically mean "different medians"

The decision path

1. Are the observations independent?
   No  -> use a paired test (paired t-test / Wilcoxon signed-rank)
   Yes -> continue

2. n per group >= ~15-20 and no extreme outliers?
   Yes -> Welch t-test is robust; use it
   No  -> continue

3. Data clearly skewed, bounded, or with heavy tails
   (concentrations, counts, ratios, scores)?
   Yes -> Mann-Whitney U
   No  -> Welch t-test

4. Tiny n (3-5 per group)?
   -> No test resolves shape. Report descriptives + effect size,
      treat any p-value as exploratory.

Stop using normality tests as gatekeepers

Running Shapiro-Wilk and switching tests based on p > 0.05 is a widespread habit and a weak one:

With small n, Shapiro-Wilk has almost no power — it "passes" nearly everything, including clearly non-normal data.
With large n, it flags trivial, harmless deviations — exactly where the t-test is most robust.
The pre-test itself inflates your overall error rate.

Better practice: decide from the nature of the measurement and a quick look at the distribution (histogram, QQ plot, outlier check), then pre-register that choice. Optical density values, cell counts, trace-metal concentrations and index scores each have known typical shapes — use that knowledge instead of an underpowered test.

Report the effect, not only the p-value

A p-value tells you how surprising the data are under the null hypothesis. It does not tell you how large the difference is, and with n = 6 it is a noisy number.

Always report alongside it:

Welch t-test        ->  mean difference + 95% CI, Cohen's d
Mann-Whitney U      ->  median difference (Hodges-Lehmann shift),
                        rank-biserial correlation or common-language
                        effect size
Both                ->  n per group, exact p, test name and tails

A reviewer-proof sentence looks like this:

Exposed fish showed higher hepatic Cd than reference fish (median 0.42 vs 0.19 mg/kg; Mann-Whitney U, n = 10 per group, p = 0.004; rank-biserial r = 0.72).

Common mistakes worth avoiding

Multiple pairwise t-tests across three or more groups. Use ANOVA or Kruskal-Wallis with a post-hoc correction instead of a stack of two-group tests.
Treating technical replicates as n. Duplicate wells from the same sample are not independent biological observations; average them first.
Switching tests until one is significant. Pick the test before you look at p.
Reporting "p = 0.000". Report p < 0.001.
Assuming non-significant means equivalent. Absence of evidence is not evidence of equivalence — for that you need an equivalence test.

Doing this in Labreador

The Group Comparison module handles the mechanics for you: it evaluates distribution shape, selects an appropriate parametric or non-parametric test, applies post-hoc corrections for more than two groups, and reports effect sizes with confidence intervals — not just a bare p-value. Significance brackets are drawn directly on the plots, and everything runs locally in your browser, so no data leaves your device.

Related guides

Citation

Helczman, M. (2026). Labreador - Bioassay Analysis Platform (Version 1.6.0) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.21676846