Tukey HSD
Every pairwise comparison after an ANOVA, with adjusted p-values and confidence intervals.
df 2, 27
k = 3 groups, 27 error df
Pooled within-group variance
| Pair | Difference | q | p | 95% interval | Verdict |
|---|---|---|---|---|---|
| A − B | 0.371 | 1.882 | 0.3909 | [-0.32, 1.062] | No difference found |
| A − C | -0.494 | 2.506 | 0.1980 | [-1.185, 0.197] | No difference found |
| B − C | -0.865 | 4.388 | 0.0120 | [-1.556, -0.174] | Differ |
Group means
An ANOVA tells you some group differs, not which. Tukey’s HSD compares every pair and keeps the overall false-positive rate at α across all of them at once. With three groups of ten, a difference of 0.865 gives q = 4.388 and p = 0.012.
Why you cannot just run t-tests
With three groups there are three pairs; with five there are ten. If each pair is tested at α = 0.05 the chance of at least one false positive climbs with the number of pairs — for ten independent comparisons it is about 40%, not 5%. Running t-tests pair by pair therefore finds differences that are not there, and the more groups you have the worse it gets.
Tukey’s test fixes this by changing the reference distribution rather than the significance level. Instead of asking how big a single difference would be by chance, it asks how big the largest difference among k groups would be — which is a harder threshold to clear, and exactly the right one when you are looking at every pair.
The studentized range
That largest-difference distribution is the studentized range, written q. It is the range of k sample means divided by their common standard error, and it has no closed form: the critical values in every textbook appendix come from numerical integration. The test statistic for a pair is their difference divided by the same standard error, compared against that distribution rather than against t.
compared against the studentized range on k groups and N − k error degrees of freedom
- 1 Run the ANOVA first. Tukey rests on its pooled error term. If the overall F is not significant, there is no established difference for the pairs to locate.
- 2 Take the MSE and its degrees of freedom. For three groups of ten, MSE = 0.3886 on 27 degrees of freedom.
- 3 Work out the standard error for each pair. √(MSE ÷ 2 × (1/10 + 1/10)) = √(0.3886 ÷ 10) = 0.1971.
- 4 Divide each difference by it. A difference of 0.865 gives q = 0.865 ÷ 0.1971 = 4.388.
- 5 Compare against the critical q. For k = 3 and 27 degrees of freedom, q₀.₀₅ = 3.506. Since 4.388 exceeds it, that pair differs — p = 0.012.
Critical values of q at α = 0.05
Rows are error degrees of freedom, columns the number of groups. Computed from the studentized range distribution.
| df | k = 2 | k = 3 | k = 4 | k = 5 | k = 6 |
|---|---|---|---|---|---|
| 5 | 3.635 | 4.602 | 5.218 | 5.673 | 6.033 |
| 10 | 3.151 | 3.877 | 4.327 | 4.654 | 4.912 |
| 15 | 3.014 | 3.673 | 4.076 | 4.367 | 4.595 |
| 20 | 2.950 | 3.578 | 3.958 | 4.232 | 4.445 |
| 30 | 2.888 | 3.486 | 3.845 | 4.102 | 4.301 |
| 60 | 2.829 | 3.399 | 3.737 | 3.977 | 4.163 |
| ∞ | 2.772 | 3.314 | 3.633 | 3.858 | 4.030 |
Unequal group sizes, and what the intervals mean
When the groups are not all the same size the test becomes Tukey–Kramer, which uses each pair’s own two sample sizes in the standard error. The consequence is practical: a pair involving a small group gets a wider interval and needs a larger difference to clear the same threshold. The method reduces exactly to plain Tukey when the sizes match, so there is nothing to choose between them — the tool applies whichever the data calls for.
The confidence intervals are the more useful output. A 95% Tukey interval covers every pairwise difference simultaneously with 95% confidence, which is a stronger and more honest claim than a set of separate 95% intervals. An interval that excludes zero corresponds to a significant pair, but it also tells you how big the difference plausibly is — and a pair that is significant with an interval of [0.17, 1.56] is a much weaker finding than one with [4.9, 12.6].
Two assumptions carry over from the ANOVA: the groups should be roughly normal and have similar spreads, since they share a pooled MSE. Levene’s test checks the second. If the spreads clearly differ, Games–Howell is the usual replacement — it does not pool, and is not what this page computes.