Multiple Comparisons Online: Post-Hoc Tests After ANOVA
Why Multiple Comparisons Are Needed
A significant ANOVA only shows that at least one group mean differs, not which specific groups differ. Multiple comparisons (post-hoc tests) compare all pairs of groups while controlling the overall familywise error rate (FWER), avoiding the false-positive inflation caused by running many separate t-tests. The tool uses the ANOVA within-group mean square and degrees of freedom to give a confidence interval and significance flag for the mean difference of each group pair.
How to Choose a Common Method
Tukey HSD suits all pairwise comparisons when group sizes are similar and is the default recommendation, while Dunnett compares several treatment groups against a single control group. Bonferroni multiplies each t-test p-value by the number of comparisons; it is the most conservative and works in any setting, whereas LSD applies no correction and is the most sensitive but offers the weakest error control. Scheffe suits post-hoc comparisons of arbitrary linear combinations. The tool computes automatically from your data and choice.
How to Interpret the Results
The tool outputs the pairwise mean difference, standard error, adjusted p-value and 95% confidence interval, and summarizes the grouping with letter notation (groups sharing a letter are not significantly different). A confidence interval containing 0 or a p-value >= 0.05 indicates no significant difference for that pair. Also look at the absolute mean difference to judge whether the difference is practically or commercially meaningful, not merely statistically significant. Larger groups give higher power and narrower intervals; a mean-difference plot helps communicate the conclusions.
How to Use It (Step by Step)
Log in, enter the raw data for each group or paste group means, sample sizes and the within-group mean square (MS error) from your ANOVA output, choose the multiple-comparison method, and the tool returns the full comparison matrix and grouping letters. Confirm that the overall ANOVA is significant before proceeding, since post-hoc tests are usually meaningless otherwise (unless specific comparisons were pre-specified in the study design). With many groups, Bonferroni becomes conservative and Tukey is often preferable.