What are effect sizes and why we need them? Larry Hedges, Northwestern University

Publication Date: March 5, 2007
Current as of:

Effect sizes are a type of quantitative representation of the magnitude of relations, differences, or comparisons that are in some way meaningful in the research design to which they are applied. Two classic examples of effect sizes include the difference in group means and a regression coefficient relating Y with X. However, statistical significance, which has certain universal appeal because many analyses compute the p-value, is not a replacement for effect size. In addition, there are faults in using statistical significance as an index of differences. For example, large studies frequently find significant effects, small studies frequently fail to find significant effects, and it is not obvious that all statistically reliable effects are substantively important.

The reason that statistical significance is not an index of effect size is because the distribution of the test statistic depends on two things: (a) an effect size component (defined by substantively relevant population parameters and (b) a design component (including sample size). Effect sizes are a way of describing substantively important relations among population parameters in a way that is independent of the research design. This makes effect sizes from different studies comparable to one another.