Calcular e apresentar tamanhos do efeito em trabalhos científicos (3): Guia para reportar os tamanhos do efeito para análises de regressão e ANOVAs

Helena Maria Amaral Espirito Santo; Fernanda Daniel

doi:10.31211/rpics.2018.4.1.72

Authors

Helena Maria Amaral Espirito Santo Centro de Investigação Interdisciplinar Psicossocial, Instituto Superior Miguel Torga; Centro de Investigação do Núcleo de Estudos e Intervenção Cognitivo-Comportamental Universidade de Coimbra, Portugal https://orcid.org/0000-0003-2625-3754
Fernanda Daniel Centro de Investigação Interdisciplinar Psicossocial, Instituto Superior Miguel Torga, Coimbra, Portugal

DOI:

https://doi.org/10.31211/rpics.2018.4.1.72

Keywords:

ANOVA, Regression analysis, Effect size, p-value

Abstract

In the first issue of the Portuguese Journal of Behavioral and Social Research, the importance of calculating, indicating, and interpreting the effect sizes for the differences of means of two groups (d family of effect sizes) was reviewed. Effect sizes are standard metrics that allows the comparison of the results of statistical analyzes of different studies. Effect sizes also report on the impact of a factor on the variable under investigation and the association between variables. After reviewing the effect sizes for the mean differences between two groups (Espirito-Santo & Daniel, 2015) and most of the r family (Espirito-Santo & Daniel, 2017), the review of effect sizes for analysis of variance was lacking. Analysis of variance can be understood as an extension of the d family to more than two groups (ANOVA) or as an r subfamily in which the proportion of variability is attributable to one or more factors. In the r subfamily reviewed in this study, we analyse the change in the dependent variable that results from one or more independent variables. This analysis is focused on general linear models, including regression models and ANOVA. This article provides the formulas for calculating the most common effect sizes by reviewing the basic concepts of the statistics and providing illustrative examples computed in the Statistical Package for the Social Sciences (SPSS). The guidelines for the interpretation of effect sizes are also presented, as well as the cautions in their use. Also, the article is accompanied by an Excel spreadsheet to facilitate and expedite the calculations for interested readers.

Downloads

Download data is not yet available.

References

American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association. [Google Scholar]

Bakeman, R. (2005). Recommended effect size statistics for repeated measures designs. Behavior Research Methods, 37(3), 379-384. [Google Scholar]

Berben, L., Sereika, S. M., & Engberg, S. (2012). Effect size estimation: methods and examples. International Journal of Nursing Studies, 49(8), 1039-1047. [Google Scholar] [CrossRef]

Bewick, V., Cheek, L., & Ball, J. (2003). Statistics review 7: Correlation and regression. Critical Care, 7(6), 451-459. [Google Scholar] [CrossRef]

Bezeau, S., & Graves, R. (2001). Statistical power and effect sizes of clinical neuropsychology research. Journal of Clinical and Experimental Neuropsychology (Neuropsychology, Development and Cognition: Section A), 23(3), 399-406. [Google Scholar]

Carroll, R. M., & Nordholm, L. A. (1975). Sampling characteristics of Kelley“s ε2 and Hays” ω2. Educational and Psychological Measurement, 35(3), 541-554. [Google Scholar] [CrossRef]

Cohen, J. (1973). Eta-squared and partial eta-squared in fixed factor ANOVA designs. Educational and Psychological Measurement, 33, 107-112. [Google Scholar] [CrossRef]

Cohen, J. (1992a). A power primer. Psychological Bulletin, 112(1), 155-159. [Google Scholar] [CrossRef]

Cohen, J. (1992b). Statistical power analysis. Current Directions in Psychological Science, 1(3), 98-101. [Google Scholar] [CrossRef]

Cumming, G. (2012). Understanding the new statistics. New York: Routledge. [Google Scholar]

Ellis, P. D. (2010). The essential guide to effect sizes. Cambridge: Cambridge University Press. [Google Scholar]

Erceg-Hurn, D. M., & Mirosevich, V. M. (2008). Modern robust statistical methods: An easy way to maximize the accuracy and power of your research. The American Psychologist, 63(7), 591-601. [Google Scholar] [CrossRef]

Espirito-Santo, H., & Daniel, F. B. (2015). Calcular e apresentar tamanhos do efeito em trabalhos científicos (1): As limitações do p < 0,05 na análise de diferenças de médias de dois grupos [Calculating and reporting effect sizes on scientific papers (1): p < 0.05 limitations in the analysis of mean differences of two groups]. Revista Portuguesa de Investigação Comportamental e Social, 1(1), 3-16. [Google Scholar] [CrossRef]

Espirito-Santo, H., & Daniel, F. (2017). Calcular e apresentar tamanhos do efeito em trabalhos científicos (2): Guia para reportar a força das relações [Calculating and reporting effect sizes on scientific papers (2): Guide to report the strength of relationships]. Revista Portuguesa de Investigação Comportamental e Social, 3(1), 53-64. [Google Scholar] [CrossRef]

Ferguson, C. J. (2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology: Research and Practice, 40(5), 532-538. [Google Scholar] [CrossRef]

Field, A. (2005). Effect sizes. [PDF]

Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R. London: Sage. [Google Scholar]

Fisher, R. A. (1925). Statistical methods for research workers. Edinburgh: Oliver & Boyd. [Google Scholar]

Fritz, C. O., Morris, P. E., & Richler, J. J. (2012). Effect size estimates: Current use, calculations, and interpretation. Journal of Experimental Psychology: General, 141(1), 2-18. [Google Scholar] [CrossRef]

Glass, G. V., & Hakstian, A. R. (1969). Measures of association in comparative experiments: Their development and interpretation. American Educational Research Journal, 6(3), 403-414. [Google Scholar] [CrossRef]

Haase, R. F., Waechter, D. M., & Solomon, G. S. (1982). How significant is a significant difference? Average effect size of research in counseling psychology. Journal of Counseling Psychology, 29(1), 58-65. [Google Scholar] [CrossRef]

Hair, J., Black, B., Babin, B., & Anderson, R. (2009). Multivariate data analysis (7th ed.). Upper Saddle River: Pearson Higher Ed. [Google Scholar]

Hays, W. L. (1963). Statistics for psychologists. New York: Holt, Rinehart and Winston. [Google Scholar]

Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational and Behavioral Statistics, 6(2), 107-128. [Google Scholar] [CrossRef]

Herzberg, P. A. (1969). The parameters of cross-validation. Richmond, VA: William Byrd Press. [Google Scholar]

Ialongo, C. (2016). Understanding the effect size and its measures. Biochemia Medica, 26(2), 150-163. [Google Scholar] [CrossRef]

Kelley, K. (2007). Confidence intervals for standardized effect sizes: Theory, application, and implementation. Journal of Statistical Software, 20(8), 1-24. [Google Scholar] [CrossRef]

Kelley, T. L. (1935). An unbiased correlation ratio measure. Proceedings of the National Academy of Sciences of the United States of America, 21(9), 554-559. [Google Scholar] [PMC]

Kennedy, J. J. (1970). The eta coefficient in complex ANOVA designs. Educational and Psychological Measurement, 30(4), 885-889. [Google Scholar] [CrossRef]

Keppel, G., & Wickens, T. D. (2004). Design and analysis: A researcher's handbook (4th ed.). New Jersey: Pearson. [Google Scholar]

Keren, G., & Lewis, C. (1969). Partial omega squared for ANOVA designs. Educational and Psychological Measurement, 39(1), 119-128. [Google Scholar] [CrossRef]

Keselman, H. J. (1975). A Monte Carlo investigation of three estimates of treatment magnitude: Epsilon squared, eta squared, and omega squared. Canadian Psychological Review/Psychologie Canadienne, 16(1), 44-48. [Google Scholar] [CrossRef]

Keselman, H. J., Algina, J., Lix, L. M., Wilcox, R. R., & Deering, K. N. (2008). A generally robust approach for testing hypotheses and setting confidence intervals for effect sizes. Psychological Methods, 13(2), 110-129. [Google Scholar] [CrossRef]

Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56(5), 746-759. [Google Scholar] [CrossRef]

Kline, R. B. (2013). Beyond significance testing: Reforming data analysis methods in behavioral research (2nd ed.). Washington, DC: American Psychological Association. [Google Scholar]

Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4(863), 1-12. [Google Scholar] [CrossRef]

Lenhard, W., & Lenhard, A. (2016). Calculation of effect sizes. [Google Scholar] [CrossRef]

Levine, T. R., & Hullett, C. R. (2002). Eta squared, partial eta squared, and misreporting of effect size in communication research. Human Communication Research, 28(4), 612-625. [Google Scholar] [CrossRef]

Lipsey, M. W., Puzio, K., Yun, C., Hebert, M. A., Steinka-Fry, K., Cole, M. W., . . . Busick, M. D. (2012). Translating the statistical representation of the effects of education interventions into more readily interpretable forms. National Center for Special Education Research, Institute of Education Sciences. [Google Scholar] [URL]

Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. New York: Sage Publications, Inc. [Google Scholar]

Lyons, L. C., & Morris, W. A. (2018). The meta analysis calculator. [Google Scholar] [URL]

Okada, K. (2013). Is omega squared less biased? A comparison of three major effect size indices in one-way anova. Behaviormetrika, 40(2), 129-147. [Google Scholar] [CrossRef]

Olejnik, S., & Algina, J. (2000). Measures of effect size for comparative studies: Applications, interpretations, and limitations. Contemporary Educational Psychology, 25(3), 241-286. [Google Scholar] [CrossRef]

Olejnik, S., & Algina, J. (2003). Generalized eta and omega squared statistics: Measures of effect size for some common research designs. Psychological Methods, 8(4), 434-447. [Google Scholar] [CrossRef]

Pallant, J. (2011). SPSS: Survival manual (4th ed.). Crows Nest, NSW: Allen & Unwin. [Google Scholar]

Pampel, F. C. (2000). Logistic regression: A primer. Thousand Oaks: SAGE Publications. [Google Scholar]

Pearson, K. (1905). Mathematical contributions to the theory of evolution. XIV. On the general theory of skew correlation and non-linear regression. London: Dulau. [Google Scholar]

Pierce, C. A., Block, R. A., & Aguinis, H. (2004). Cautionary note on reporting eta-squared values from multifactor ANOVA designs. Educational and Psychological Measurement, 64(6), 916-924. [Google Scholar] [CrossRef]

Richardson, J. T. E. (2011). Eta squared and partial eta squared as measures of effect size in educational research. Educational Research Review, 6(2), 135-147. [Google Scholar] [CrossRef]

Roberts, J. K., & Henson, R. K. (2002). Correction for bias in estimating effect sizes. Educational and Psychological Measurement, 62(2), 241-253. [Google Scholar] [CrossRef]

Rosenthal, R. (1991). Meta-analytic procedures for social research (Revised). Newbury Park: Sage. [Google Scholar]

Rosenthal, R. (1994). Science and ethics in conducting, analyzing, and reporting psychological research. Psychological Science, 5(3), 127-134. [Google Scholar] [CrossRef]

Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. The American Psychologist, 44(10), 1276-1284. [Google Scholar] [CrossRef]

Rosnow, R. L., Rosenthal, R., & Rubin, D. B. (2000). Contrasts and correlations in effect-size estimation. Psychological Science, 11(6), 446-453. [Google Scholar] [CrossRef]

Sechrest, L., & Yeaton, W. H. (2016). Magnitudes of experimental effects in social science research. Evaluation Review, 6(5), 579-600. [Google Scholar] [CrossRef]

Smithson, M. (2003). Confidence intervals. Thousand Oaks, CA: Sage. [Google Scholar]

Snyder, P., & Lawson, S. (1993). Evaluating results using corrected and uncorrected effect size estimates. The Journal of Experimental Education, 61(4), 334-349. [Google Scholar] [JSTOR]

Steiger, J. H. (2004). Beyond the F test: Effect size confidence intervals and tests of close fit in the analysis of variance and contrast analysis. Psychological Methods, 9(2), 164-182. [Google Scholar]

Steiger, J. H., & Fouladi, R. T. (2016). Noncentrality interval estimation and the evaluation of statistical models. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 197-229). Routledge: Routledge. [Google Scholar]

Stevens, J. P. (2007). Intermediate statistics (3rd ed.). New York: Lawrence Erlbaum Associates. [Google Scholar]

Steyn, H. S. Jr., & Ellis, S. M. (2009). Estimating an effect size in one-way multivariate analysis of variance (MANOVA). Multivariate Behavioral Research, 44(1), 106-129. [Google Scholar] [CrossRef]

Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Pearson Education. [Google Scholar]

Thompson, B. (2002). "Statistical,” “practical,” and “clinical”: How many kinds of significance do counselors need to consider?. Journal of Counseling & Development, 80(1), 64-71. [Google Scholar] [CrossRef]

Thompson, B. (2007). Effect sizes, confidence intervals, and confidence intervals for effect sizes. Psychology in the Schools, 44(5), 423-432. [Google Scholar] [CrossRef]

Vacha-Haase, T., & Thompson, B. (2004). How to estimate and interpret various effect sizes. Journal of Counseling Psychology, 51(4), 473-481. [Google Scholar] [CrossRef]

Vaughan, G. M., & Corballis, M. C. (1969). Beyond tests of significance: Estimating strength of effects in selected ANOVA designs. Psychological Bulletin, 72(3), 204-213. [Google Scholar] [CrossRef]

Wherry, R. J. (1931). A new formula for predicting the shrinkage of the coefficient of multiple correlation. The Annals of Mathematical Statistics, 2(4), 440-457. [Google Scholar] [CrossRef]

Wilkinson, L., & Task Force on Statistical Inference., (1999). Statistical methods in psychology journals: Guidelines and explanations. The American Psychologist, 54(8), 594-604. [Google Scholar]

Wilson, D. B. (2010). Meta-analysis stuff. [Google Scholar] [URL]

Wilson, D. B. (2018). Practical meta-analysis effect size calculator. [Google Scholar] [URL]

Zhang, G., & Algina, J. (2011). A robust Root Mean Square Standardized Effect Size in one-way fixed-effects ANOVA. Journal of Modern Applied Statistical Methods, 10(1), 77-96. [Google Scholar] [CrossRef]