Statistical Cognition

As consultants, members of DaSAL are frequently exposed to new statistical techniques that drive our continual learning. Unsurprisingly, we often find learning new techniques and approaches to be a challenging process. Perhaps more surprising, however, are findings that even trained researchers find it challenging to understand the fundamental statistical techniques that are used in almost all psychological research, and may be overconfident in their true level of understanding.

Shedding light on this phenomenon is research on statistical cognition, the study of how people understand and statistical concepts and the presentation of statistical analyses. Some statistical cognition research has examined how people interpret findings from NHST and estimation like confidence intervals. While researchers can often make more accurate interpretations of data using confidence intervals than significance testing (Coulson et al., 2012) researchers’ understanding of confidence intervals is far from  perfect. In fact, several findings have shown that researchers misunderstand confidence intervals (Belia et al., 2005; Hoekstra et al., 2014). Reading about common misconceptions of fundamental statistics to our field highlights the need to review our own understanding of these “basic” concepts even as we develop our knowledge of increasingly complex statistical procedures. Below are some interesting articles that provide insight into some common shortcomings of our statistical cognition.

Belia, S., Fidler, F., Williams, J., & Cumming, G. (2005). Researchers misunderstand confidence intervals and standard error bars. Psychological Methods, 10, 389-396.

Beyth-Marom, R., Fidler, F., & Cumming, G. (2008). Statistical cognition: Towards evidence-based practice in statistics and statistics education. Statistics Education Research Journal., 7, 20-39

Coulson, M., Healey, M., Fidler, F., & Cumming, G. (2010). Confidence intervals permit, but do not guarantee, better inference than statistical significance testing. Frontiers in Quantitative Psychology and Measurement, 1:26. doi:10.3389/fpsyg.2010.00026.

Hoekstra, R., Morey, R. D., Rouder, J. N., Wagenmakers, E.-J. (2014). Robust misinterpretation of confidence intervals. Psychonomic Bulletin & Review, 1-8.