Loading...
Implementing statcheck during peer review is related to a steep decline in statistical-reporting inconsistencies
Nuijten,Michèle B. Wicherts,Jelte M.
Nuijten,Michèle B.
Wicherts,Jelte M.
Abstract
We investigated whether statistical-reporting inconsistencies could be avoided if journals implement the tool statcheck in the peer-review process. In a preregistered pretest-posttest quasi-experiment covering more than 7,000 articles and more than 147,000 extracted statistics, we compared the prevalence of reported p values that were inconsistent with their degrees of freedom and test statistics in two journals that implemented statcheck in their peer-review process (Psychological Science and Journal of Experimental and Social Psychology) and two matched control journals (Journal of Experimental Psychology: General and Journal of Personality and Social Psychology) before and after statcheck was implemented. Preregistered multilevel logistic regression analyses showed that the decrease in both inconsistencies and decision inconsistencies around p = .05 is considerably steeper in statcheck journals than in control journals, offering preliminary support for the notion that statcheck can be a useful tool for journals to avoid statistical-reporting inconsistencies in published articles. We discuss limitations and implications of these findings.
Description
Date
2024-06
Journal Title
Journal ISSN
Volume Title
Publisher
Research Projects
Organizational Units
Journal Issue
Keywords
Metascience, Peer review, Statcheck, Statistical-reporting inconsistencies
Citation
Nuijten, M B & Wicherts, J M 2024, 'Implementing statcheck during peer review is related to a steep decline in statistical-reporting inconsistencies', Advances in Methods and Practices in Psychological Science, vol. 7, no. 2, 25152459241258945. https://doi.org/10.1177/25152459241258945