Loading...
The potential of preregistration in psychology: Assessing preregistration producibility andpreregistration-study consistency
van den Akker,Olmo R. Bakker,Marjan van Assen,Marcel A.L.M. Pennington,Charlotte R. Verweij,Leone Elsherif,Mahmoud M. Claesen,Aline Gaillard,Stefan D.M. Yeung,Siu Kit Frankenberger,Jan Luca
van den Akker,Olmo R.
Bakker,Marjan
van Assen,Marcel A.L.M.
Pennington,Charlotte R.
Verweij,Leone
Elsherif,Mahmoud M.
Claesen,Aline
Gaillard,Stefan D.M.
Yeung,Siu Kit
Frankenberger,Jan Luca
Abstract
Study preregistration has become increasingly popular in psychology, but its potential to restrict researcher degrees of freedom has not yet been empirically verified. We used an extensive protocol to assess the producibility (i.e., the degree to which a study can be properly conducted based on the available information) of preregistrations and the consistency between preregistrations and their corresponding papers for 300 psychology studies. We found that preregistrations often lack methodological details and that undisclosed deviations from preregistered plans are frequent. These results highlight that biases due to researcher degrees of freedom remain possible in many preregistered studies. More comprehensive registration templates typically yielded more producible preregistrations. We did not find that the producibility and consistency of preregistrations differed over time or between original and replication studies. Furthermore, we found that operationalizations of variables were generally preregistered more producible and consistently than other study parts. Inconsistencies between preregistrations and published studies were mainly encountered for data collection procedures, statistical models, and exclusion criteria. Our results indicate that, to unlock the full potential of preregistration, researchers in psychology should aim to write more producible preregistrations, adhere to these preregistrations more faithfully, and more transparently report any deviations from their preregistrations. This could be facilitated by training and education to improve preregistration skills, as well as the development of more comprehensive templates.
Description
The authors declare that they have no conflicts of interest. Preliminary data related to this study were presented online on September 18, 2021, as part of the Metascience Symposium 2021. The final results related to this study \ were presented at the Perspectives on Scientific Error Event on February 29, 2024, and The Royal Society’s “The Promises and Pitfalls of Preregistration” Meeting on March 4, 2024
Date
2024
Journal Title
Journal ISSN
Volume Title
Publisher
Files
Research Projects
Organizational Units
Journal Issue
Keywords
metaresearch, preregistration, preregistration deviations, preregistration producibility, preregistration-study consistency
Citation
van den Akker, O R, Bakker, M, van Assen, M A L M, Pennington, C R, Verweij, L, Elsherif, M M, Claesen, A, Gaillard, S D M, Yeung, S K, Frankenberger, J L, Krautter, K, Cockcroft, J P, Kreuer, K S, Evans, T R, Heppel, F M, Schoch, S F, Korbmacher, M, Yamada, Y, Albayrak-Aydemir, N, Alzahawi, S, Sarafoglou, A, Sitnikov, M M, Dechterenko, F, Wingen, S, Grinschgl, S, Hartmann, H, Stewart, S L K, de Oliveira, C M F, Ashcroft-Jones, S, Baker, B J & Wicherts, J M 2024, 'The potential of preregistration in psychology : Assessing preregistration producibility andpreregistration-study consistency', Psychological Methods. https://doi.org/10.1037/met0000687