Loading...
Replicability, robustness, and reproducibility in psychological science
Nosek,Brian A. Hardwicke,Tom E. Moshontz,Hannah Allard,Aurélien Corker,Katherine S. Dreber,Anna Fidler,Fiona Hilgard,Joe Kline Struhl,Melissa Nuijten,Michèle B.
Nosek,Brian A.
Hardwicke,Tom E.
Moshontz,Hannah
Allard,Aurélien
Corker,Katherine S.
Dreber,Anna
Fidler,Fiona
Hilgard,Joe
Kline Struhl,Melissa
Nuijten,Michèle B.
Abstract
Replication—an important, uncommon, and misunderstood practice—is gaining appreciation in psychology. Achieving replicability is important for making research progress. If findings are not replicable, then prediction and theory development are stifled. If findings are replicable, then interrogation of their meaning and validity can advance knowledge. Assessing replicability can be productive for generating and testing hypotheses by actively confronting current understandings to identify weaknesses and spur innovation. For psychology, the 2010s might be characterized as a decade of active confrontation. Systematic and multi-site replication projects assessed current understandings and observed surprising failures to replicate many published findings. Replication efforts highlighted sociocultural challenges such as disincentives to conduct replications and a tendency to frame replication as a personal attack rather than a healthy scientific practice, and they raised awareness that replication contributes to self-correction. Nevertheless, innovation in doing and understanding replication and its cousins, reproducibility and robustness, has positioned psychology to improve research practices and accelerate progress.
Description
This work was supported by grants to B.A.N. from Arnold Ventures, the John Templeton Foundation, Templeton World Charity Foundation, and Templeton Religion Trust. T.E.H. received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 841188. F.F. is funded by an Australian Research Council Future Fellowship (FT150100297). A.M.S. is currently funded through a grant by the Dutch Research Council (NWO) on “Increasing the Reliability and Efficiency of Psychological Science.” L.D.S. has received funding from the National Institutes of Health, the Agency for Healthcare Research and Quality, and the Patient-Centered Outcomes Research Institute.
Date
2022
Journal Title
Journal ISSN
Volume Title
Publisher
Research Projects
Organizational Units
Journal Issue
Keywords
replication, reproducibility, robustness, generalizability, research methods, statistical inference, validity, theory, metascience
Citation
Nosek, B A, Hardwicke, T E, Moshontz, H, Allard, A, Corker, K S, Dreber, A, Fidler, F, Hilgard, J, Kline Struhl, M, Nuijten, M B, Rohrer, J, Romero, F, Scheel, A, Scherer, L, Schönbrodt, F D & Vazire, S 2022, 'Replicability, robustness, and reproducibility in psychological science', Annual Review of Psychology, vol. 73, pp. 719-748. https://doi.org/10.31234/osf.io/ksfvq, https://doi.org/10.1146/annurev-psych-020821-114157