Loading...
Citation patterns following a strongly contradictory replication result: Four case studies from psychology
Hardwicke,Tom E. Szűcs,Dénes Thibault,Robert T. Crüwell,Sophia van den Akker,Olmo R. Nuijten,Michèle B. Ioannidis,John P. A.
Hardwicke,Tom E.
Szűcs,Dénes
Thibault,Robert T.
Crüwell,Sophia
van den Akker,Olmo R.
Nuijten,Michèle B.
Ioannidis,John P. A.
Abstract
Replication studies that contradict prior findings may facilitate scientific self-correction by triggering a reappraisal of the original studies; however, the research community's response to replication results has not been studied systematically. One approach for gauging responses to replication results is to examine how they affect citations to original studies. In this study, we explored postreplication citation patterns in the context of four prominent multilaboratory replication attempts published in the field of psychology that strongly contradicted and outweighed prior findings. Generally, we observed a small postreplication decline in the number of favorable citations and a small increase in unfavorable citations. This indicates only modest corrective effects and implies considerable perpetuation of belief in the original findings. Replication results that strongly contradict an original finding do not necessarily nullify its credibility; however, one might at least expect the replication results to be acknowledged and explicitly debated in subsequent literature. By contrast, we found substantial citation bias: The majority of articles citing the original studies neglected to cite relevant replication results. Of those articles that did cite the replication but continued to cite the original study favorably, approximately half offered an explicit defense of the original study. Our findings suggest that even replication results that strongly contradict original findings do not necessarily prompt a corrective response from the research community.
Description
The authors received no specific funding for this work. The Meta-Research Innovation Center at Stanford (METRICS) is supported by a grant from the Laura and John Arnold Foundation. The Meta-Research Innovation Center Berlin (METRIC-B) is supported by a grant from the Einstein Foundation and Stiftung Charité. The work of J. P. A. Ioannidis is supported by an unrestricted grant from Sue and Bob O’Donnell. R. T. Thibault is supported by a postdoctoral fellowship from the Fonds de la recherche en santé du Québec. T. E. Hardwicke receives funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement 841188. D. Szűcs receives funding from the James S. McDonnell Foundation 21st Century Science Initiative in Understanding Human Cognition (Grant 220020370). O. R. van den Akker is supported by a Consolidator Grant (IMPROVE) from the European Research Council (Grant 726361).
Date
2021
Journal Title
Journal ISSN
Volume Title
Publisher
Research Projects
Organizational Units
Journal Issue
Keywords
replication, citations, meta-research, self-correction, citation bias, open data, open materials, preregistered, EGO-DEPLETION, SELF, EXPOSURE
Citation
Hardwicke, T E, Szűcs, D, Thibault, R T, Crüwell, S, van den Akker, O R, Nuijten, M B & Ioannidis, J P A 2021, 'Citation patterns following a strongly contradictory replication result : Four case studies from psychology', Advances in Methods and Practices in Psychological Science, vol. 4, no. 3. https://doi.org/10.1177/25152459211040837