Estimating the Reproducibility of Experimental Philosophy
Cova,Florian Strickland,Brent Abatista,Angela Allard,Aurélien Andow,James Attie,Mario Beebe,James Berniūnas,Renatas Boudesseul,Jordane Colombo,Matteo
Cova,Florian
Strickland,Brent
Abatista,Angela
Allard,Aurélien
Andow,James
Attie,Mario
Beebe,James
Berniūnas,Renatas
Boudesseul,Jordane
Colombo,Matteo
Abstract
Responding to recent concerns about the reliability of the published literature in psychology and other disciplines, we formed the X-Phi Replicability Project (XRP) to estimate the reproducibility of experimental philosophy (osf.io/dvkpr). Drawing on a representative sample of 40 x-phi studies published between 2003 and 2015, we enlisted 20 research teams across 8 countries to conduct a high-quality replication of each study in order to compare the results to the original published findings. We found that x-phi studies – as represented in our sample – successfully replicated about 70% of the time. We discuss possible reasons for this relatively high replication rate in the field of experimental philosophy and offer suggestions for best research practices going forward.
Description
Funding Information: José Hernández-Conde carried out his work as a Visiting Scholar at the University of Pittsburgh’s HPS Department. He was financially supported by a PhD scholarship and mobility grant from the University of the Basque Country, and by the Spanish Ministry of Economy and Competitiveness research project No. FFI2014-52196-P. His replication research was supported by the Pittsburgh Empirical Philosophy Lab. Funding Information: Antonio Gaitán Torres and Hugo Viciana benefited from funding from the Ministerio de Economía y Competitividad for the project “La constitución del sujeto en la interacción social” (Grant No. FFI2015-67569-C2-1-P & FFI2015-67569-C2-2-P). Funding Information: This project could not have been possible without the financial support of multiple organizations. Florian Cova’s work on this project was supported by a grant from the Cogito Foundation (Grant No. S-131/13, “Towards an Experimental Philosophy of Aesthetics”). Funding Information: This project could not have been possible without the financial support of multiple organizations. Florian Cova’s work on this project was supported by a grant from the Cogito Foundation (Grant No. S-131/13, “Towards an Experimental Philosophy of Aesthetics”). Brent Strickland’s work was supported by two grants from the Agence Nationale de la Recherche (Grants No. ANR-10-IDEX-0001-02 PSL*, ANR-10-LABX-0087 IEC). Matteo Colombo, Noah van Dongen, Felipe Romero and Jan Sprenger’s work was supported by the European Research Council (ERC) through Starting Grant. No. 640638 (“Making Scientific Inferences More Objective”). Rodrigo Diaz and Kevin Reuter would like to acknowledge funding from the Swiss National Science Foundation, Grant No. 100012_169484. Antonio Gaitán Torres and Hugo Viciana benefited from funding from the Ministerio de Economía y Competitividad for the project “La constitución del sujeto en la interacción social” (Grant No. FFI2015-67569-C2-1-P & FFI2015-67569-C2-2-P). José Hernández-Conde carried out his work as a Visiting Scholar at the University of Pittsburgh’s HPS Department. He was financially supported by a PhD scholarship and mobility grant from the University of the Basque Country, and by the Spanish Ministry of Economy and Competitiveness research project No. FFI2014-52196-P. His replication research was supported by the Pittsburgh Empirical Philosophy Lab. Hanna Kim’s work was supported by the Pittsburgh Empirical Philosophy Lab. Shen-yi Liao’s work was supported by the University of Puget Sound Start-up Funding. Tania Moerenhout carried out her work as a Visiting Researcher at the Center for Bioethics and Health Law, University of Pittsburgh, PA (Aug 2016-July 2017). Aurélien Allard, Miklos Kurthy, and Paulo Sousa are grateful to Rashmi Sharma for her help in the replication of Knobe & Burra (2006), in particular for her help in translating the demographic questions from English to Hindi. Ivar Hannikainen and Florian Cova would like to thank Uri Simonsohn for his help in discussing the meaning and best interpretation of p-curves. Finally, we would like to thank all the authors of original studies who accepted to take the time to answer our questions, share their original material and data, and discuss the results of our replication attempts with us. Funding Information: Rodrigo Diaz and Kevin Reuter would like to acknowledge funding from the Swiss National Science Foundation, Grant No. 100012_169484. Funding Information: Brent Strickland’s work was supported by two grants from the Agence Nationale de la Recherche (Grants No. ANR-10-IDEX-0001-02 PSL*, ANR-10-LABX-0087 IEC). Funding Information: Matteo Colombo, Noah van Dongen, Felipe Romero and Jan Sprenger’s work was supported by the European Research Council (ERC) through Starting Grant. No. 640638 (“Making Scientific Inferences More Objective”). Publisher Copyright: © 2018, Springer Science+Business Media B.V., part of Springer Nature.
Date
2021-03
Journal Title
Journal ISSN
Volume Title
Publisher
Research Projects
Organizational Units
Journal Issue
Keywords
Experimental philosophy.
Citation
Cova, F, Strickland, B, Abatista, A, Allard, A, Andow, J, Attie, M, Beebe, J, Berniūnas, R, Boudesseul, J, Colombo, M, Cushman, F, Diaz, R, N’Djaye Nikolai van Dongen, N, Dranseika, V, Earp, B D, Torres, A G, Hannikainen, I, Hernández-Conde, J V, Hu, W, Jaquet, F, Khalifa, K, Kim, H, Kneer, M, Knobe, J, Kurthy, M, Lantian, A, Liao, S Y, Machery, E, Moerenhout, T, Mott, C, Phelan, M, Phillips, J, Rambharose, N, Reuter, K, Romero, F, Sousa, P, Thalabard, E, Tobia, K, Viciana, H, Wilkenfeld, D & Zhou, X 2021, 'Estimating the Reproducibility of Experimental Philosophy', Review of Philosophy and Psychology, vol. 12, no. 1, pp. 9-44. https://doi.org/10.1007/s13164-018-0400-9
License
info:eu-repo/semantics/restrictedAccess