Thumbnail Image
Item

Consensus-based guidance for conducting and reporting multi-analyst studies

Aczel,Balazs
; 
Szaszi,Barnabas
; 
Nilsonne,Gustav
; 
van den Akker,Olmo R
; 
Albers,Casper J
; 
van Assen,Marcel
; 
Bastiaansen,Jojanneke A
; 
Benjamin,Daniel
; 
Boehm,Udo
; 
Botvinik-nezer,Rotem
... show 10 more
Abstract
Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.
Description
Funding Netherlands Organisation for Scientific Research (406-17-568) Alexandra Sarafoglou Natural Sciences and Engineering Research Council of Canada (BP-546283-2020) Samuel St-Jean Fonds de Recherche du Québec - Nature et Technologies (290978) Samuel St-Jean European Research Council (726361) Jelte Wicherts Olmo R van den Akker European Research Council (681466) Yoram K Kunkels VIDI fellowship organisation (016.Vidi.188.001) Don van Ravenzwaaij VENI fellowship grant (Veni 191G.037) Laura F Bringmann National Science Foundation (1760052) Matthew J Salganik Weizmann Institute of Science (Israel National Postdoctoral Award Program for Advancing Women in Science) Rotem Botvinik-Nezer John Templeton Foundation, Templeton World Charity Foundation, Templeton Religion Trust, and Arnold Ventures Brian A Nosek Institut Européen d'Administration des Affaires Eric Luis Uhlmann European Research Council (640638) Noah NN van Dongen
Date
2021
Journal Title
Journal ISSN
Volume Title
Publisher
Collections
Tilburg University Research Output Collection
Show all metadata
Files
Thumbnail Image
MTO_vd_Akker_consensus_based_guidance_eLife_2021.pdf
Adobe PDF, 467.78 KB
Research Projects
Organizational Units
Journal Issue
Keywords
DELPHI, None, REPRODUCIBILITY, analytical variability, expert consensus, metascience, multi-analyst, science forum, statistical practice
Citation
Aczel, B, Szaszi, B, Nilsonne, G, van den Akker, O R, Albers, C J, van Assen, M, Bastiaansen, J A, Benjamin, D, Boehm, U, Botvinik-nezer, R, Bringmann, L F, Busch, N A, Caruyer, E, Cataldo, A M, Cowan, N, Delios, A, Van Dongen, N N, Donkin, C, Van Doorn, J B, Dreber, A, Dutilh, G, Egan, G F, Gernsbacher, M A, Hoekstra, R, Hoffmann, S, Holzmeister, F, Huber, J, Johannesson, M, Jonas, K J, Kindel, A T, Kirchler, M, Kunkels, Y K, Lindsay, D S, Mangin, J, Matzke, D, Munafò, M R, Newell, B R, Nosek, B A, Poldrack, R A, Van Ravenzwaaij, D, Rieskamp, J, Salganik, M J, Sarafoglou, A, Schonberg, T, Schweinsberg, M, Shanks, D, Silberzahn, R, Simons, D J, Spellman, B A, St-jean, S, Starns, J J, Uhlmann, E L, Wicherts, J & Wagenmakers, E 2021, 'Consensus-based guidance for conducting and reporting multi-analyst studies', Elife, vol. 10, e72185. https://doi.org/10.7554/eLife.72185
URI
https://hdl.handle.net/20.500.14602/76117
Embedded videos