Thumbnail Image
Item

Insights into the accuracy of social scientists' forecasts of societal change

Grossmann,Igor
; 
Rotella,Amanda A.
; 
Hutcherson,Cendri
; 
Sharpinskyi,Konstantyn
; 
Varnum,Michael E. W.
; 
Achter,Sebastian K.
; 
Dhami,Mandeep
; 
Guo,Xinqi Evie
; 
Kara-Yakoubian,Mane R.
; 
Mandel,David
... show 10 more
Abstract
How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender-career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists' forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data.
Description
This programme of research was supported by the Basic Research Program at the National Research University Higher School of Economics (M. Fabrykant), John Templeton Foundation grant no. 62260 (I.G. and P.E.T.), Kega 079UK-4/2021 (P.K.), Ministerio de Ciencia e Innovación España grants no. PID2019-111512RB-I00-HMDM and no. HDL-HS-280218 (A.A.), etc.
Date
2023
Journal Title
Journal ISSN
Volume Title
Publisher
Collections
Tilburg University Research Output Collection
Show all metadata
Files
Thumbnail Image
SP_Jaeger_insights_into_the_accuracy_NHB_2023.pdf
Adobe PDF, 1.73 MB
Research Projects
Organizational Units
Journal Issue
Keywords
IMPLICIT, ATTITUDES
Citation
Grossmann, I, Rotella, A A, Hutcherson, C, Sharpinskyi, K, Varnum, M E W, Achter, S K, Dhami, M, Guo, X E, Kara-Yakoubian, M R, Mandel, D, Raes, L, Tay, L, Vie, A, Wagner, L, Adamkovic, M, Arami, A, Arriaga, P, Bandara, K, Banik, G, Bartos, F, Baskin, E, Bergmeir, C, Bialek, M K, Borsting, C T, Browne, D M, Caruso, E, Chen, R, Chie, B-T J, Chopik, W N, Collins, R, Cong, C W G, Conway, L, Davis, M V, Day, M A, Dhaliwal, N D, Durham, J, Dziekan, M T, Elbaek, C, Shuman, E, Fabrykant, M, Firat, M T, Fong, G A, Frimer, J M, Gallegos, J B, Goldberg, S, Gollwitzer, A, Goyal, J, Graf-Vlachy, L D, Gronlund, S, Hafenbraedl, S, Hartanto, A J, Hirshberg, M J, Hornsey, M, Howe, P D L, Izadi, A, Jaeger, B, Kacmar, P, Kim, Y J, Krenzler, R G, Lannin, D, Lin, H-W, Lou, N M, Lua, V Y Q W, Lukaszewski, A L, Ly, A R, Madan, C, Maier, M M, Majeed, N S, March, D A, Marsh, A, Misiak, M, Myrseth, K O R M, Napan, J, Nicholas, J, Nikolopoulos, K, Otterbring, T, Paruzel-Czachura, M, Pauer, S, Protzko, J, Raffaelli, Q, Ropovik, I, Ross, R M, Roth, Y, Roysamb, E, Schnabel, L, Schuetz, A, Seifert, M, Sevincer, A T, Sherman, G T, Simonsson, O, Sung, M-C, Tai, C-C, Talhelm, T, Teachman, B A, Tetlock, P E, Thomakos, D, Tse, D C K, Twardus, O J, Tybur, J M, Ungar, L, Vandermeulen, D, Vaughan Williams, L, Vosgerichian, H A, Wang, Q, Wang, K, Whiting, M E, Wollbrant, C E, Yang, T, Yogeeswaran, K, Yoon, S, Alves, V R, Andrews-Hanna, J R, Bloom, P A, Boyles, A, Charis, L, Choi, M, Darling-Hammond, S, Ferguson, Z E, Kaiser, C R, Karg, S T, Ortega, A L, Mahoney, L, Marsh, M S, Martinie, M F R C, Michaels, E K, Millroth, P, Naqvi, J B, Ng, W, Rutledge, R B, Slattery, P, Smiley, A H, Strijbis, O, Sznycer, D, Tsukayama, E, van Loon, A, Voelkel, J G, Wienk, M N A & Wilkening, T 2023, 'Insights into the accuracy of social scientists' forecasts of societal change', Nature Human Behaviour, vol. 7, no. 4, pp. 484-501. https://doi.org/10.1038/s41562-022-01517-1
URI
https://hdl.handle.net/20.500.14602/78970
Embedded videos