Loading...
Investigating the utility of fixed-margin sampling in network psychometrics
Epskamp,Sacha Fried,Eiko van Borkulo,Claudia Robinaugh,Don Marsman,Maarten Dalege,Jonas Rhemtulla,Mijke Cramer,A.O.J.
Epskamp,Sacha
Fried,Eiko
van Borkulo,Claudia
Robinaugh,Don
Marsman,Maarten
Dalege,Jonas
Rhemtulla,Mijke
Cramer,A.O.J.
Abstract
Steinley, Hoffman, Brusco, and Sher (2017) proposed a new method for evaluating the performance of psychological network models: fixed-margin sampling. The authors investigated LASSO regularized Ising models (eLasso) by generating random datasets with the same margins as the original binary dataset, and concluded that many estimated eLasso parameters are not distinguishable from those that would be expected if the data were generated by chance. We argue that fixed-margin sampling cannot be used for this purpose, as it generates data under a particular null-hypothesis: a unidimensional factor model with interchangeable indicators (i.e., the Rasch model). We show this by discussing relevant psychometric literature and by performing simulation studies. Results indicate that while eLasso correctly estimated network models and estimated almost no edges due to chance, fixed-margin sampling performed poorly in classifying true effects as “interesting” (Steinley et al. 2017, p. 1004). Further simulation studies indicate that fixed-margin sampling offers a powerful method for highlighting local misfit from the Rasch model, but performs only moderately in identifying global departures from the Rasch model. We conclude that fixed-margin sampling is not up to the task of assessing if results from estimated Ising models or other multivariate psychometric models are due to chance.
Description
Date
2021
Journal Title
Journal ISSN
Volume Title
Publisher
Research Projects
Organizational Units
Journal Issue
Keywords
FORBES, IRT, Ising models, LIMITED REPLICABILITY, MARKON, MODEL, PSYCHOPATHOLOGY SYMPTOM NETWORKS, SELECTION, WRIGHT, exploratory data analysis, fixed-margin sampling, network Psychometrics
Citation
Epskamp, S, Fried, E, van Borkulo, C, Robinaugh, D, Marsman, M, Dalege, J, Rhemtulla, M & Cramer, A O J 2021, 'Investigating the utility of fixed-margin sampling in network psychometrics', Multivariate Behavioral Research, vol. 56, no. 2, pp. 314-328. https://doi.org/10.1080/00273171.2018.1489771