Loading...
Identifying careless responding in ecological momentary assessment: Inconsistent signals from different detection methods in the WARN-D data
Vogelsmeier,Leonie V. D. E. Siepe,Björn S. Ulitzsch,Esther Eisele,Gudrun Geeraerts,Joran Klocek,Adam Proppert,Ricarda K.K. Rieble,Carlotta Tutunji,Rayyan Dejonckheere,Egon
Vogelsmeier,Leonie V. D. E.
Siepe,Björn S.
Ulitzsch,Esther
Eisele,Gudrun
Geeraerts,Joran
Klocek,Adam
Proppert,Ricarda K.K.
Rieble,Carlotta
Tutunji,Rayyan
Dejonckheere,Egon
Abstract
Collecting intensive longitudinal data via methods like ecological momentary assessment (EMA) is necessary to study dynamics in psychological constructs. However, detecting careless responding is more important than ever, as this methodology burdens participants more than traditional single survey studies. If undetected, careless responding reduces data quality and can lead to invalid findings by distorting correlations between items and constructs and compromising the psychometric properties of scales. In turn, this hinders progress toward cumulative psychological science. In recent years, several data-driven approaches have been proposed that assess whether an individual was attentive or careless for each timepoint-specific measurement after data collection. However, the variety and complexity of available methods can be overwhelming for researchers. In this paper, we summarized, applied, and compared 13 methods that flag careless responses in a rich EMA dataset from the WARN-D study with 206,318 observations (from n = 1194 participants). Together with our detailed supplementary tutorials, the paper serves as a guide for researchers to evaluate and report careless responding in their own data. At the same time, we show that the development of methods for careless responding detection is still in its early stages, as the studied approaches produced very different results. We provide recommendations for which methods are current
Description
Date
2025
Journal Title
Journal ISSN
Volume Title
Publisher
PsyArXiv Preprints
Research Projects
Organizational Units
Journal Issue
Keywords
Ecological Momentary Assessment, Mixture Modeling, Data Quality, Experience Sampling Methodology, Insufficient Effort Responding, Measurement
Citation
Vogelsmeier, L V D E, Siepe, B S, Ulitzsch, E, Eisele, G, Geeraerts, J, Klocek, A, Proppert, R K K, Rieble, C, Tutunji, R, Dejonckheere, E, Kuppens, P & Fried, E I 2025 'Identifying careless responding in ecological momentary assessment : Inconsistent signals from different detection methods in the WARN-D data' PsyArXiv Preprints. https://doi.org/10.31234/osf.io/4dfbz_v1
License
info:eu-repo/semantics/openAccess