Loading...
The electrophysiology of lexical prediction of emoji and text
Weissman,Benjamin Cohn,Neil Tanner,Darren
Weissman,Benjamin
Cohn,Neil
Tanner,Darren
Abstract
As emoji often appear naturally alongside text in utterances, they provide a way to study how prediction unfolds in multimodal sentences in direct comparison to unimodal sentences. In this experiment, participants (N = 40) read sentences in which the sentence-final noun appeared in either word form or emoji form, a between-subjects manipulation. The experiment featured both high constraint sentences and low constraint sentences to examine how the lexical processing of emoji interacts with prediction processes in sentence comprehension. Two wellestablished ERP components linked to lexical processing and prediction - the N400 and the Late Frontal Positivity - are investigated for sentence-final words and emoji to assess whether, to what extent, and in what linguistic contexts emoji are processed like words. Results indicate that the expected effects, namely an N400 effect to an implausible lexical item compared to a plausible one and an LFP effect to an unexpected lexical item compared to an expected one, emerged for both words and emoji. This paper discusses the similarities and differences between the stimulus types and constraint conditions, contextualized within theories of linguistic prediction, ERP components, and a multimodal lexicon.
Description
Copyright © 2024 Elsevier Ltd. All rights reserved.
Date
2024-06-06
Journal Title
Journal ISSN
Volume Title
Publisher
Research Projects
Organizational Units
Journal Issue
Keywords
Emoji, Event-related potential (ERP), Late frontal positivity (LFP), Multimodality, N400, Prediction
Citation
Weissman, B, Cohn, N & Tanner, D 2024, 'The electrophysiology of lexical prediction of emoji and text', Neuropsychologia, vol. 198, 108881. https://doi.org/10.1016/j.neuropsychologia.2024.108881
License
info:eu-repo/semantics/openAccess