Wave to Syntax: Probing spoken language models for syntax
Shen,Gaofei Alishahi,Afra Bisazza,Arianna ChrupaĆa,Grzegorz
Shen,Gaofei
Alishahi,Afra
Bisazza,Arianna
ChrupaĆa,Grzegorz
Abstract
Understanding which information is encoded in deep models of spoken and written language has been the focus of much research in recent years, as it is crucial for debugging and improving these architectures. Most previous work has focused on probing for speaker characteristics, acoustic and phonological information in models of spoken language, and for syntactic information in models of written language. Here we focus on the encoding of syntax in several self-supervised and visually grounded models of spoken language. We employ two complementary probing methods, combined with baselines and reference representations to quantify the degree to which syntactic structure is encoded in the activations of the target models. We show that syntax is captured most prominently in the middle layers of the networks, and more explicitly within models with more parameters.
Description
Publisher Copyright: © 2023 International Speech Communication Association. All rights reserved.
Date
2023
Journal Title
Journal ISSN
Volume Title
Publisher
Research Projects
Organizational Units
Journal Issue
Keywords
computational linguistics, speech recognition, syntax
Citation
Shen, G, Alishahi, A, Bisazza, A & ChrupaĆa, G 2023, Wave to Syntax : Probing spoken language models for syntax. in Proc. INTERSPEECH 2023. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, pp. 1259-1263. https://doi.org/10.21437/Interspeech.2023-679