I Need a CAVAA: How Conversational Agent Voting Advice Applications (CAVAAs) Affect Users' Political Knowledge and Tool Experience
Kamoen,Naomi Liebrecht,C.
Kamoen,Naomi
Liebrecht,C.
Abstract
In election times, millions of voters consult Voting Advice Applications (VAAs) to learn more about political parties and their standpoints. While VAAs have been shown to enhance political knowledge and increase electoral turnout, research also demonstrates that voters frequently experience comprehension problems when responding to the political attitude statements in a VAA. We describe two studies in which we test a new type of VAA, called Conversational Agent VAA (CAVAA), in which users can easily access relevant information about the political issues in the VAA statements by asking questions to a chatbot. Study 1 reports about an online experiment (N = 229) with a 2 (Type: traditional VAA/CAVAA) x 2 (Political sophistication: low/high) design. Results show that CAVAA users report higher perceived political knowledge scores and also answer more factual knowledge questions correctly than users of a regular VAA. Also, participants' CAVAA experience was evaluated better. In Study 2 (N = 180), we compared three CAVAA designs (a structured design with buttons, a non-structured design with an open text field, and a semi-structured design with both buttons and an open text field), again for higher and lower politically sophisticated users. While the three designs score equally high on factual and perceived knowledge indicators, the experience of the structured CAVAA was evaluated more positively than the non-structured version. To explore the possible cause for these results, we conducted an additional qualitative content analysis on 90 chatbot-conversations (30 per chatbot version). This analysis shows that users more frequently access additional information in a structured design than in a non-structured design, whereas the number of break-offs is the same. This suggests that the structured design delivers the best experience, because it provides the best trigger to ask questions to the chatbot.
Description
Funding Information: Three Master's theses we have supervised form the basis of this research paper. The authors would therefore like to thank Simone van Limpt (MSc) for collecting the data of Study 1, Pleun Mekel (MSc) for collecting the data of Study 2, and Tessa McCartan (MSc) for analyzing Mekel's data for the exploratory qualitative content analysis. A summarized description of both experimental studies appeared previously in Tekstblad, i.e., Kamoen et al. (2020). We would also like to thank the reviewers for their close reading of our manuscript. Publisher Copyright: Copyright © 2022 Kamoen and Liebrecht.
Date
2022-05-12
Journal Title
Journal ISSN
Volume Title
Publisher
Research Projects
Organizational Units
Journal Issue
Keywords
Voting Advice Applications (VAAs), Conversational Agents, Chatbot design, Usefulness, Ease of use, Playfulness, Voting intention
Citation
Kamoen, N & Liebrecht, C 2022, 'I Need a CAVAA : How Conversational Agent Voting Advice Applications (CAVAAs) Affect Users' Political Knowledge and Tool Experience', Frontiers in Artificial Intelligence, vol. 5, 835505. https://doi.org/10.3389/frai.2022.835505
License
info:eu-repo/semantics/openAccess