Thumbnail Image
Item

Are We Paying Attention to Her? Investigating Gender Disambiguation and Attention in Machine Translation.

Manna,Chiara
; 
Alishahi,Afra
; 
Blain,Frédéric
; 
Vanmassenhove,Eva
Abstract
While gender bias in modern Neural Machine Translation (NMT) systems has received much attention, the traditional evaluation metrics for these systems do not fully capture the extent to which models integrate contextual gender cues. We propose a novel evaluation metric called Minimal Pair Accuracy (MPA) which measures the reliance of models on gender cues for gender disambiguation. We evaluate a number of NMT models using this metric, we show that they ignore available gender cues in most cases in favour of (statistical) stereotypical gender interpretation. We further show that in anti-stereotypical cases, these models tend to more consistently take male gender cues into account while ignoring the female cues. Finally, we analyze the attention head weights in the encoder component of these models and show that while all models to some extent encode gender information, the male gender cues elicit a more diffused response compared to the more concentrated and specialized responses to female gender cues.
Description
DBLP License: DBLP's bibliographic metadata records provided through http://dblp.org/ are distributed under a Creative Commons CC0 1.0 Universal Public Domain Dedication. Although the bibliographic metadata records are provided consistent with CC0 1.0 Dedication, the content described by the metadata records is not. Content may be subject to copyright, rights of privacy, rights of publicity and other restrictions.
Date
2025
Journal Title
Journal ISSN
Volume Title
Publisher
European Association for Machine Translation
Collections
Tilburg University Research Output Collection
Show all metadata
Files
Thumbnail Image
are_we_paying.pdf
Adobe PDF, 1.42 MB
Research Projects
Organizational Units
Journal Issue
Keywords
neural machine translation, machine translation
Citation
Manna, C, Alishahi, A, Blain, F & Vanmassenhove, E 2025, Are We Paying Attention to Her? Investigating Gender Disambiguation and Attention in Machine Translation. in Proceedings of the 3rd Workshop on Gender-Inclusive Translation Technologies (GITT 2025). vol. abs/2505.08546, CoRR, European Association for Machine Translation, Workshop on Gender-Inclusive Translation Technologies , Geneva, Switzerland, 23/06/25. < https://aclanthology.org/2025.gitt-1.1/ >
URI
https://hdl.handle.net/20.500.14602/102566
License
info:eu-repo/semantics/openAccess
cb
Embedded videos