Loading...
Learning language through pictures
Chrupala,Grzegorz Kadar,Akos Alishahi,Afra
Chrupala,Grzegorz
Kadar,Akos
Alishahi,Afra
Abstract
We propose Imaginet, a model of learning visually grounded representations of language from coupled textual and visual input. The model consists of two Gated Recurrent Unit networks with shared word embeddings, and uses a multi-task objective by receiving a textual description of a scene and trying to concurrently predict its visual representation and the next word in the sentence. Like humans, it acquires meaning representations for individual words from descriptions of visual scenes. Moreover, it learns to effectively use sequential structure in semantic interpretation of multi-word phrases.
Description
Date
2015
Journal Title
Journal ISSN
Volume Title
Publisher
Association for Computational Linguistics
Files
Loading...
P15_2019.pdf
Adobe PDF, 993.91 KB
Loading...
P15_2019.Notes.pdf
Adobe PDF, 3 MB
Research Projects
Organizational Units
Journal Issue
Keywords
Citation
Chrupala, G, Kadar, A & Alishahi, A 2015, Learning language through pictures. in C Zong & M Strube (eds), Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). Association for Computational Linguistics, Beijing, China, pp. 112-118, 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China, 26/07/15. < http://www.aclweb.org/anthology/P/P15/P15-2019.pdf >