University of Hull logo

Event processing in the visual world: Projected motion paths during spoken sentence comprehension

Kamide, Yuki; Lindsay, Shane; Scheepers, Christoph; Kukona, Anuenue

Abstract

Motion events in language describe the movement of an entity to another location along a path. In two eye-tracking experiments we found that comprehension of motion events involves the online construction of a spatial mental model that integrates language with the visual world. In the first experiment, participants listened to sentences describing the movement of an agent to a goal location with verbs suggesting a more upwards (e.g., “jump”) or more downwards oriented path (e.g., “crawl”) while concurrently viewing a visual scene depicting the agent, the goal, and some ‘empty space’ in between. We found that in the rare event of fixating the empty space region between agent and goal, visual attention was biased upwards or downwards depending on the kind of verb. In Experiment 2, the sentences were presented concurrently with scenes featuring a central ‘obstruction’ which would not only impose further constraints on verb-related motion paths, but also increase the likelihood of fixating the area in-between the agent and the goal. The results from this experiment corroborated and refined the previous findings. Specifically, eye-movement effects started immediately after hearing the verb and were in line with data from an additional mouse tracking task which encouraged a more explicit spatial re-enactment of the motion event. In revealing how event comprehension operates in the visual world, these findings suggest a mental simulation process whereby spatial details of motion events are mapped onto the world through visual attention. The strength and detectability of such effects in overt eye-movements is constrained by the visual world and the fact that perceivers rarely fixate regions of empty space.

Journal Article Type Article
Publication Date May 1, 2016
Journal Journal of experimental psychology : learning, memory, and cognition
Print ISSN 0278-7393
Electronic ISSN 1939-1285
Publisher American Psychological Association
Peer Reviewed Peer Reviewed
Volume 42
Issue 5
Pages 804-812
DOI https://doi.org/10.1037/xlm0000199
Keywords Motion event processing, Sentence comprehension, Spatial processing, Verb semantics, Eye movements
Publisher URL http://psycnet.apa.org/doiLanding?doi=10.1037%2Fxlm0000199
Additional Information Author's accepted manuscript of article which has been accepted for future publication in: Journal of experimental psychology : learning, memory, and cognition. This article may not exactly replicate the final version published in the APA journal. It is not the copy of record. http://www.apa.org/pubs/journals/xlm/

Files


You might also like


Downloadable Citations