Abstract:
While neuroimaging and behavioral studies have shown that sensory-motor systems are recruited during semantic processing, how and when this occurs has not yet been clearly established. Our purpose was to observe the different contexts in which motor activation can contribute to language comprehension and learning, using interactive and ecologically valid environments. In our first study, novice learners acquired a reduced second language (L2) lexicon through interactive computer games. Behavioral and electroencephalography (EEG) results indexed rapid L2 word learning. Interestingly, even-related potential (ERP) results revealed a gender congruency effect such that only words that had the same grammatical gender across participants’ L1 and L2 gave rise to an N400 effect for match vs mismatch auditory word and image pairs, indicating that these words were better encoded. In a second study, we used an action-sentence compatibility effect (ACE) paradigm to evaluate how motor preparation affects language processing. ERP results showed greater N400 amplitude for congruent compared to incongruent action-sentence trials, suggesting that compatibility between motor and language processes produced interference. In studies 3 and 4, we combined virtual reality (VR) and EEG to investigate interactions between language processing and motor activation. In the first of these studies, participants heard action verbs in their native language and performed varied actions on a virtual object in a Cave automatic virtual environment (CAVE) during a Go-Nogo task. Time frequency analysis showed motor activation for both Go and Nogo conditions during action verb processing and prior to movement proper. In addition, greater motor activation for Go versus Nogo trials. Our final (projected) study is a registered report that aims to determine the neural correlates of embodied L2 learning by having participants encode auditory action verbs using an interactive virtual reality head-mounted display system and specific real-life actions on a virtual object. Using behavioral and EEG measures in a pre-post training design, this condition will be compared to a control condition in which participants will simply point to the virtual object.
The body of the work reported in this dissertation represents a significant step towards better understanding the subtle relationship between motor and semantic processes. By making use of new technologies that allow for manipulating and controlling the environment, our work opens up fresh perspectives for taking into account the contextual nature of how we learn and understand language.
