Abstract:
Different phenomena play a central role in the development, maintenance, and quality of conversations. Among them, feedback is known to play a central role. This thesis proposes an experimental and theoretical contribution to the description of listeners’ behavior and their feedback production. Based on an interdisciplinary approach, three main axes are developed.
The first axis focuses on the creation and annotation of different corpora, including one created specifically for the needs of the thesis: the SMYLE corpus, comprising audiovisual and neurophysiological data. SMYLE involves 30 participant dyads engaged in narrating three stories, followed by fifteen minutes of free conversation. The narration task includes two experimental conditions: an attentive listener condition and a distracted listener condition.
The second axis addresses computational modeling by developing a hierarchical model for predicting feedback, multimodal and interpretable. Next, we propose a theoretical feedback model using a feature structure representation. This model aims to describe feedback in terms of formand function while studying their interactions.
The third axis presents an experimental work. A first study explores the perception of feedback based on its production delay. A second study analyzes the impact of listener distraction on feedback production and interaction quality. Finally, a third exploratory study aims to define listening styles, showing alignment of styles among interlocutors.