Prediction of events in human language processing and pretrained language models

We are pleased to announce a new paper published by Philippe Blache (LPL) in collaboration with researchers from Hong Kong Polytechnic University (Procore Project) and Purdue University:

Reference: James Britton, Yan Cong, Yu-Yin Hsu, Emmanuele Chersoni, Philippe Blache. On the influence of discourse connectives on the predictions of humans and language models. Frontiers in Human Neuroscience, 2024, 18, pp.1363120. ⟨10.3389/fnhum.2024.1363120⟩. ⟨hal-04717106

Full-text article: https://doi.org/10.3389/fnhum.2024.1363120

Abstract:
This paper investigates how humans and language models process event predictions in sentences, focusing on the role of discourse connectives like and, but, and because. Humans find congruent sentence sequences easier to understand, and these connectives help clarify event relationships, especially when predictions need to be reversed (e.g., concessives and contrastives). The study used Italian and Mandarin Chinese story stimuli to test plausibility and coherence with or without connectives. Results show Mandarin language models are somewhat sensitive to these factors but struggle with prediction reversals, while Italian models showed no significant alignment with human data.

 

Credits: Authors

A neuro-cognitive model of comprehension based on prediction and unification

We are pleased to announce the publication of the latest article by Philippe Blache, a CNRS researcher at the LPL, in the journal Frontiers in Human Neuroscience, dedicated to the language model also discussed at the conference held at the Collège de France last February:

Reference: Philippe Blache. A neuro-cognitive model of comprehension based on prediction and unification. Frontiers in Human Neuroscience, 2024, 18.  

Full text article: https://doi.org/10.3389/fnhum.2024.1356541

 

Credits: Ph. Blache

Prediction is to understand: a neuro-cognitive model of language based on prediction

We are pleased to announce the seminar “Prediction is to understand: a neuro-cognitive model of language based on prediction” given by Philippe Blache, research director at the LPL, this Friday 2 February at the Collège de France.

 This talk is as part of the seminar series entitled “Teaching languages to machines” and will be preceded by a lecture on “Multimodalities: NLP and images, NLP and speech” by Benoît Sagot (Annual Chair in Computer Science and Digital Sciences in partnership with Inria).

 The seminar will take place face-to-face, but will also be broadcast on the Collège de France website.

 Link (French): https://www.college-de-france.fr/fr/agenda/seminaire/apprendre-les-langues-aux-machines

______________

Update of February 7th!

The seminar is available now online on https://www.college-de-france.fr/fr/agenda/seminaire/apprendre-les-langues-aux-machines/predire-est-comprendre-un-modele-neuro-cognitif-du-langage-fonde-sur-la-prediction

 

Illustration copyright: Pieter Bruegel the Elder, The (great) tower of Babel, about 1563, Kunsthistorisches Museum, Vienna (Austria) - public domain