Talk by Emily Cheng

Title

Processing complexity as a signature of linguistic meaning construction in minds and machines.

Abstract

How do LLMs and the brain wrangle a system as complex as language? In this talk, I will present some results on the representational complexity of language processing. I show that, when processing words in context, an increase in the dimension of neural representation manifolds indexes abstract meaning construction in LLMs and the brain. This observation holds both over the processing timecourse of a word, as well as over the course of training for LLMs. For both LLMs and the brain, surprisal--- a common psycholinguistic measure of word processing difficulty-- predicts when dimensionality peaks during word processing. That is, harder words tend to recruit dimensions later in the processing timecourse. Overall, results suggest that neural representational geometry can signal meaningful behavior in information processing systems.

Speaker

Emily Cheng is a 4th year PhD student with Marco Baroni at the Universitat Pompeu Fabra, where she studies computational linguistics. Her research focuses on linguistic structure, its potential causes, and its representation in humans and large language models.