Last Fridays Talks: Speech & Language

Last Fridays Talks

Each last-Friday-of-the-month, we are hosting the Last Fridays Talks, where one of our seven Collaboratories will present insights from their current work.

Join us for a discussion on results and recent papers, followed by some socializing afterwards for everyone who wish to attend. 

Talk 1

Designing Human-Centred AI for Explainable and Reliable Information Systems by Greta Warren, Postdoctoral Researcher in the Natural Language Processing section, Department of Computer Science, University of Copenhagen, and P1 Affiliate.

Abstract

Artificial Intelligence (AI) systems play an increasingly central role in the information we consume, produce, and share. However, generative AI models (e.g., large language models) and their ability to accelerate the spread of misinformation pose significant risks to people’s ability to access reliable information vital for societies to function, while the underlying processes of these models are opaque and uninterpretable to their users. 

In this talk, I will present my research on designing, developing and evaluating explainable human-centred AI systems that support people’s reasoning and agency in identifying reliable information. I will discuss several recent works on explainable AI methods for fact-checking, from (1) identifying stakeholders' needs through qualitative research with professional fact-checkers and journalists, to (2) developing a framework for generating fact-checking explanations that meet these criteria, and (3) evaluating these methods through large-scale mixed-methods experiments. I will conclude by reflecting on implications for designing effective human-centred AI systems and outlining my vision for future research on human-centred AI to support human reasoning in complex information-seeking and decision-making tasks. 

Bio

Greta Warren is a Postdoctoral Researcher at the University of Copenhagen in the Natural Language Processing section in the Department of Computer Science. She is also affiliated with the Danish Pioneer Centre for Artificial Intelligence. Her work in human-AI interaction draws from artificial intelligence, human-computer interaction, and cognitive psychology and examines how explainable AI systems can be designed to support human reasoning in complex information-seeking and decision-making tasks, such as identifying misinformation.

Her current work examines human-centred explanations for automated fact-checking, as part of the ExplainYourself project, funded by the European Research Council. She holds a Ph.D. in Computer Science from University College Dublin and a B.A. (Hons) in Psychology from Trinity College, Dublin. Webpage:  https://gretawarren.github.io/

Talk 2

The Illusion of Generalization in Tabular Language Models by Ratish Puduppully, Assistant Professor in the NLP group, IT University of Copenhagen.

Abstract

Large language models are increasingly applied to tabular prediction tasks through serialization and fine-tuning, with impressive benchmark results. This talk reveals why those results are misleading. We re-evaluate a state-of-the-art tabular language model on a standard benchmark of 165 datasets and find three critical problems. First, binary and multiclass classification tasks barely surpass majority-class baselines, with strong aggregate performance driven by binning tasks that exploit shortcuts in task formulation. Second, top-performing datasets show extensive train-test contamination that standard detection methods fail to catch. Third, fine-tuning a base LLM on general instructions (Alpaca) with no tabular data closes most of the performance gap. I conclude with recommendations for better evaluation of tabular LMs.

Bio

Ratish Puduppully is an Assistant Professor in the NLP group at IT University of Copenhagen. His research focuses on structured data understanding, multilingual NLP, and long context modeling in language models. He received his PhD from the University of Edinburgh, where his thesis on table-to-text generation with neural planning won the SICSA Best Dissertation Award in Informatics in Scotland.