The seminar will begin at 12:00 p.m. Refreshments will be available starting at 1:15 p.m.
Abstract
Recent advances in large language models have enabled them to process texts exceeding 100,000 tokens in length, fueling demand for long-form language processing tasks such as the summarization or translation of books. However, LLMs struggle to take full advantage of the information within such long contexts, which contributes to factually incorrect and incoherent text generation. In this talk, Mohit Iyyer will first demonstrate an issue that plagues even modern LLMs: their tendency to assign high probability to implausible long-form continuations of their input. He will then describe a contrastive sequence-level ranking model that mitigates this problem at decoding time and that can also be adapted to the reinforcement learning from human feedback alignment paradigm. Next, he will consider the growing problem of long-form evaluation: As the length of the inputs and outputs of long-form tasks grows, how do we even measure progress (via both humans and machines)? He proposes a high-level framework that first decomposes a long-form text into simpler atomic units before then evaluating each unit on a specific aspect. He demonstrates the framework’s effectiveness at evaluating factuality and coherence on tasks such as biography generation and book summarization. He will also discuss the rapid proliferation of LLM-generated long-form text, which plagues not only evaluation (e.g., via Mechanical Turkers using ChatGPT to complete tasks) but also society as a whole, and he will describe novel watermarking strategies to detect such text. Finally, he will conclude by discussing his future research vision, which aims to extend long-form language processing to multilingual, multimodal, and collaborative human-centered settings.
Speaker Biography
Mohit Iyyer is an associate professor in computer science at the University of Massachusetts Amherst, with a primary research interest in natural language generation. He is the recipient of Best Paper Awards at the 2016 and 2018 Annual Conferences of the North American Chapter of the Association for Computational Linguistics, an Outstanding Paper Award at the 2023 Conference of the European Chapter of the Association for Computational Linguistics, and a Best Demo Award at the 2015 Conference on Neural Information Processing Systems; he also received the 2022 Samsung AI Researcher of the Year award. Iyyer obtained his PhD in computer science from the University of Maryland, College Park in 2017 and spent the following year as a researcher at the Allen Institute for AI.