Abstract
Language models are highly effective at understanding and generating text, holding immense potential as intuitive, personalized interfaces for accessing information. Expanding their ability to gather and synthesize large volumes of information will further unlock transformative applications, ranging from generative search engines to AI literature assistants. In this talk, Tianyu Gao will present his research on advancing LMs for information processing at scale. First, he will present his evaluation framework for LM-based information-seeking systems, emphasizing the importance of providing citations for verifying the model-generated answers. His evaluation highlights shortcomings in LMs’ abilities to reliably process long-form texts (e.g., dozens of webpages), which he addresses by developing state-of-the-art long-context LMs that outperform leading industry efforts while using a small fraction of the computational budget. Gao will then introduce his foundational work on using contrastive learning to produce performant text embeddings, which form the cornerstone of effective and scalable search. In addition to building systems that can process large-scale information, he will discuss his contributions to creating efficient pre-training and adaptation methods for LMs, which enable scalable deployment of LM-powered applications across diverse settings. Finally, Gao will share his vision for the next generation of autonomous information processing systems and outline the foundational challenges that must be addressed to realize this vision.
Speaker Biography
Tianyu Gao is a fifth-year PhD student in the Department of Computer Science at Princeton University, advised by Danqi Chen. His research focuses on developing principled methods for training and adapting language models, many of which have been widely adopted across academia and industry. Driven by transformative applications such as using language models as information-seeking tools, his work also advances robust evaluation and fosters a deeper understanding to guide the future development of language models. Gao led the first workshop on long-context foundation models at the 2024 International Conference on Machine Learning, won an outstanding paper award at the 2022 Annual Meeting of the Association for Computational Linguistics , and received an IBM PhD Fellowship Award in 2023. He received his BEng from Tsinghua University in 2020.