Refreshments are available starting at 10:30 a.m. The seminar will begin at 10:45 a.m.
Abstract
Current artificial intelligence systems can synthesize images, solve math problems, and write code. Despite these advances, they still struggle with basic tasks that humans and animals perform effortlessly. One potential idea why humans and animals can perform basic tasks well is because they have a predictive world model that integrates perception, reasoning, and planning capabilities. Can we build such a model in a bottom-up fashion from sensorimotor data and primarily visual observations?
In this talk, Amir Bar will propose a path toward building such a world model. He will introduce Visual Prompting, a new paradigm that unifies many computer vision tasks and can readily adapt pre-trained models to novel tasks without fine-tuning. Building on this, Bar will present an extension to planning using generative world models—showing that action-conditioned video models can act as simulators of the environment that support real-world decision-making, with a case study in visual navigation. Finally, Bar will discuss future directions for improving the capabilities of world models and the challenges we face to enable their real-world deployment.
Speaker Biography
Amir Bar is a postdoctoral researcher at Meta AI, working on self-supervised learning with Yann LeCun. Previously, he completed his PhD at Tel Aviv University and was a visiting PhD student at the University of California, Berkeley’s Artificial Intelligence Research Lab, where he was advised by Amir Globerson and Trevor Darrell. Bar began his PhD following the acquisition of the startup Zebra Medical Vision, where he led the AI team and developed multiple FDA-approved algorithms currently in clinical use worldwide. His work on video models won the Ego4D PNR Temporal Localization Challenge at the 2022 Conference on Computer Vision and Pattern Recognition.