Distinguished Seminar Series on Data Science & Artificial Intelligence - "Exploring Knowledge Learning from Video Generative Modeling" by Dr Jiashi FENG

Current AI models primarily acquire knowledge through generative text modeling, as exemplified by large language models. However, videos, which contain rich information such as visual reasoning and task execution, remain largely underexplored. These modalities offer complementary knowledge that text alone cannot capture.

In this talk, I will present our recent efforts to leverage video generative modeling for learning from video data. First, I will introduce VideoWorld, a generative video learning framework designed to extract visual reasoning and planning capabilities from unlabeled videos. This model has demonstrated applicability in tasks such as playing Go and robotic manipulation. Next, I will discuss our recent findings on the limitations of video generative models when used as world models. Finally, I will conclude with a discussion on existing challenges and outline potential future directions.