Telling the Story of an Image

For both humans and machines, the ability to learn and recognize the semantically meaningful contents of the visual world is an essential and important functionality. Our works in human psychophysics and fMRI experiments have shown, along with a number of seminal works by others, that the human visual system possesses an amazing ability in perceiving objects, scenes and human social activities in a brief glance of the picture. Such findings have inspired us to build computer vision algorithms that could perform similar tasks. In particular, we hope our algorithms can recognize both the individual components as well as the holistic semantic meaning of an image. In this talk, we begin by showing that low-level image patches and their occurrence statistics can provide a rich description for generic natural scene categorization. This is done by using a hierarchical generative graphical model. I will then present a number of works to tackle the problem of generic objects classification, particularly the problem of one-shot learning of object categories. We also show how non-parametric Dirichlet Process latent topic model can be naturally cast into an incremental learning scheme for learning and filtering online images. Putting together objects and scenes, we introduce a recent algorithm as a first attempt for an integrated and holistic understanding of images depicting an assortment of human activities. We use a newly collected and annotated sport event dataset for validating our results. In our works mentioned above, we emphasize on the designs of the model representation and the usage of generative models. We take advantage of the Bayesian formulations in learning and inference so that our models can be trained with little human supervision, with contaminated datasets, and under one-shot learning or incremental learning scenarios.
Prof. Fei-Fei Li's main research interest is in vision, particularly high-level visual recognition. In computer vision, Fei-Fei’s interests span from object and natural scene categorization to human activity categorizations in both videos and still images. In human vision, she has studied the interaction of attention and natural scene and object recognition, and decoding the human brain fMRI activities involved in natural scene categorization by using pattern recognition algorithms. Fei-Fei graduated from Princeton University in 1999 with a physics degree, and a minor in engineering physics. She received her PhD in electrical engineering from the California Institute of Technology in 2005. From 2005 to the end of 2006, Fei-Fei was an assistant professor in the Electrical and Computer Engineering Department at University of Illinois Urbana-Champaign. She is currently an Assistant Professor in the Computer Science Department at Princeton University. She also holds courtesy appointments in the Psychology Department and the Neuroscience Program at Princeton. She is a recipient of the 2006 Microsoft Research New Faculty Fellowship. (Fei-Fei publishes under the name L. Fei-Fei.)