Learning to See the 3D world

Abstract: Cameras allow us to effortlessly capture the visual world around us and share memorable moments of our lives. While current computer vision systems work remarkably well on recognizing 2D patterns in images, they often have difficulty recovering the complete 3D geometry of dynamic scenes. On the other hand, humans can perceive complex dynamic scenes in terms of physical surfaces, objects, and scenes in 3D and imagine plausible scene appearance from novel viewpoints. In this talk, I will present my research on reconstructing and rendering our 3D world. Specifically, I will present our work on creating compelling 3D photography from a single image, estimating dense, geometrically consistent depth from casually captured cellphone videos, and learning neural implicit representations for free-viewpoint videos. The core idea is to integrate constraints from the physical model behind the visual observations into learning-based algorithms. I will highlight future challenges and plans for building intelligent machines that learn to see, recreate, and interact with the 3D world.

Bio: Jia-Bin Huang is an Assistant Professor in the Bradley Electrical and Computer Engineering at Virginia Tech. He received his Ph.D. degree from the Department of Electrical and Computer Engineering at the University of Illinois, Urbana-Champaign. His research interests include computer vision, computer graphics, and machine learning with a focus on visual analysis and synthesis with physically grounded constraints. His research received the best student paper award in IAPR International Conference on Pattern Recognition (ICPR) and the best paper award in the ACM Symposium on Eye Tracking Research & Applications (ETRA). Huang is the recipient of the Dissertation Completion Fellowships, Thomas & Margaret Huang award from UIUC, NSF CRII award, Samsung Global Outreach Award, 3M non-tenured faculty award, and a Google faculty research award.