Human-AI Systems for Visual Information Access

In my work, I create hybrid human- and AI-powered intelligent interactive systems to provide access to visual information in the real world. By combining the advantages of humans and AI, these systems can be nearly as robust and flexible as humans, and nearly as quick and low-cost as automated AI, enabling us to solve problems that are currently impossible with either alone.

I developed and deployed human-AI systems for two application domains: accessibility and environmental sensing. To make physical interfaces accessible for blind people, I developed systems to interpret static and dynamic interfaces, enabling blind people to independently access them through audio feedback or tactile overlays. For environmental sensing, I developed and deployed a camera sensing system that collects human labels to bootstrap automatic processes to answer real-world visual questions, allowing end users to actionalize AI in their everyday lives.

AI systems often require huge amount of up front training data to get started, but targeted human intelligence can bootstrap the systems with relatively little data. Although humans may be slower initially, quickly bootstrapping to automated approaches provides a good balance, enabling human-AI systems to be scalable and rapidly deployable.

Bio: Anhong Guo is a Ph.D. candidate in the Human-Computer Interaction Institute in the School of Computer Science at Carnegie Mellon University, advised by Dr. Jeffrey Bigham. He is also a Snap Inc. Research Fellow, and a Swartz Innovation Fellow for Entrepreneurship. He has published in many top academic conferences in interface technologies, wearable computing, accessibility and computer vision, including two best paper nominees. Before CMU, he received his Master’s in HCI from Georgia Tech. He has also worked in the Ability and Intelligent User Experiences groups in Microsoft Research, the HCI group of Snap Research, the Accessibility Engineering team at Google, and the Mobile Innovation Center of SAP America. See more at: https://guoanhong.com