Understanding User Cognition: from Spatial Ability to Code Writing and Review

Virtual dissertation defense

ABSTRACT: Understanding how developers carry out computer science activities can help to improve software engineering productivity and guide the use and development of supporting tools and environments. Previous studies have explored how programmers conduct computing activities, but they rely on traditional survey instruments, which may not be reliable in all contexts. Advances in medical imaging and eye-tracking have recently been applied to software engineering, paving the way for grounded neurobiological understanding of fundamental cognitive processes involved therein.

In this thesis, I will present a systematic solution to objectively measure and understand user cognition in software engineering activities by introducing three research components that range from foundational to high-level tasks in modern software engineering. I will also present the design and methodology for studying cognitive processes in software engineering with multiple state-of-the-art psycho-physiological measures, including fMRI, fNIRS and eye tracking, and discuss the implications. First, I will introduce the examination of the relationship between data structure manipulations and spatial ability. Then, I will discuss the investigation of cognitive processes in more semantically-rich and industry-related activities including code writing and code review, and also present the investigation of biases and differences across different user groups.

Among the contributions, the work of this thesis is one of the first that leverages various objective measures to provide a systematic solution for understanding user cognition in programming activities. With the experimental design, mathematical models and analysis, it provides basic principles and guidelines for future research to adapt multiple psychophysiological measures to software engineering. This thesis also shows potential for broad impact in CS pedagogy, technology transfer, and broadening participation.