Baseball Umpire Calls as a Naturally Occurring Data Source for Revealing Principles of Bias and Learning in Perceptual Judgments
The very expertise with which experimental psychologists wield their tools for achieving laboratory control may have had the unwelcome effect of blinding us to the possibilities of discovering principles of behavior not by conducting experiments but rather by analyzing naturally occurring data sets. Uncovering principles of psychology by analyzing naturally occurring data is an exciting endeavor because of 1) the rise of well curated and large data sets involving collections of tagged images, text corpora, Wikipedia edit histories, trends in Twitter tag usage, demographics, consumer product sales, patent use and dependencies, sporting event outcomes, scientific citations, etc., 2) novel analytic methods for inferring causal relations from observational data, 3) the data often come from strongly motivated decisions and life-changing behaviors of social importance, and 4) the data sets allow us to explore the interplay between internal psychological processes and external environments, artifacts, and social institutions.
As a case study of harvesting naturally occurring data to reveal psychological principles, I will describe my collaboration with Brian Mills to study home plate umpire calls of strikes and balls. Major League Baseball home plate umpires have collectively made millions of professional pitch calls, and these calls can be compared to trajectory information recorded since 2008 for each pitch using tracking technology that is accurate to within 1 MPH and 1 inch. Furthermore, pitch, umpire, and game data are publicly available and relatively easily scraped using modern analysis tools. Using this data, we characterize how umpires' perceptual judgments are influenced by situational factors and their own experience making calls. We fit a parametric model to account for variation in judgment policies in terms of a strike zone's horizontal and vertical center, shape, and sharpness, and the umpire's guessing probability and bias.
Robert Goldstone is Distinguished Professor in the Psychological and Brain Sciences department and Cognitive Science program at Indiana University. His research interests include concept learning and representation, perceptual learning, educational applications of cognitive science, decision making, collective behavior, and computational modeling of human cognition. He won the 2000 APA Distinguished Scientific Award for Early Career Contribution to Psychology, and a 2004 Troland research award from the National Academy of Sciences. He was the executive editor of Cognitive Science from 2001-2005. He has been elected as a fellow of the Society of Experimental Psychologists, the Cognitive Science Society, and the American Academy of Arts and Sciences.