kNNs and Hyperparameter Optimization

Teaching Faculty Candidate Seminar

Zoom link for remote participants

Abstract:  Nearest neighbor classifiers form a rich yet intuitive machine learning model with many interesting and illuminating properties. We’ll begin with a simple demonstration of how such a model performs inference (with no training!) and then dive deeper into the various components of the model, including its underlying inductive bias and some (high-level) theoretical results. This will motivate a broader discussion of one of the most important ideas in machine learning: hyperparameter optimization. Hyperparameter optimization (and the closely related task of model selection) seeks to answer the fundamental question of how one can make critical design choices about machine learning models in a principled way. Time-permitting, we will conclude with a brief meta-discussion of how the techniques on display in this teaching demonstration translate to real courses/classrooms.

Bio:  Henry Chai is an assistant teaching professor in the machine learning department at Carnegie Mellon University. He primarily teaches and refines the department’s various introductory machine learning courses while also developing novel courses on topics such as generative AI and Bayesian machine learning. In addition, he conducts pedagogical research on the practice of teaching, using quantitative and qualitative research methods to develop evidence-based best practices in the classroom. He received his PhD in 2021 from Washington University in St. Louis where he was advised by Dr. Roman Garnett. His dissertation work lives at the intersection of Bayesian machine learning, probabilistic numerics, and active learning and can be concisely summarized by the following question: how can we efficiently and accurately reason about inherently intractable quantities?