Sequential nonparametric inference by betting

Abstract:

Sequential inference methods refer to statistical procedures in which the sample size (and possibly other design choices) are not decided apriori, but instead depend on the observations collected. Such methods often lead to statistical or computational advantages over their fixed sample size or batch counterparts. In this talk, I will describe a general framework for constructing powerful sequential methods for some nonparametric sequential inference problems.

I will begin by introducing an important nonparametric testing problem, called two-sample testing, and describe a framework for constructing powerful sequential two-sample tests based on the principle of testing-by-betting. The key idea is to reframe the task of sequential testing into that of selecting payoff functions that maximize the wealth of a fictitious bettor, betting against the null in a repeated game. To design the payoff functions, I will describe a simple strategy that proceeds by constructing predictable estimates of the witness function associated with a class of integral probability metrics (IPMs). The statistical properties of the resulting test can then be characterized in terms of the regret of this prediction strategy. I will then instantiate the general testing strategy for some popular IPMs; such as the Kolmogorov-Smirnov metric, and the kernel-MMD metric. This framework for constructing two-sample tests is quite general, and I will describe how it can be applied to a much larger class of testing problems, and to other inference tasks such as estimation, and change-detection. To conclude the talk, I will briefly discuss my plans for future work.

Bio:

Shubhanshu Shekhar is a postdoctoral researcher in the Department of Statistics and Data Science at Carnegie Mellon University working with Prof. Aaditya Ramdas. Prior to this, he obtained his PhD in Electrical Engineering from the University of California, San Diego, where he was advised by Prof. Tara Javidi. His research interests lie broadly in the areas of machine learning and nonparametric statistics.