Adaptive Design of Switchback Experiments: A Markov Chain Perspective

Abstract: Suppose an online platform wants to compare a treatment and control policy, e.g., two different matching algorithms in a ridesharing system, or two different inventory management algorithms in an online retail site. Standard randomized controlled trials are typically not feasible, since the goal is to estimate policy performance on the entire system. Instead, the typical current practice involves dynamically alternating between the two policies for fixed lengths of time, and comparing the average performance of each over the intervals in which they were run as an estimate of the treatment effect, called a “switchback” design.

In this talk, we cast switchback design optimization as a problem of estimating the difference in steady state reward between two Markov chains: a “control” chain and a “treatment” chain.  We discuss the rich structure exhibited by this formulation of the problem, including reduction of the variance optimal design to a convex optimization problem, in the case where parametric or nonparametric maximum likelihood is used for estimation.  We’ll close with a discussion of open issues and discussion of alternative experimental designs of interest.

Joint work with Jose Blanchet, Peter Glynn, Mohammad Rasouli, and Linjia Wu.

Speaker Bio: Ramesh Johari is a Professor at Stanford University, with a full-time appointment in the Department of Management Science and Engineering (MS&E), and courtesy appointments in the Departments of Computer Science (CS) and Electrical Engineering (EE). He is a member of the Operations Research group and the Social Algorithms Lab (SOAL) in MS&E, the Information Systems Laboratory in EE, and the Institute for Computational and Mathematical Engineering.  He is also an Associate Director of Stanford Data Science.  He received an A.B. in Mathematics from Harvard, a Certificate of Advanced Study in Mathematics from Cambridge, and a Ph.D. in Electrical Engineering and Computer Science from MIT.  He served as co-chair of the ACM Economics and Computation (EC) program committee in 2019, and he is an Area Co-Editor of the Revenue Management and Market Analytics Area for Operations Research, and associate editor for Management Science (in the Stochastic Models and Simulation area) and Stochastic Systems.  His research interests are in online platform and marketplace design, experimentation and data science for online platforms, and (more recently) application of these techniques to personalized health care via telemedicine.

Join Zoom Meeting https://umich.zoom.us/j/97598571292

Meeting ID: 975 9857 1292

Passcode: XXXXXX (Will be sent via email to attendees)

Zoom Passcode information is also available upon request to Shelly (Michele) Feldkamp ([email protected]).