Hawes Hall, Classroom 101, Harvard Business School
Enhancing Efficiency and Robustness for Switchback Experiments: A Practical Model-assisted Design and Analysis
Switchback experiments are increasingly employed when traditional A/B testing is impractical due to constraints such as user randomization limitations or interference among experimental units. In the presence of carryover effects, however, existing designs often face challenges like a low statistical power, a random number of collected observations, and limited practical applicability. This talk presents a novel model-assisted design framework for switchback experiments that guarantees a fixed number of valid observations, enabling more reliable and consistent experimentation. The proposed design introduces an adaptive mechanism that uniformly distributes observations across seasonal cycles, ensuring treatment-control balance and reducing sensitivity to peak periods. Additionally, the design incorporates refinements that adapt assignments more to past periods, creating temporal coupling between treatments and controls to enhance robustness, even in less predictable dynamic systems. We provide a comprehensive analysis of the design’s bias and variance and develop tools to support statistical inference. Numerical studies, including a realistic ride-sharing simulation, demonstrate that the proposed design achieves superior accuracy and efficiency compared to existing approaches. This work contributes broadly to the field of experimental design under interference, bridging practical considerations with robust statistical methodologies.
The paper can be found here.
Hossein Alidaee will be the discussant.
Iavor Bojinov
Associate Professor of Business Administration
Iavor Bojinov is an Associate Professor of Business Administration at Harvard Business School. He is the co-PI of the AI and Data Science Operations Lab and a faculty affiliate in the Department of Statistics at Harvard University and the Harvard Data Science Initiative. His research focuses on developing novel statistical methodologies to make business experimentation more rigorous, safer, and efficient, specifically homing in on the application of experimentation to the operationalization of artificial intelligence (AI), the process by which AI products are developed and integrated into real-world applications. His work has been published in top academic journals such as Annals of Applied Statistics,Biometrika, The Journal of the American Statistical Association, The Journal of Econometrics,Quantitative Economics, Management Science, and Science, and has been cited in Forbes, The New York Times, The Washington Post, and Reuters, among other outlets. More broadly, as one of the few scholars who work at the intersection of data science and business, he was the first author to have spotlight featured articles in both the Harvard Business Review and the Harvard Data Science Review.
Professor Bojinov is also the co-creator of the first-year required MBA course “Data Science for Managers” and has previously taught the “Competing in the Age of AI” and “Technology and Operations Management” courses. Before joining Harvard Business School, Professor Bojinov worked as a data scientist leading the causal inference effort within the Applied Research Group at LinkedIn. He holds a Ph.D. and an MA in Statistics from Harvard and an MSci in Mathematics from King’s College London.