Michael Ekstrand

Continuous Automated Reproducibility

Back to Publications

Michael D. Ekstrand. 2026. Continuous Automated Reproducibility: Practical Tools and Workflows for Reproducible Research Workflows. Tutorial to be presented at Proceedings of the 20th ACM Conference on Recommender Systems (RecSys '26), Sep 28–Oct 2, 2026.

Nada (Roddy Piper) in John Carpenter’s They Live, standing in the bank lobby holding a shotgun and wearing sunglasses. The caption reads “I have come here to chew bubblegum and reproduce research.”

Abstract

While the RecSys community has been promoting and discussing the importance of reproducible research for many years, there remains a significant lack of practical resources to help RecSys researchers and practitioners, particularly those just getting started in the field, to actually build reproducible experiments and research workflows.

This hands-on tutorial intends to fill that gap. Based on the idea of “continuous reproducibility”, I will present open-source tools and practical experimental design and implementation techniques to build recommender systems experiments that are reproducible from Day 1. This tutorial will be useful for researchers at various stages who want to make their work more reproducible, as well as to industry practitioners and researchers who want to build reliable, adaptable, and reproducible pipelines top train and evaluate recommender systems for production applications.