Other Recommender Research

In addition to the various recommender systems projects I lead, I have also been involved in several side projects with other collaborators.

Surveys and Position Papers

I have written or co-authored a few general recommender system papers, including our book chapter and survey for Foundations and Trends in HCI:


Daniel Kluver, Michael D. Ekstrand, and Joseph A. Konstan. 2018. Rating-Based Collaborative Filtering: Algorithms and Evaluation. In Social Information Access. Peter Brusilovsky and Daqing He, eds. Springer-Verlag, Lecture Notes in Computer Science vol. 10100, pp. 344–390. DOI 10.1007/978-3-319-90092-6_10. ISBN 978-3-319-90091-9. Cited 93 times. Cited 120 times.


Michael D. Ekstrand, John T. Riedl, and Joseph A. Konstan. 2011. Collaborative Filtering Recommender Systems. Foundations and Trends® in Human-Computer Interaction 4(2) (February 2011), 81–173. FnT HCI 4(2) (February 2011). DOI 10.1561/1100000009. Cited 632 times. Cited 1530 times.

And position papers on recommender systems research and development, either generally or applied to particular areas:


Michael D. Ekstrand and Joseph A. Konstan. 2019. Recommender Systems Notation: Proposed Common Notation for Teaching and Research. Computer Science Faculty Publications and Presentations 177. Boise State University. DOI 10.18122/cs_facpubs/177/boisestate. arXiv:1902.01348. Cited 4 times. Cited 9 times.


Michael D. Ekstrand. 2017. Challenges in Evaluating Recommendations for Children. In Proceedings of the International Workshop on Children & Recommender Systems (KidRec), at RecSys 2017. Proc. KidRec. Cited 8 times.


Jennifer D. Ekstrand and Michael D. Ekstrand. 2016. First Do No Harm: Considering and Minimizing Harm in Recommender Systems Designed for Engendering Health. In Proceedings of the Workshop on Recommender Systems for Health at RecSys ’16. Cited 10 times. Cited 11 times.


Michael D. Ekstrand and Martijn C. Willemsen. 2016. Behaviorism is Not Enough: Better Recommendations through Listening to Users. In Proceedings of the Tenth ACM Conference on Recommender Systems (RecSys ’16, Past, Present, and Future track). ACM. Proc. RecSys ’16 (Past, Present, and Future track). DOI 10.1145/2959100.2959179. Acceptance rate: 36%. Cited 81 times. Cited 100 times.

Additional position papers can be found under Reproducible Research.

Rating Interfaces

In this project, led by Tien Nguyen and Daniel Kluver, we examined different interfaces for improving the process of rating movies by giving the user additional information to help guide their rating. We tried several things:

  • Showing the user tags related to the movie, to help them recall its characteristics.
  • Movies similar to the movie to rate for each of the valid rating values, to provide the user with a point of reference.
  • Combining these two interfaces.

The result was published in RecSys 2013.


Tien T. Nguyen, Daniel Kluver, Ting-Yu Wang, Pik-Mai Hui, Michael D. Ekstrand, Martijn C. Willemsen, and John Riedl. 2013. Rating Support Interfaces to Improve User Experience and Recommender Accuracy. In Proceedings of the 7th ACM Conference on Recommender Systems (RecSys ’13). ACM. Proc. RecSys ’13. DOI 10.1145/2507157.2507188. Acceptance rate: 24%. Cited 42 times. Cited 56 times.

Information Content of Ratings

In this project, led by Daniel Kluver and Tien Nguyen, we attempt to quantify how much information (in the Shannon information theory sense) is contained in a rating of a movie, and use this as the basis for comparing different rating interfaces based on their efficiency (bits per second).

One of the particularly fun developments in this paper is an experimental protocol for estimating a lower bound on the mutual information between ratings and the preference constructs the user’s brain, allowing us to reason about the amount of information about preference, not just information, is in a rating. Unfortunately, this protocol requires a ridiculous number of users to achieve any kind of power, but it’s a very nice theoretical development in my opinion.


Daniel Kluver, Tien T. Nguyen, Michael Ekstrand, Shilad Sen, and John Riedl. 2012. How Many Bits per Rating?. In Proceedings of the Sixth ACM Conference on Recommender Systems (RecSys ’12). ACM, pp. 99–106. Proc. RecSys ’12. DOI 10.1145/2365952.2365974. Acceptance rate: 20%. Cited 40 times. Cited 41 times.

Database-Embedded Recommenders

This project, led by Justin Levandoski, embedded recommender technology into an SQL database.


Justin J. Levandoski, Mohamed Sarwat, Mohamed F. Mokbel, and Michael D. Ekstrand. 2012. RecStore: An Extensible And Adaptive Framework for Online Recommender Queries Inside the Database Engine. In Proceedings of the 15th International Conference on Extending Database Technology (EDBT ’12). ACM, pp. 86–96. Proc. EDBT ’12. DOI 10.1145/2247596.2247608. Acceptance rate: 23%. Cited 15 times. Cited 18 times.