Ph.D (2014), Computer Science, University of Minnesota, Minneapolis, MN. Thesis: Towards Recommender Engineering: Tools and Experiments for Identifying Recommender Differences. Advisers: John T. Riedl and Joseph A. Konstan
B.S. (2007), Computer Engineering (With Distinction), Iowa State University, Ames, IA.
- Assistant Professor, Dept. of Computer Science, Boise State University
- Co-founder, People and Information Research Team (PIReT)
- Assistant Professor, Dept. of Computer Science, Texas State University
- Graduate Research Assistant, GroupLens Research, Dept. of Computer Science, University of Minnesota
- Su 2012, F 2013
- Instructor, University of Minnesota
- Summer 2010
- Research Intern, Autodesk Research, Toronto, CA
- 2007–2008, S 2011
- Teaching Assistant, University of Minnesota
- Undergraduate Research Assistant, Scalable Computing Laboratory, Ames Lab, Iowa State University
Boise State University
- Spring 2017
- CS 597 (Recommender Systems); 13 students
- Fall 2016
- CS 410 / CS 510 (Databases); 28 students
Texas State University
- Spring 2016
- CS 3320 (Internet Software Development); 48 students
- CS 5369Q/4379Q (Recommender Systems); 26 students
- Fall 2015
- CS 4332 (Introduction to Database Systems); 39 students
- Spring 2015
- CS 5369Q/4379Q (Recommender Systems); 28 students
- CS 4350 (Unix Systems Programming); 32 students
- Fall 2014
- CS 4332 (Introduction to Database Systems); 50 students
In addition, I have supervised several independent study students.
I co-created the Recommender Systems specialization on Coursera, along with its two previous single-class versions, with Joseph A. Konstan.
University of Minnesota
- Fall 2013
- CSCI 5980-1 (Introduction to Recommender Systems), co-taught with Joseph A. Konstan; also offered as a MOOC on Coursera.
- Summer 2012
- CSCI 1902 (Structure of Computer Programming II)
- Spring 2011
- CSCI 5125 (Collaborative and Social Computing), as teaching assistant
- CSCI 1902 (Structure of Computer Programming II), as teaching assistant (3 terms)
M.S. Students Supervised
- Vaibhav Mahant (M.S. 2016, Texas State University; thesis: Improving Top-N Evaluation of Recommender Systems)
- Sushma Channamsetty (M.S. 2016, Texas State University; thesis: Recommender Response to User Profile Diversity and Popularity Bias)
- Mohammed Imran R Kazi (M.S. 2016, Texas State University; thesis: Exploring Potentially Discriminatory Biases in Book Recommendation)
- Shuvabrata Saha (M.S. 2016, Texas State University; co-advised with Dr. Apan Qasem; thesis: A Multi-objective Autotuning Framework For The Java Virtual Machine)
Author formatting key:, , . Citation counts from Microsoft Academic via Microsoft Cognitive Services.
2016. Dependency Injection with Static Analysis and Context-Aware Policy. Journal of Object Technology 15, 1 (February 2016), pp 1:1–31. DOI:10.5381/jot.2016.15.5.a1.and .
2015. Teaching Recommender Systems at Large Scale: Evaluation and Lessons Learned from a Hybrid MOOC. Transactions on Computer-Human Interaction 22, 2, Article 10 (April 2015), 23 pages. DOI:10.1145/2728171., , , , and .
2011. Collaborative Filtering Recommender Systems. Foundations and Trends® in Human-Computer Interaction 4, 2 (February 2011), pp 81–173. DOI:10.1561/1100000009., , and .
Refereed Conference Publications
These are full papers published in peer-reviewed conference proceedings.
2017. Sturgeon and the Cool Kids: Problems with Top-N Recommender Evaluation. In Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference.and .
2015. Letting Users Choose Recommender Algorithms: An Experimental Study. In Proceedings of the Ninth ACM Conference on Recommender Systems (RecSys '15). ACM. DOI:10.1145/2792838.2800195. Acceptance rate: 21%., , , and .
2014. User Perception of Differences in Recommender Algorithms. In Proceedings of the Eighth ACM Conference on Recommender Systems (RecSys '14). ACM. DOI:10.1145/2645710.2645737. Acceptance rate: 23%., , , and .
2014. Teaching Recommender Systems at Large Scale: Evaluation and Lessons Learned from a Hybrid MOOC. In Proceedings of the First ACM Conference on Learning @ Scale (ACM L@S '14). ACM. DOI:10.1145/2556325.2566244. Acceptance rate: 37%., , , , and .
2013. Rating Support Interfaces to Improve User Experience and Recommender Accuracy. In Proceedings of the Seventh ACM Conference on Recommender Systems (RecSys '13). ACM. DOI:10.1145/2507157.2507188. Acceptance rate: 24%., , , , , , and .
2012. How Many Bits per Rating?. In Proceedings of the Sixth ACM Conference on Recommender Systems (RecSys '12). ACM, pp 99–106. DOI:10.1145/2365952.2365974. Acceptance rate: 20%., , , , and .
2012. RecStore: An Extensible And Adaptive Framework for Online Recommender Queries Inside the Database Engine. In Proceedings of the 15th International Conference on Extending Database Technology (EDBT '12). ACM, 86–96. DOI:10.1145/2247596.2247608. Acceptance rate: 23%., , , and .
2011. Rethinking The Recommender Research Ecosystem: Reproducibility, Openness, and LensKit. In Proceedings of the Fifth ACM Conference on Recommender Systems (RecSys '11). ACM, 133–140. DOI:10.1145/2043932.2043958. Acceptance rate: 27% (20% for oral presentation, which this received)., , , and .
2011. RecBench: Benchmarks for Evaluating Performance of Recommender System Architectures. Proceedings of the VLDB Endowment 4, 11 (August 2011), 911–920. Acceptance rate: 18%., , , , , and .
2011. Searching for Software Learning Resources Using Application Context. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). ACM, 195–204. DOI:10.1145/2047196.2047220. Acceptance rate: 25%., , , , and .
2010. Automatically Building Research Reading Lists. In Proceedings of the Fourth ACM Conference on Recommender Systems (RecSys '10). ACM, 159–166. DOI:10.1145/1864708.1864740. Acceptance rate: 19%., , , , , and .
2009. rv you're dumb: Identifying Discarded Work in Wiki Article History. In Proceedings of the 5th International Symposium on Wikis and Open Collaboration (WikiSym '09). ACM, 10 pp. DOI:10.1145/1641309.1641317. Acceptance rate: 36% (Selected as Best Paper).and .
These are short research papers published in conference proceedings. They are also peer-reviewed.
2017. Recommender Response to Diversity and Popularity Bias in User Profiles. Short paper in Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference.and .
2012. When Recommenders Fail: Predicting Recommender Failure for Algorithm Selection and Combination. Short paper in Proceedings of the Sixth ACM Conference on Recommender Systems (RecSys '12). ACM, 233–236. DOI:10.1145/2365952.2366002. Acceptance rate: 32%.and .
Workshop papers, position papers, posters, workshops, demos, and other publications. These have typically undergone some review, but are not fully-refereed scientific publications.
2017. The Demographics of Cool: Popularity and Recommender Performance for Different Groups of Users. In RecSys 2017 Poster Proceedings.and .
2017. Challenges in Evaluating Recommendations for Children. In Proceedings of the International Workshop on Children & Recommender Systems (KidRec) at RecSys 2017.
2017. The FATREC Workshop on Responsible Recommendation. In Proceedings of the 11th ACM Conference on Recommender Systems.and .
2016. Behaviorism is Not Enough: Better Recommendations through Listening to Users. In Proceedings of the Tenth ACM Conference on Recommender Systems (RecSys '16). ACM. DOI:10.1145/2959100.2959179. Acceptance rate: 36% (Past, Present, and Future track).and .
2016. First Do No Harm: Considering and Minimizing Harm in Recommender Systems Designed for Engendering Health. In Proceedings of the Workshop on Recommender Systems for Health at RecSys '16.and .
2014. Building Open-Source Tools for Reproducible Research and Education. In Sharing, Re-use and Circulation of Resources in Cooperative Scientific Work, a workshop at ACM CSCW 2014..
2011. UCERSTI 2: Second Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces. Workshop at the Fifth ACM Conference on Recommender Systems (RecSys '11). ACM, 395–396. DOI:10.1145/2043932.2044020., , and .
2011. LensKit: a modular recommender framework. Demo presented at the Fifth ACM Conference on Recommender Systems (RecSys '11). ACM, 349–350. DOI:10.1145/2043932.2044001., , , and .
- 2014: Texas State University Research Enhancement Program (competitive internal research grant), $8000: Temporal Analysis of Recommender Systems.
- November 21, 2016: ‘Recommending for People’ colloquium at the University at Albany Dept. of Computer Science.
- October 27, 2016: ‘Introduction to Recommender Systems’ at the Clearwater Developer Conference.
- September 20, 2015: ‘Challenges in Scaling Recommender Systems Research’ at the Workshop on Large-Scale Recommender Systems at RecSys '15 in Vienna, Austria.
- September 19, 2015: ‘Levelling Up your Academic Career’ at the Doctoral Symposium at RecSys '15 in Vienna, Austria.
- 2012: ‘Flexible Recommender Experiments with LensKit’ at the RecSys Challenge Workshop at RecSys '12 in Dublin, Ireland.
- 2012: ‘The MovieLens Data Set’ (invited talk) at the RecSys Challenge Workshop at RecSys '12 in Dublin, Ireland.
I have built several open-source software packages in the course of my research and other work. Open-source software distribution is a key piece of my research dissemination strategy. My more significant development efforts include:
- LensKit, a toolkit for building, researching, and studying recommender systems. LensKit has been used in over 20 published papers. http://lenskit.org
- Grapht, a dependency injection framework for Java with novel configuration and static analysis capabilities. http://grapht.grouplens.org
- Goanna (now defunct), a graphical tool for visualizing InfiniBand networks and compute clusters. Written while at the Scalable Computing Laboratory.
- General chair, ACM RecSys 2018
- Track co-chair, 2018 Conference on Fairness, Accountability, and Transparency Systems track
- Distinguished Reviewer, ACM Transactions on Interactive Intelligent Systems (2017–present)
- Steering committee member, ACM Conference on Recommender Systems, 2017–present
- Steering committee member, Conference on Fairness, Accountability, and Transparency, 2017–present
- Organizer, FATREC Workshop on Responsible Recommendation at RecSys 2017
- Publicity co-chair, ACM RecSys 2016
- Program committee, ACM WWW Track on Behavior Analysis and Personalization (2016, 2017)
- External advisor, CrowdRec (EU Framework Programme collaborative research project, 2014–2016)
- Program committee, ACM RecSys (2014, 2015, 2016) and poster session (2016)
- Program committee, FLAIRS Special Track on Recommender Systems (2015, 2016, 2017)
- Proceedings co-chair, ACM CHI 2012–2013
- Demos co-chair, ACM RecSys 2012
- Reviewer for numerous conferences and journals, including:
- ACM conferences CHI (2015, 2013, 2016, 2017), CSCW (2015, 2017), IUI (2017), SAC Recommender Systems track (2013, 2017), UIST (2012, 2016), WikiSym (2012)
- Workshops at ACM RecSys: IntRS (2016–2017), LSRS (2016), KidRec (2017)
- ICWSM 2012
- ACM journals TIST, TOIS, TWEB, TKDD, and TIIS
- IEEE journals TDSC, TKDE
- International Conference on Service-Oriented Computing (2016)
- PLOS ONE (2016)
- User Modeling and User-Adapted Interaction
- Information Retrieval Journal
- ACM Computing Surveys (2014, 2015)
- User Modeling
- Artificial Intelligence Review (Springer)
- Hindawi journals Advances in Multimedia, Advances in Artificial Intelligence
- International Journal of Artificial Intelligence Tools (2016)
- JMLR Open Source (2016)
- IBM Journal of Research and Development (2016)
Department and University Service
- Boise State CS Dept. Graduate Recruiting Committee (2017–)
- Texas State CS Dept. Undergraduate Committee (2014–2016)
- Texas State CS Dept. Written Comp Exam Grading (2014–2016)
- UMN CS Graduate Student Association secretary (2009–2010)
- Judge, 2015 Travis Elementary School Science Fair