Ph.D (2014), Computer Science, University of Minnesota, Minneapolis, MN. Thesis: Towards Recommender Engineering: Tools and Experiments for Identifying Recommender Differences. Advisers: John T. Riedl and Joseph A. Konstan
B.S. (2007), Computer Engineering (With Distinction), Iowa State University, Ames, IA.
- Assistant Professor, Dept. of Computer Science, Boise State University
- Co-founder, People and Information Research Team (PIReT)
- Assistant Professor, Dept. of Computer Science, Texas State University
- Graduate Research Assistant, GroupLens Research, Dept. of Computer Science, University of Minnesota
- Su 2012, F 2013
- Instructor, University of Minnesota
- Summer 2010
- Research Intern, Autodesk Research, Toronto, CA
- 2007–2008, S 2011
- Teaching Assistant, University of Minnesota
- Undergraduate Research Assistant, Scalable Computing Laboratory, Ames Lab, Iowa State University
Boise State University
- Spring 2018
- CS 410/510 (Databases)
- Fall 2017
- CS 533 (Introduction to Data Science); 22 students
- Spring 2017
- CS 597 (Recommender Systems); 13 students
- Fall 2016
- CS 410 / CS 510 (Databases); 28 students
Texas State University
- Spring 2016
- CS 3320 (Internet Software Development); 48 students
- CS 5369Q/4379Q (Recommender Systems); 26 students
- Fall 2015
- CS 4332 (Introduction to Database Systems); 39 students
- Spring 2015
- CS 5369Q/4379Q (Recommender Systems); 28 students
- CS 4350 (Unix Systems Programming); 32 students
- Fall 2014
- CS 4332 (Introduction to Database Systems); 50 students
In addition, I have supervised several independent study students.
I co-created the Recommender Systems specialization on Coursera, along with its two previous single-class versions, with Joseph A. Konstan.
University of Minnesota
- Fall 2013
- CSCI 5980-1 (Introduction to Recommender Systems), co-taught with Joseph A. Konstan; also offered as a MOOC on Coursera.
- Summer 2012
- CSCI 1902 (Structure of Computer Programming II)
- Spring 2011
- CSCI 5125 (Collaborative and Social Computing), as teaching assistant
- CSCI 1902 (Structure of Computer Programming II), as teaching assistant (3 terms)
M.S. Students Supervised
- Mucun Tian (M.S. expected 2019)
- Vaibhav Mahant (M.S. 2016, Texas State University; thesis: Improving Top-N Evaluation of Recommender Systems)
- Sushma Channamsetty (M.S. 2016, Texas State University; thesis: Recommender Response to User Profile Diversity and Popularity Bias)
- Mohammed Imran R Kazi (M.S. 2016, Texas State University; thesis: Exploring Potentially Discriminatory Biases in Book Recommendation)
- Shuvabrata Saha (M.S. 2016, Texas State University; co-advised with Dr. Apan Qasem; thesis: A Multi-objective Autotuning Framework For The Java Virtual Machine)
Author formatting key:, , . Citation counts from Microsoft Academic via Microsoft Cognitive Services. With this data, I have an h-index of 12 and i10-index of 13.
2018. Rating-Based Collaborative Filtering: Algorithms and Evaluation. In Social Information Access. Peter Brusilovsky, ed. Springer-Verlag. ISBN 978-3-319-90091-9., , and .
2016. Dependency Injection with Static Analysis and Context-Aware Policy. Journal of Object Technology 15, 1 (February 2016), pp 1:1–31. DOI:10.5381/jot.2016.15.5.a1.and .
2015. Teaching Recommender Systems at Large Scale: Evaluation and Lessons Learned from a Hybrid MOOC. Transactions on Computer-Human Interaction 22, 2, Article 10 (April 2015), 23 pages. DOI:10.1145/2728171. Cited 12 times., , , , and .
2011. RecBench: Benchmarks for Evaluating Performance of Recommender System Architectures. Proceedings of the VLDB Endowment 4, 11 (August 2011), 911–920. Acceptance rate: 18%. Cited 6 times., , , , , and .
Refereed Conference Publications
These are full papers published in peer-reviewed conference proceedings.
2018. All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness. In Proceedings of the Conference on Fairness, Accountability and Transparency. PMLR 81:172–186. Acceptance rate: 24%., , , , , , and .
2018. Privacy for All: Ensuring Fair and Equitable Privacy Protections. In Proceedings of the Conference on Fairness, Accountability and Transparency. PMLR 81:35–47. Acceptance rate: 24%., , and .
2017. Sturgeon and the Cool Kids: Problems with Top-N Recommender Evaluation. In Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference.and .
2015. Letting Users Choose Recommender Algorithms: An Experimental Study. In Proceedings of the Ninth ACM Conference on Recommender Systems (RecSys ’15). ACM. DOI:10.1145/2792838.2800195. Acceptance rate: 21%. Cited 16 times., , , and .
2014. User Perception of Differences in Recommender Algorithms. In Proceedings of the Eighth ACM Conference on Recommender Systems (RecSys ’14). ACM. DOI:10.1145/2645710.2645737. Acceptance rate: 23%. Cited 70 times., , , and .
2014. Teaching Recommender Systems at Large Scale: Evaluation and Lessons Learned from a Hybrid MOOC. In Proceedings of the First ACM Conference on Learning @ Scale (ACM L@S ’14). ACM. DOI:10.1145/2556325.2566244. Acceptance rate: 37%. Cited 19 times., , , , and .
2013. Rating Support Interfaces to Improve User Experience and Recommender Accuracy. In Proceedings of the Seventh ACM Conference on Recommender Systems (RecSys ’13). ACM. DOI:10.1145/2507157.2507188. Acceptance rate: 24%. Cited 21 times., , , , , , and .
2012. RecStore: An Extensible And Adaptive Framework for Online Recommender Queries Inside the Database Engine. In Proceedings of the 15th International Conference on Extending Database Technology (EDBT ’12). ACM, 86–96. DOI:10.1145/2247596.2247608. Acceptance rate: 23%. Cited 8 times.<>, , , and .
2011. Rethinking The Recommender Research Ecosystem: Reproducibility, Openness, and LensKit. In Proceedings of the Fifth ACM Conference on Recommender Systems (RecSys ’11). ACM, 133–140. DOI:10.1145/2043932.2043958. Acceptance rate: 27% (20% for oral presentation, which this received). Cited 133 times., , , and .
2011. Searching for Software Learning Resources Using Application Context. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST ’11). ACM, 195–204. DOI:10.1145/2047196.2047220. Acceptance rate: 25%. Cited 21 times., , , , and .
2010. Automatically Building Research Reading Lists. In Proceedings of the Fourth ACM Conference on Recommender Systems (RecSys ’10). ACM, 159–166. DOI:10.1145/1864708.1864740. Acceptance rate: 19%. Cited 69 times., , , , , and .
2009. rv you’re dumb: Identifying Discarded Work in Wiki Article History. In Proceedings of the 5th International Symposium on Wikis and Open Collaboration (WikiSym ’09). ACM, 10 pp. DOI:10.1145/1641309.1641317. Acceptance rate: 36% (Selected as Best Paper). Cited 23 times.and .
These are short research papers published in conference proceedings. They are also peer-reviewed.
2017. Recommender Response to Diversity and Popularity Bias in User Profiles. Short paper in Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference.and .
2012. When Recommenders Fail: Predicting Recommender Failure for Algorithm Selection and Combination. Short paper in Proceedings of the Sixth ACM Conference on Recommender Systems (RecSys ’12). ACM, 233–236. DOI:10.1145/2365952.2366002. Acceptance rate: 32%. Cited 19 times.and .
Workshops, Seminars, Posters, Etc.
2018. Recommending Texts to Children with an Expert in the Loop. In Proceedings of the 2nd International Workshop on Children & Recommender Systems (KidRec) at IDC 2018., , .
Do Different Groups Have Comparable Privacy Tradeoffs?. At Moving Beyond a ‘One-Size Fits All’ Approach: Exploring Individual Differences in Privacy, a workshop at CHI 2018., , , and . 2018.
2017. The Demographics of Cool: Popularity and Recommender Performance for Different Groups of Users. In RecSys 2017 Poster Proceedings.and .
2017. Challenges in Evaluating Recommendations for Children. In Proceedings of the International Workshop on Children & Recommender Systems (KidRec) at RecSys 2017..
2016. Behaviorism is Not Enough: Better Recommendations through Listening to Users. In Proceedings of the Tenth ACM Conference on Recommender Systems (RecSys ’16). ACM. DOI:10.1145/2959100.2959179. Acceptance rate: 36% (Past, Present, and Future track). Cited 6 times.and .
2016. First Do No Harm: Considering and Minimizing Harm in Recommender Systems Designed for Engendering Health. In Proceedings of the Workshop on Recommender Systems for Health at RecSys ’16.and .
2014. Building Open-Source Tools for Reproducible Research and Education. In Sharing, Re-use and Circulation of Resources in Cooperative Scientific Work, a workshop at ACM CSCW 2014..
2017. The FATREC Workshop on Responsible Recommendation. In Proceedings of the 11th ACM Conference on Recommender Systems.and .
2011. UCERSTI 2: Second Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces. Workshop at the Fifth ACM Conference on Recommender Systems (RecSys ’11). ACM, 395–396. DOI:10.1145/2043932.2044020. Cited 3 times., , and .
- 2018–2023: NSF award 1751278, $482,081: CAREER: User-Based Simulation Methods for Quantifying Sources of Error and Bias in Recommender Systems
- 2017: $19K Boise State College of Education Empathy Grant LITERATE: Locating Informational Texts for Engaging Readers And Teaching Equitably (co-PI; with PI Katherine Wright & co-PI Sole Pera)
- 2014: $8K Texas State University Research Enhancement Program (competitive internal research grant) Temporal Analysis of Recommender Systems (PI)
- November 16, 2017: ‘Making Information Systems Good for People’ at Whitman College (Walla Walla, WA)
- June 26, 2017: ‘Recommending for People’ seminar at RecSysNL at TU Delft
- June 20, 2017: ‘Recommending for People’ seminar at Jheronimus Academy of Data Science
- June 19, 2017: ‘Recommending for People’ seminar at UCL Mons
- June 15, 2017: ‘Responsible Recommendation’ at the Brussels Big Data and Ethics Meetup, the inaugural event of the DigitYser Big DAta community
- November 21, 2016: ‘Recommending for People’ colloquium at the University at Albany Dept. of Computer Science
- October 27, 2016: ‘Introduction to Recommender Systems’ at the Clearwater Developer Conference
- September 20, 2015: ‘Challenges in Scaling Recommender Systems Research’ at the Workshop on Large-Scale Recommender Systems at RecSys ’15 in Vienna, Austria
- September 19, 2015: ‘Levelling Up your Academic Career’ at the Doctoral Symposium at RecSys ’15 in Vienna, Austria
- 2012: ‘Flexible Recommender Experiments with LensKit’ at the RecSys Challenge Workshop at RecSys ’12 in Dublin, Ireland
- 2012: ‘The MovieLens Data Set’ (invited talk) at the RecSys Challenge Workshop at RecSys ’12 in Dublin, Ireland
I have built several open-source software packages in the course of my research and other work. Open-source software distribution is a key piece of my research dissemination strategy. My more significant development efforts include:
- LensKit, a toolkit for building, researching, and studying recommender systems. LensKit has been used in over 20 published papers. http://lenskit.org
- Grapht, a dependency injection framework for Java with novel configuration and static analysis capabilities. http://grapht.grouplens.org
- Goanna (now defunct), a graphical tool for visualizing InfiniBand networks and compute clusters. Written while at the Scalable Computing Laboratory.
- General co-chair, ACM RecSys 2018
- Participant in Dagstuhl Perspectives Workshop Towards Cross-Domain Performance Modeling and Prediction: IR/RecSys/NLP
- Co-organizer, FairUMAP workshop at UMAP 2018
- Track co-chair, 2018 Conference on Fairness, Accountability, and Transparency Systems track
- Distinguished Reviewer, ACM Transactions on Interactive Intelligent Systems (2017–present)
- Steering committee member, ACM Conference on Recommender Systems, 2017–present
- Steering committee member, Conference on Fairness, Accountability, and Transparency, 2017–present
- Program committee, ACM RecSys (2014–2017) and poster session (2016)
- Organizer, FATREC Workshop on Responsible Recommendation at RecSys 2017
- Program committee, FLAIRS Special Track on Recommender Systems (2015, 2016, 2017)
- Publicity co-chair, ACM RecSys 2016
- Program committee, ACM WWW Track on Behavior Analysis and Personalization (2016, 2017)
- External advisor, CrowdRec (EU Framework Programme collaborative research project, 2014–2016)
- Proceedings co-chair, ACM CHI 2012–2013
- Demos co-chair, ACM RecSys 2012
- Reviewer for numerous conferences and journals, including:
- ACM conferences CHI (2015, 2013, 2016, 2017), CSCW (2015, 2017), IUI (2017), SAC Recommender Systems track (2013, 2017), UIST (2012, 2016, 2017), WikiSym (2012)
- Workshops at ACM RecSys: IntRS (2016–2017), LSRS (2016), KidRec (2017)
- FATML 2017
- International Journal of Artificial Intelligence Tools (2016)
- JMLR Open Source (2016)
- IBM Journal of Research and Development (2016)
- ICWSM 2012
- ACM journals TIST, TOIS, TWEB, TKDD, and TIIS
- IEEE journals TDSC, TKDE
- Interacting with Computers
- International Conference on Service-Oriented Computing (2016)
- PLOS ONE (2016)
- User Modeling and User-Adapted Interaction
- Information Retrieval Journal
- ACM Computing Surveys (2014, 2015)
- User Modeling
- Artificial Intelligence Review (Springer)
- Hindawi journals Advances in Multimedia, Advances in Artificial Intelligence
Department and University Service
- Boise State Ph.D in Computing Steering Committee (Fall 2017)
- Boise State CS Dept. Curriculum Committee (2017–2018)
- Boise State CS Dept. Graduate Recruiting Committee (2017)
- Texas State CS Dept. Undergraduate Committee (2014–2016)
- Texas State CS Dept. Written Comp Exam Grading (2014–2016)
- UMN CS Graduate Student Association secretary (2009–2010)
- December 2017 — Boise Public Library panel on preparing for a career in computer science
- Judge, 2015 Travis Elementary School Science Fair