Education
- Ph.D (2014)
- Computer Science, University of Minnesota.
- Advisers: John T. Riedl and Joseph A. Konstan
- B.S. (2007)
- Computer Engineering, Iowa State University.
Employment History
- 2022–present
- Associate Professor, Dept. of Computer Science, Boise State University
- Co-director, People and Information Research Team (PIReT)
- 2016–2022
- Assistant Professor, Dept. of Computer Science, Boise State University
- Co-director, People and Information Research Team (PIReT)
- 2014–2016
- Assistant Professor, Dept. of Computer Science, Texas State University
- 2008–2014
- Graduate Research Assistant, GroupLens Research, University of Minnesota
- Su 2012, F 2013
- Instructor, University of Minnesota
- Summer 2010
- Research Intern, Autodesk Research, Toronto, CA
- 2007–2008, S 2011
- Teaching Assistant, University of Minnesota
- 2005–2007
- Undergrad Research Assistant, Scalable Computing Laboratory, Iowa State University
Students
Current Graduate Students
- Amifa Raj (Ph.D, expected 2023)
- Ngozi Ihemelandu (Ph.D, expected 2023)
Completed Graduate Students
- Srabanti Guha (M.S. 2023; project: Explaining Misallocated Exposure across Multiple Rankings)
- Carlos Segura Cerna (M.S. 2020; project: Recommendation Server for LensKit; software engineer at Cradlepoint)
- Mucun Tian (M.S. 2019; thesis: Estimating Error and Bias of Offline Recommender System Evaluation Results; Sr. Scientist at Pandora)
- Vaibhav Mahant (M.S. 2016, Texas State University; thesis: Improving Top-N Evaluation of Recommender Systems; now at Sagezza / Goldman Sachs)
- Sushma Channamsetty (M.S. 2016, Texas State University; thesis: Recommender Response to User Profile Diversity and Popularity Bias; Sr. Software Engineer at Q2)
- Mohammed Imran R Kazi (M.S. 2016, Texas State University; thesis: Exploring Potentially Discriminatory Biases in Book Recommendation; software engineer at eBay)
- Shuvabrata Saha (M.S. 2016, Texas State University; co-advised with Dr. Apan Qasem; thesis: A Multi-objective Autotuning Framework For The Java Virtual Machine; software developer at PHEAA)
Undergraduate Student Research
I have supported and mentored the following undergraduate research students: Christine Pinney (BSU, UGRA + REU), Liana Shiroma (Colby Coll., REU 2021), Stephen Randall (U. Pitt, REU 2021), Connor Wood (BSU, REU 2020 + UGRA), Ananda Montoly (Smith Coll., REU 2020), Sandra Ambriz (BSU, HERC + UGRA).
Funding key:
- UGRA: undergraduate research assistant hired from research funds
- REU: Research Experience for Undergraduates
- HERC: Higher Education Research Consortium
Research Funding
External Grants
- 2023–2025: NSF Collaborative Research: CCRI: New: A Research News Recommender Infrastructure with Live Users for Algorithm and Interface Experimentation ($1.4M; BSU PI, my share $150K; PI Joseph A. Konstan, UMN).
- 2018–2023: NSF award CHS 17-51278, $514,081: CAREER: User-Based Simulation Methods for Quantifying Sources of Error and Bias in Recommender Systems (PI). Total includes REU supplements.
Internal Grants
- 2017: $19K Boise State College of Education Civility Grant LITERATE: Locating Informational Texts for Engaging Readers And Teaching Equitably (co-PI; with PI Katherine Wright & co-PI Sole Pera)
- 2014: $8K Texas State University Research Enhancement Program (competitive internal research grant) Temporal Analysis of Recommender Systems (PI)
Publications
Author formatting key:
- , ,
- † presenter
- ‡ equal first authors
- § undergraduate student
Citation counts from Google Scholar.
* These publications have citations merged in Google Scholar; count is reported on the most most final version, such as the journal expansion of a conference article.
Software and Data
I have built several open-source software packages and data sets in the course of my research and other work. Open-source software distribution and open data are key pieces of my research dissemination strategy. My most significant development efforts are:
- LensKit, a toolkit for building, researching, and studying recommender systems. As of Nov. 1, 2022, the original Java software (in development 2010–2018; paper RecSys11) is known to be used in 70 papers and theses and was used by over 2500 students to complete programming assignments in the Recommender Systems MOOC. The Python software (2018–, papers CIKM20-lk and Reveal18-lk) is used in over 30 papers, theses, and educational resources, including the PBS show Crash Course AI, and has been downloaded over 9000 times from the Python Package Index in the last 6 months (according to PyPIStats). The current version is 0.14.2, released on July 16, 2022; it is the 23rd release of LensKit for Python.
https://lenskit.org (current list of known uses: https://lenskit.org/research/) - Book Data Tools, software tools to integrate multiple public sources of book and book consumption data into a data set for studying social effects in book publication, reading, and recommendation. Used in UMUAI21 and RecSys18. https://bookdata.piret.info
My work has also produced a number of utility packages to support this software and other efforts, including:
- seedbank, a Python package for consistently seeding random number generators.
https://seedbank.lenskit.org - csr, a Python package for managing sparse matrices in CSR format compatible with the Numba JIT for scientific python, and with Intel MKL acceleration for several operations.
https://csr.lenskit.org - binpickle, a Python package for saving scientific data structures (such as machine learning models) to disk in either compressed or memory-mappable format. LensKit uses this package to serialize models for both storage and shared-memory parallelism. https://binpickle.lenskit.org
- happylog, a Rust package for easily configuring log output for command-line programs.
https://github.com/mdekstrand/happylog - Grapht, a dependency injection framework for Java with novel configuration and static analysis capabilities (paper JOT16). http://grapht.grouplens.org
Invited Talks
- May 2023: Invited talk at ICA post-conference panel
- Mar. 2023: Seminar at the University of Texas at Austin HCI group
- Jan. 2023: Seminar at the University of Washington RAISE group
- Nov. 2022: Keynote at IBIS2022 (Information-Based Inductive Systems and Machine Learning) workshop (Tsukuba, Japan)
- Nov. 2022: Seminar at Waseda University (Japan)
- Oct. 2022: Keynote at EvalRS workshop on rounded evaluation of recommender systems at CIKM 2022
- Sep. 2022: Guest lecture on IR fairness and test collections for University of Maine IR course
- Mar. 2022: ‘You Might Also Think This Is Unfair’ at University of Michigan School of Information (online)
- Nov. 2021: ‘Information Systems for Human Flourishing’ at Vector Institute, Toronto, Canada (online)
- Oct. 2020: Guest lecture on recommender systems and fairness for Carnegie Mellon University Human-AI Interaction course
- Apr. 2020: Guest lecture on recommender systems and fairness for Emory University recommender systems course
- Mar. 2020: ‘User, Agent, Subject, Spy’ seminar at Boise State University Ph.D in Computing Colloquium
- Oct. 2019: ‘Online Recommendation: What? Where? Why? How?’ session at the Idaho Library Association 2019 Conference
- Aug. 2019: ‘User, Agent, Subject, Spy’ seminar at Microsoft Research Montréal
- Jul. 2019: ‘User, Agent, Subject, Spy’ seminar at Criteo AI Labs, Paris, France
- May 2019: ‘Recommendations, Decisions, Feedback Loops, and Maybe Saving the Planet’ at the CRA CCC Visioning Workshop on Economics and Fairness.
- Dec. 2018: ‘User, Agent, Subject, Spy’ seminar at Clemson University
- Nov. 2018: ‘User, Agent, Subject, Spy’ seminar at Carnegie Mellon University Human-Computer Interaction Institute
- Nov. 2018: Guest lecture on recommender systems for Carnegie Mellon University Human-AI Interaction course
- Nov. 2017: ‘Making Information Systems Good for People’ at Whitman College (Walla Walla, WA)
- Jun. 2017: ‘Recommending for People’ seminar at RecSysNL at TU Delft
- Jun. 2017: ‘Recommending for People’ seminar at Jheronimus Academy of Data Science
- Jun. 2017: ‘Recommending for People’ seminar at UCL Mons
- Jun. 2017: ‘Responsible Recommendation’ at the Brussels Big Data and Ethics Meetup, the inaugural event of the DigitYser Big Data community
- Nov. 2016: ‘Recommending for People’ colloquium at the University at Albany Dept. of Computer Science
- Oct. 2016: ‘Introduction to Recommender Systems’ at the Clearwater Developer Conference
- Sep. 2015: ‘Challenges in Scaling Recommender Systems Research’ at the Workshop on Large-Scale Recommender Systems at RecSys ’15 in Vienna, Austria
- Sep. 2015: ‘Levelling Up your Academic Career’ at the Doctoral Symposium at RecSys ’15 in Vienna, Austria
- Sep. 2012: ‘Flexible Recommender Experiments with LensKit’ at the RecSys Challenge Workshop at RecSys ’12 in Dublin, Ireland
- Sep. 2012: ‘The MovieLens Data Set’ (invited talk) at the RecSys Challenge Workshop at RecSys ’12 in Dublin, Ireland
Teaching
Boise State University
| Term | Course | Title | Credits | Students |
|---|---|---|---|---|
| S23 | CS 538 | Recommender Systems | 3 | 12 |
| F22 | CS 533 | Intro to Data Science | 3 | 27 |
| S22 | CS 230 | Ethics in Computing | 3 | 61 |
| F21 | CS 533 | Intro to Data Science | 3 | 43 |
| S21 | CS 538 | Recommender Systems | 3 | 11 |
| F20 | CS 533 | Intro to Data Science | 3 | 22 |
| S20 | CS 697 | Equity and Discrimination | 3 | 3 |
| S20 | CS 410 | Databases | 3 | 36 |
| F19 | CS 533 | Intro to Data Science | 3 | 28 |
| S19 | CS 538 | Recommender Systems | 3 | 12 |
| F18 | CS 410/510 | Databases | 3 | 40 |
| Su18 | CS 310-HU | Intro to Databases | 1 | 6 |
| S18 | CS 410/510 | Databases | 3 | 22 |
| F17 | CS 533 | Intro to Data Science | 3 | 22 |
| S17 | CS 597 | Recommender Systems | 3 | 13 |
| F16 | CS 410/510 | Databases | 3 | 28 |
Texas State University
- CS 4332 (Intro to Database Systems)
- CS 3320 (Internet Software Development)
- CS 5369Q/4379Q (Recommender Systems)
- CS 4350 (Unix Systems Programming)
Coursera
I co-created the Recommender Systems specialization on Coursera, along with its two previous single-class versions, with Joseph A. Konstan. This course has reached over 95,000 learners across its 3 iterations.
University of Minnesota
- Instructor for CS 5980-1 (Intro to Recommender Systems)
- Summer instructor for CS 1902 (Structure of Computer Programming II)
- TA for CSCI 5125 (Collaborative and Social Computing) and CSCI 1902
Teaching Professional Development
- Boise State University teaching portfolio faculty learning community.
- Boise State University Ten for Teaching program.
- Boise State University Center for Teaching and Learning Course Design Institute, a one-week intensive session in Summer 2017.
- CTL workshops on service learning, mastery-based grading, and other topics.
- Texas State University’s Program for Excellence in Teaching and Learning (2014–2015).
- Preparing Future Faculty at the University of Minnesota.
Service
Ongoing Professional Service, Memberships, and Honors
- Executive committee, ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2020–2023
- Co-chair, FAccT Network, 2019–present
- Steering committee, ACM Conference on Recommender Systems (RecSys), 2017–present
- Steering committee, ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2017–present (inaugural member)
- Senior Member of the Association for Computing Machinery
Program Committee and Editorial Service
- Track co-chair, Responsibility, Ethics, and Compliance, UMAP 2023
- Track co-chair, Fairness, Accountability, Transparency, and Ethics (FATE), TheWebConf 2023
- Program co-chair, 16th ACM Conference on Recommender Systems (RecSys 2022)
- Guest editor, 2021 special issue of User Modeling and User-Adapted Interaction on fairness in user modeling.
- Distinguished Reviewer, ACM Transactions on Interactive Intelligent Systems (TiiS) (2017–present)
- ACM Conference on Recommender Systems (Senior PC 2019–2021, PC 2014–2017)
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (2018–2021, Area Chair 2018)
- ACM CIKM (Resource Track PC 2020–2021)
- ACM SIGIR (PC 2020–2021 Full and Short Papers; 2021 Perspective and Resource papers)
- NeurIPS Ethical Review panel (2021)
- TheWebConf Track on Behavior Analysis and Personalization (Senior PC 2021, PC 2016–2020)
- Track chair, User Modeling and Adaptive Personalization (UMAP) 2021
- User Modeling and Adaptive Personalization (2019–2020)
- Workshop on Fairness, Accountability, and Transparency in Machine Learning (FATML) (2017)
- FLAIRS Special Track on Recommender Systems (2015–2017)
- SAC Recommender Systems track (2013, 2017)
- Ad-hoc conference reviews for CHI, CSCW, IUI, UIST, WikiSym, UMAP, ICWSM.
- Reviewed for Communications of the ACM; ACM journals TDS, TOCHI, TIST, TOIS, TWEB, TKDD, and TIIS; IEEE journals TDSC and TKDE; Interacting with Computers; UMUAI; Information Retrieval Journal; ACM Computing Surveys; Artificial Intelligence Review; and others.
- Grant proposal reviews for NSF (US 2019, 2020, 2021, 2022), NWO (NL), FWF & WWTF (AT)
Other Professional Service
- Ph.D symposium mentor, CIKM 2023
- Co-organizer, SimuRec Workshop on Simulation and Synthetic Data for Recommender Systems at RecSys 2021
- Sponsorship co-chair, ACM FAccT 2021–2022
- Doctoral symposium co-chair, RecSys 2020
- Organized and moderated panel at RecSys 2019 on responsible recommendation
- Co-organizer, TREC Track on Fairness in Information Retrieval (2019–2022)
- PR & Publicity co-chair, 2nd Conference on Fairness, Accountability, and Transparency (ACM FAT* 2019)
- General co-chair, ACM RecSys 2018
- Publications working group, FAccT steering committee (2017)
- Co-organizer, FATREC Workshop on Responsible Recommendation at RecSys 2017, 2018, 2020, 2021
- Co-organizer, Workshop on Fairness, Accountability, Confidentiality, Transparency, and Safety in Information Retrieval (FACTS-IR) at SIGIR 2019
- Co-organizer, FairUMAP workshop at UMAP 2018–2020
- Track co-chair, 2018 Conference on Fairness, Accountability, and Transparency Systems track
- Participant in Dagstuhl Perspectives Workshop Towards Cross-Domain Performance Modeling and Prediction: IR/RecSys/NLP
- Publicity co-chair, ACM RecSys 2016
- External advisor, CrowdRec (EU Framework Programme collaborative research project, 2014–2016)
- Proceedings co-chair, ACM CHI 2012–2013
- Demos co-chair, ACM RecSys 2012
Department and University Service
- 2020–2021 CS Faculty Search Committee
- COEN SAGE Scholars Program Mentor (2019–2021)
- Boise State College of Engineering Curriculum Committee (2019–present)
- Boise State Ph.D in Computing Steering Committee (2017–present)
- Boise State CS Dept. Curriculum Committee (2017–present)
- Boise State CS Dept. Graduate Recruiting Committee (2017)
- Texas State CS Dept. Undergraduate Committee (2014–2016)
- Texas State CS Dept. Written Comp Exam Grading (2014–2016)
- UMN CS Graduate Student Association secretary (2009–2010)
Community and Civic Service
- January 2023 — joined amicus brief before SCOTUS on Gonzalez v. Google.
- July 2020 — taught continuing education session for Idaho Council for Libraries.
- October 2019 — presented at Idaho Library Association Annual Conference.
- February 2019 — addressed Idaho State House Judiciary Committee on H.B. 118, regulating pretrial risk assessment algorithms; through subsequent engagement, I contributed language that is in the final enacted legislation.
- December 2017 — Boise Public Library panel on preparing for a career in computer science.
- Judge, 2015 — Travis Elementary School Science Fair.
Media Mentions
- “The Deadline Dilemma”. (Carolyn Kuimelis, Teaching newsletter from Chronicle of Higher Education, December 1, 2022. https://www.chronicle.com/newsletter/teaching/2022-12-01).
- “Out of the Blue”. (Ravi Shankar, The New Indian Express, May 1, 2022. https://www.newindianexpress.com/opinions/columns/ravi-shankar/2022/may/01/outof-theblue-2447591.html). Quotes from Washington Post article below.
- “Elon Musk wants Twitter’s algorithm to be public. It’s not that simple.” (Reed Albergotti, The Washington Post, April 16, 2022. https://www.washingtonpost.com/technology/2022/04/16/elon-musk-twitter-algorithm/).
- Quoted at length about how artificial intelligence learns from social signals in “Can AI be horny?” (Chris Stokel-Walker, Input, April 28, 2021; Bustle Digital Group. https://www.inputmag.com/culture/artificial-intelligence-ai-archillect-twitter-horny-sex).
- Quoted in several articles about FAccT suspending Google’s sponsorship for the 2021 conference, in my role as FAccT Sponsor Co-chair and a member of the Executive Committee. These articles include:
- “AI ethics research conference suspends Google sponsorship.” (Khari Johnson, VentureBeat, March 2, 2021. https://venturebeat.com/2021/03/02/ai-ethics-research-conference-suspends-google-sponsorship/)
- “Conference suspends Google sponsorship after ethics experts’ exit.” (D. Matthews, Times Higher Education, March 8, 2021. https://www.timeshighereducation.com/news/conference-suspends-google-sponsorship-after-ethics-experts-exit)
- “Tech transparency conference suspends Google sponsorship over transparency concerns.” (Colleen Flaherty, Inside Higher Ed, March 9, 2021. https://www.insidehighered.com/news/2021/03/09/tech-transparency-conference-suspends-google-sponsorship-over-transparency-concerns)
- “Google offered a professor $60,000, but he turned it down. Here’s why.” (Rachel Metz, CNN Business, March 24, 2021. https://www.cnn.com/2021/03/24/tech/google-ai-ethics-reputation/index.html). I am not the professor who declined funding, but am quoted for context.
- “How one employee’s exit shook Google and the AI industry.” (Rachel Metz, CNN Business, March 11, 2021. https://www.cnn.com/2021/03/11/tech/google-ai-ethics-future/index.html).
- Quoted about voter file data leaks in “D.C. makes it shockingly easy to snoop on your fellow voters.” (Brian Fung, The Switch [a blog by The Washington Post], June 14, 2016. https://www.washingtonpost.com/news/the-switch/wp/2016/06/14/d-c-s-board-of-elections-makes-it-shockingly-easy-to-snoop-on-your-fellow-voters/)
- Quoted about recommender systems principles in “TV seems to know what you want to see; algorithms at work.” (Scott Collins, Los Angeles Times, November 21, 2014. https://www.latimes.com/entertainment/tv/la-et-st-tv-section-algorithm-20141123-story.html)