Education
- Ph.D (2014)
- Computer Science, University of Minnesota.
- Advisers: John T. Riedl and Joseph A. Konstan
- B.S. (2007)
- Computer Engineering, Iowa State University.
Employment History
- 2023–present
- Assistant Professor, Dept. of Information Science, Drexel University
- 2022–2023
- Associate Professor, Dept. of Computer Science, Boise State University
- Co-director, People and Information Research Team (PIReT)
- 2016–2022
- Assistant Professor, Dept. of Computer Science, Boise State University
- Co-director, People and Information Research Team (PIReT)
- 2014–2016
- Assistant Professor, Dept. of Computer Science, Texas State University
- 2008–2014
- Graduate Research Assistant, GroupLens Research, University of Minnesota
- Su 2012, F 2013
- Instructor, Dept. of Computer Science, University of Minnesota
- Summer 2010
- Research Intern, Autodesk Research (Toronto)
- 2007–2008, 2011
- Teaching Assistant, Dept. of Computer Science, University of Minnesota
- 2005–2007
- Undergrad RA, Scalable Computing Laboratory, Iowa State University
Students
Ph.D. Graduates
- Ngozi Ihemelandu (Ph.D. 2024)
- Amifa Raj (Ph.D. 2023; Applied Scientist at Microsoft)
M.S. Graduates
- Srabanti Guha (M.S. 2023; project: Explaining Misallocated Exposure across Multiple Rankings)
- Carlos Segura Cerna (M.S. 2020; project: Recommendation Server for LensKit; software engineer at Cradlepoint)
- Mucun Tian (M.S. 2019; thesis: Estimating Error and Bias of Offline Recommender System Evaluation Results; Sr. Scientist at Pandora)
- Vaibhav Mahant (M.S. 2016, Texas State University; thesis: Improving Top-N Evaluation of Recommender Systems; now at Sagezza / Goldman Sachs)
- Sushma Channamsetty (M.S. 2016, Texas State University; thesis: Recommender Response to User Profile Diversity and Popularity Bias; Sr. Software Engineer at Q2)
- Mohammed Imran R Kazi (M.S. 2016, Texas State University; thesis: Exploring Potentially Discriminatory Biases in Book Recommendation; software engineer at eBay)
- Shuvabrata Saha (M.S. 2016, Texas State University; co-advised with Dr. Apan Qasem; thesis: A Multi-objective Autotuning Framework For The Java Virtual Machine; software developer at PHEAA)
Undergraduate Student Research
I have supported and mentored the following undergraduate research students: Christine Pinney (BSU, UGRA + REU), Liana Shiroma (Colby Coll., REU 2021), Stephen Randall (U. Pitt, REU 2021), Connor Wood (BSU, REU 2020 + UGRA), Ananda Montoly (Smith Coll., REU 2020), Sandra Ambriz (BSU, HERC + UGRA).
Funding key:
- UGRA: undergraduate research assistant hired from research funds
- REU: Research Experience for Undergraduates
- HERC: Higher Education Research Consortium
Research Funding
External Grants
- 2023–2025: NSF 22-32553: Collaborative Research: CCRI: New: A Research News Recommender Infrastructure with Live Users for Algorithm and Interface Experimentation ($1.4M; Drexel PI, my share $150K; PI Joseph A. Konstan, UMN).
- 2018–2024: NSF 17-51278: CAREER: User-Based Simulation Methods for Quantifying Sources of Error and Bias in Recommender Systems ($514,081; PI). Total includes REU supplements.
Internal Grants
- 2017: Boise State College of Education Civility Grant LITERATE: Locating Informational Texts for Engaging Readers And Teaching Equitably ($19K; co-PI; with PI Katherine Wright & co-PI Sole Pera)
- 2014: Texas State University Research Enhancement Program (competitive internal research grant) Temporal Analysis of Recommender Systems ($8K; PI)
Publications
Author formatting key:
, , ; †presenter, §undergraduate student.Citation counts from Google Scholar (total 4642, h-index 28).
◊ These publications have citations merged in Google Scholar; count is reported on the most most final version, such as the journal expansion of a conference article.
Invited Talks
- Mar. 2024: Keynote at IR4U2 workshop at ECIR 2024
- Feb. 2024: Seminar at University of Colorado at Boulder
- Oct. 2023: Virtual seminar at the University of Glasgow
- May 2023: Invited talk at ICA post-conference panel
- Mar. 2023: Seminar at the University of Texas at Austin HCI group
- Jan. 2023: Seminar at the University of Washington RAISE group
- Nov. 2022: Keynote at IBIS2022 (Information-Based Inductive Systems and Machine Learning) workshop (Tsukuba, Japan)
- Nov. 2022: Seminar at Waseda University (Japan)
- Oct. 2022: Keynote at EvalRS workshop on rounded evaluation of recommender systems at CIKM 2022
- Sep. 2022: Guest lecture on IR fairness and test collections for University of Maine IR course
- Mar. 2022: ‘You Might Also Think This Is Unfair’ at University of Michigan School of Information (online)
- Nov. 2021: ‘Information Systems for Human Flourishing’ at Vector Institute, Toronto, Canada (online)
- Oct. 2020: Guest lecture on recommender systems and fairness for Carnegie Mellon University Human-AI Interaction course
- Apr. 2020: Guest lecture on recommender systems and fairness for Emory University recommender systems course
- Mar. 2020: ‘User, Agent, Subject, Spy’ seminar at Boise State University Ph.D in Computing Colloquium
- Oct. 2019: ‘Online Recommendation: What? Where? Why? How?’ session at the Idaho Library Association 2019 Conference
- Aug. 2019: ‘User, Agent, Subject, Spy’ seminar at Microsoft Research Montréal
- Jul. 2019: ‘User, Agent, Subject, Spy’ seminar at Criteo AI Labs, Paris, France
- May 2019: ‘Recommendations, Decisions, Feedback Loops, and Maybe Saving the Planet’ at the CRA CCC Visioning Workshop on Economics and Fairness.
- Dec. 2018: ‘User, Agent, Subject, Spy’ seminar at Clemson University
- Nov. 2018: ‘User, Agent, Subject, Spy’ seminar at Carnegie Mellon University Human-Computer Interaction Institute
- Nov. 2018: Guest lecture on recommender systems for Carnegie Mellon University Human-AI Interaction course
- Nov. 2017: ‘Making Information Systems Good for People’ at Whitman College (Walla Walla, WA)
- Jun. 2017: ‘Recommending for People’ seminar at RecSysNL at TU Delft
- Jun. 2017: ‘Recommending for People’ seminar at Jheronimus Academy of Data Science
- Jun. 2017: ‘Recommending for People’ seminar at UCL Mons
- Jun. 2017: ‘Responsible Recommendation’ at the Brussels Big Data and Ethics Meetup, the inaugural event of the DigitYser Big Data community
- Nov. 2016: ‘Recommending for People’ colloquium at the University at Albany Dept. of Computer Science
- Oct. 2016: ‘Introduction to Recommender Systems’ at the Clearwater Developer Conference
- Sep. 2015: ‘Challenges in Scaling Recommender Systems Research’ at the Workshop on Large-Scale Recommender Systems at RecSys ’15 in Vienna, Austria
- Sep. 2015: ‘Levelling Up your Academic Career’ at the Doctoral Symposium at RecSys ’15 in Vienna, Austria
- Sep. 2012: ‘Flexible Recommender Experiments with LensKit’ at the RecSys Challenge Workshop at RecSys ’12 in Dublin, Ireland
- Sep. 2012: ‘The MovieLens Data Set’ (invited talk) at the RecSys Challenge Workshop at RecSys ’12 in Dublin, Ireland
Teaching
Drexel University
- DSCI 641 (Recommender Systems for Data Science)
- INFO 659 (Intro to Data Analytics)
Boise State University
- CS 410/510 (Databases)
- CS 533 (Intro to Data Science)
- CS 538 (Recommender Systems)
- CS 697 (Special Topics: Equity and Discrimination in Computing Systems)
Texas State University
- CS 4332 (Intro to Database Systems)
- CS 3320 (Internet Software Development)
- CS 5369Q/4379Q (Recommender Systems)
- CS 4350 (Unix Systems Programming)
Coursera
I co-created the Recommender Systems specialization on Coursera, along with its two previous single-class versions, with Joseph A. Konstan. This course has reached over 95,000 learners across its 3 iterations.
University of Minnesota
- Instructor for CS 5980-1 (Intro to Recommender Systems)
- Summer instructor for CS 1902 (Structure of Computer Programming II)
- TA for CSCI 5125 (Collaborative and Social Computing) and CSCI 1902
Teaching Professional Development
- Boise State University teaching portfolio faculty learning community.
- Boise State University Ten for Teaching program.
- Boise State University Center for Teaching and Learning Course Design Institute, a one-week intensive session in Summer 2017.
- CTL workshops on service learning, mastery-based grading, and other topics.
- Texas State University’s Program for Excellence in Teaching and Learning (2014–2015).
- Preparing Future Faculty at the University of Minnesota.
Service
Ongoing Professional Service, Memberships, and Honors
- Editorial board, Foundations and Trends in Information Retrieval (2023–)
- Co-chair, FAccT Network, 2019–
- Steering committee, ACM Conference on Recommender Systems (RecSys), 2017–
- Senior Member of the Association for Computing Machinery (since 2019)
- Distinguished Reviewer, ACM Transactions on Interactive Intelligent Systems (TiiS) (2017–present)
Program Committee and Editorial Service
- ACM FAccT (AC 2018, 2023–2024; PC 2019–2021)
- ACM SIGIR main program (AC 2024; PC 2020–2021, 2023), Perspectives (PC 2021), short papers (PC 2021), resource track (PC 2021)
- ECIR main program (PC 2024), short papers (PC 2024), IR for Good (PC 2024), tutorials (PC 2024)
- Best paper committee, ACM SIGIR 2023
- ACM RecSys main program (SPC 2019–2021, 2023; PC 2014–2017), Reproducibility (PC 2021, 2023), LBR (PC 2019–2020), Posters (PC 2016–2017)
- SIGIR Asia-Pacific (SPC 2023)
- Best paper committee, TheWebConf 2023
- Track chair, UMAP 2023 (Responsibility, Compliance, and Ethics)
- Program co-chair, 16th ACM Conference on Recommender Systems (RecSys 2022)
- Guest editor, 2021 special issue of User Modeling and User-Adapted Interaction (UMUAI) on fairness in user modeling.
- TheWebConf User Modeling, Behavior, & Personalization (SPC 2021; PC 2016, 2018–2020), Behavior Analysis and Recommendation (PC 2016)
- Track Chair, UMAP 2021
- ACM CIKM resource track (PC 2021), Resource (PC 2020)
- ACM WSDM (PC 2020–2021)
- Ethics reviewer, NeurIPS 2021
- UMAP (PC 2018–2020)
- CHI Posters (PC 2019)
- FLAIRS Special Track on Recommender Systems (PC 2015–2017)
- ACM SAC Recommender Systems (PC 2013, 2016)
- NeurIPS
- Additional conference reviews for CHI (2012, 2015–2017, 2019–2020), CSCW (2014, 2017, 2019–2020), FAccT (2020), ICSOC (2016), IUI (2016), and UIST (2012, 2016–2017, 2020).
- Journal reviews for Advances in AI, Artificial Intelligence Review, CACM, CSUR, IBM Journal of Research and Development, INRT, Information Retrieval Journal, Interacting with Computers, International Journal of Artificial Intelligence Tools, JMLR Open Source, JRC, Journal of Librarianship & Information Science, PLOS ONE, PeerJ Computer Science, TDS, TDSC, TIST, TKDE, TOCHI, TOIS, TORS, TSC, TWEB, TiiS, and UMUAI.
- Reviewer for numerous workshops at RecSys, UMAP, and elsewhere.
Other Professional Service
- Executive committee, ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2020–2023
- Steering committee, ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2017–2023 (inaugural member)
- Co-author and signatory, FAccT Statement on AI Harms and Policy (2023); covered by VentureBeat and The Hill (op-ed)
- Co-organizer, CRAFT panel “Theories of Change in Responsible AI” at FAccT 2023
- Ph.D. symposium mentor, CIKM 2023
- Co-organizer, SimuRec Workshop on Simulation and Synthetic Data for Recommender Systems at RecSys 2021
- Sponsorship co-chair, ACM FAccT 2021–2022
- Doctoral symposium co-chair, RecSys 2020
- Organized and moderated panel at RecSys 2019 on responsible recommendation
- Co-organizer, TREC Track on Fairness in Information Retrieval (2019–2022)
- PR & Publicity co-chair, 2nd Conference on Fairness, Accountability, and Transparency (ACM FAT* 2019)
- General co-chair, ACM RecSys 2018
- Publications working group, FAccT steering committee (2017)
- Co-organizer, FATREC Workshop on Responsible Recommendation at RecSys 2017, 2018, 2020, 2021
- Co-organizer, Workshop on Fairness, Accountability, Confidentiality, Transparency, and Safety in Information Retrieval (FACTS-IR) at SIGIR 2019
- Co-organizer, FairUMAP workshop at UMAP 2018–2020
- Track co-chair, 2018 Conference on Fairness, Accountability, and Transparency Systems track
- Participant in Dagstuhl Perspectives Workshop Towards Cross-Domain Performance Modeling and Prediction: IR/RecSys/NLP (2017)
- Publicity co-chair, ACM RecSys 2016
- External advisor, CrowdRec (EU Framework Programme collaborative research project, 2014–2016)
- Proceedings co-chair, ACM CHI 2012–2013
- Demos co-chair, ACM RecSys 2012
Department and University Service
- Drexel IS 2023-2024 Faculty Search Committee
- Drexel IS Ph.D. committee (2023-2024)
- Boise State 2020–2021 CS Faculty Search Committee
- Boise State COEN SAGE Scholars Program Mentor (2019–2021)
- Boise State College of Engineering Curriculum Committee (2019–2022)
- Boise State Ph.D. in Computing Steering Committee (2017–2022)
- Boise State CS Dept. Curriculum Committee (2017–2022)
- Boise State CS Dept. Graduate Recruiting Committee (2017)
- Texas State CS Dept. Undergraduate Committee (2014–2016)
- Texas State CS Dept. Written Comp Exam Grading (2014–2016)
- UMN CS Graduate Student Association secretary (2009–2010)
Community and Civic Service
- January 2023 — joined amicus brief before SCOTUS on Gonzalez v. Google.
- July 2020 — taught continuing education session for Idaho Council for Libraries.
- October 2019 — presented at Idaho Library Association Annual Conference.
- February 2019 — addressed Idaho State House Judiciary Committee on H.B. 118, regulating pretrial risk assessment algorithms; through subsequent engagement, I contributed language that is in the final enacted legislation.
- December 2017 — Boise Public Library panel on preparing for a career in computer science.
- 2015 — Judge for Travis Elementary School Science Fair.
Media Mentions
- “Getting to know you: This is what chatGPT says Philly is famous for”. (Vicky Diaz-Camacho, Billy Penn at WHYY, March 11, 2024. https://billypenn.com/2024/03/11/chatgpt-artificial-intelligence-philadelphia-known-for-cheesesteaks/).
- “The Deadline Dilemma”. (Carolyn Kuimelis, Teaching newsletter from Chronicle of Higher Education, December 1, 2022. https://www.chronicle.com/newsletter/teaching/2022-12-01).
- “Out of the Blue”. (Ravi Shankar, The New Indian Express, May 1, 2022. https://www.newindianexpress.com/opinions/columns/ravi-shankar/2022/may/01/outof-theblue-2447591.html). Quotes from Washington Post article below.
- “Elon Musk wants Twitter’s algorithm to be public. It’s not that simple.” (Reed Albergotti, The Washington Post, April 16, 2022. https://www.washingtonpost.com/technology/2022/04/16/elon-musk-twitter-algorithm/).
- Quoted at length about how artificial intelligence learns from social signals in “Can AI be horny?” (Chris Stokel-Walker, Input, April 28, 2021; Bustle Digital Group. https://www.inputmag.com/culture/artificial-intelligence-ai-archillect-twitter-horny-sex).
- Quoted in several articles about FAccT suspending Google’s sponsorship for the 2021 conference, in my role as FAccT Sponsor Co-chair and a member of the Executive Committee. These articles include:
- “AI ethics research conference suspends Google sponsorship.” (Khari Johnson, VentureBeat, March 2, 2021. https://venturebeat.com/2021/03/02/ai-ethics-research-conference-suspends-google-sponsorship/)
- “Conference suspends Google sponsorship after ethics experts’ exit.” (D. Matthews, Times Higher Education, March 8, 2021. https://www.timeshighereducation.com/news/conference-suspends-google-sponsorship-after-ethics-experts-exit)
- “Tech transparency conference suspends Google sponsorship over transparency concerns.” (Colleen Flaherty, Inside Higher Ed, March 9, 2021. https://www.insidehighered.com/news/2021/03/09/tech-transparency-conference-suspends-google-sponsorship-over-transparency-concerns)
- “Google offered a professor $60,000, but he turned it down. Here’s why.” (Rachel Metz, CNN Business, March 24, 2021. https://www.cnn.com/2021/03/24/tech/google-ai-ethics-reputation/index.html). I am not the professor who declined funding, but am quoted for context.
- “How one employee’s exit shook Google and the AI industry.” (Rachel Metz, CNN Business, March 11, 2021. https://www.cnn.com/2021/03/11/tech/google-ai-ethics-future/index.html).
- Quoted about voter file data leaks in “D.C. makes it shockingly easy to snoop on your fellow voters.” (Brian Fung, The Switch [a blog by The Washington Post], June 14, 2016. https://www.washingtonpost.com/news/the-switch/wp/2016/06/14/d-c-s-board-of-elections-makes-it-shockingly-easy-to-snoop-on-your-fellow-voters/)
- Quoted about recommender systems principles in “TV seems to know what you want to see; algorithms at work.” (Scott Collins, Los Angeles Times, November 21, 2014. https://www.latimes.com/entertainment/tv/la-et-st-tv-section-algorithm-20141123-story.html)