Michael D. Ekstrand, Ph.D

Curriculum Vitae

People and Information Research Team (PIReT)
Dept. of Computer Science
Boise State University
1910 University Drive
Boise, ID 83725-2055


https://md.ekstrandom.net
+1 (208) 426-5761

Education

Ph.D (2014)
Computer Science, University of Minnesota. Thesis: Towards Recommender Engineering: Tools and Experiments for Identifying Recommender Differences.
Advisers: John T. Riedl and Joseph A. Konstan
B.S. (2007)
Computer Engineering, Iowa State University.

Employment History

2016–present
Assistant Professor, Dept. of Computer Science, Boise State University
Co-director, People and Information Research Team (PIReT)
2014–2016
Assistant Professor, Dept. of Computer Science, Texas State University
2008–2014
Graduate Research Assistant, GroupLens Research, University of Minnesota
Su 2012, F 2013
Instructor, University of Minnesota
Summer 2010
Research Intern, Autodesk Research, Toronto, CA
2007–2008, S 2011
Teaching Assistant, University of Minnesota
2005–2007
Undergrad Research Assistant, Scalable Computing Laboratory, Iowa State University

Teaching History

Boise State University

  • CS 410/510 (Databases)
  • CS 533 (Intro to Data Science)
  • CS 538 (Recommender Systems)

Texas State University

  • CS 4332 (Intro to Database Systems)
  • CS 3320 (Internet Software Development)
  • CS 5369Q/4379Q (Recommender Systems)
  • CS 4350 (Unix Systems Programming)

Coursera

I co-created the Recommender Systems specialization on Coursera, along with its two previous single-class versions, with Joseph A. Konstan.

University of Minnesota

  • Instructor for CS 5980-1 (Intro to Recommender Systems)
  • Summer instructor for CS 1902 (Structure of Computer Programming II)
  • TA for CSCI 5125 (Collaborative and Social Computing) and CSCI 1902

Ph.D Students

  • Amifa Raj (started Fall 2018)
  • Ngozi Ihemelandu (started Fall 2019)

M.S. Students

Teaching Professional Development

  • Boise State University Center for Teaching and Learning Course Design Institute, a one-week intensive session in Summer 2017.
  • CTL workshops on service learning, mastery-based grading, and other topics.
  • Texas State University’s Program for Excellence in Teaching and Learning (2014–2015).
  • Preparing Future Faculty at the University of Minnesota.

Publications

Author formatting key: myself, advised student, other Boise State student. Citation counts from Microsoft Academic via Microsoft Cognitive Services.

Book Chapters

Daniel Kluver, Michael D. Ekstrand, and Joseph A. Konstan. 2018. Rating-Based Collaborative Filtering: Algorithms and Evaluation. In Social Information Access. Peter Brusilovsky and Daqing He, eds. Springer-Verlag, Lecture Notes in Computer Science vol. 10100, pp. 344–390. ISBN 978-3-319-90091-9. DOI 10.1007/978-3-319-90092-6_10. Cited 29 times.

Journal Publications

Michael D. Ekstrand, Katherine Landau Wright, and Maria Soledad Pera. 2020. Enhancing Classroom Instruction with Online News. Aslib Journal of Information Management (June 2020). DOI 10.1108/AJIM-11-2019-0309.

Michael D. Ekstrand and Michael Ludwig. 2016. Dependency Injection with Static Analysis and Context-Aware Policy. Journal of Object Technology 15(1) (February 2016), 1:1–31. DOI 10.5381/jot.2016.15.5.a1. Cited 3 times.

Joseph A. Konstan, J.D. Walker, D. Christopher Brooks, Keith Brown, and Michael D. Ekstrand. 2015. Teaching Recommender Systems at Large Scale: Evaluation and Lessons Learned from a Hybrid MOOC. Transactions on Computer-Human Interaction 22(2) (April 2015). DOI 10.1145/2728171. Cited 17 times.

Justin J. Levandoski, Michael D. Ekstrand, Michael J. Ludwig, Ahmad Eldawy, Mohamed F. Mokbel, and John T. Riedl. 2011. RecBench: Benchmarks for Evaluating Performance of Recommender System Architectures. Proceedings of the VLDB Endowment 4(11) (August 2011), 911–920. Acceptance rate: 18%. Cited 8 times.

Michael D. Ekstrand, John T. Riedl, and Joseph A. Konstan. 2011. Collaborative Filtering Recommender Systems. Foundations and Trends® in Human-Computer Interaction 4(2) (February 2011), 81–173. DOI 10.1561/1100000009. Cited 549 times.

Conference Publications

These papers have been published in peer-reviewed conference proceedings.

Michael D. Ekstrand. 2020. LensKit for Python: Next-Generation Software for Recommender Systems Experiments. To appear in Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM ’20). arXiv:1809.03125 [cs.IR]. DOI 10.1145/3340531.3412778.

Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand, Asia J. Biega, and Ben Carterette. 2020. Evaluating Stochastic Rankings with Expected Exposure. To appear in Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM ’20). ACM. arXiv:2004.13157 [cs.IR]. Acceptance rate: 21%. Cited 1 times.

Mucun Tian and Michael D. Ekstrand. 2020. Estimating Error and Bias in Offline Evaluation Results. Short paper in Proceedings of the 2020 Conference Human Information Interaction and Retrieval (CHIIR ’20). ACM, 5 pp. DOI 10.1145/3343413.3378004. arXiv:2001.09455 [cs.IR]. NSF PAR 10146883. Acceptance rate: 47%.

Michael D. Ekstrand, Mucun Tian, Mohammed R. Imran Kazi, Hoda Mehrpouyan, and Daniel Kluver. 2018. Exploring Author Gender in Book Rating and Recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems (RecSys ’18). ACM, pp. 242–250. DOI 10.1145/3240323.3240373. arXiv:1808.07586v1 [cs.IR]. Acceptance rate: 17.5%. Cited 29 times.

Michael D. Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D. Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018. All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (FAT* 2018). PMLR, Proceedings of Machine Learning Research 81:172–186. Acceptance rate: 24%. Cited 22 times.

Michael D. Ekstrand, Rezvan Joshaghani, and Hoda Mehrpouyan. 2018. Privacy for All: Ensuring Fair and Equitable Privacy Protections. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (FAT* 2018). PMLR, Proceedings of Machine Learning Research 81:35–47. Acceptance rate: 24%. Cited 12 times.

Michael D. Ekstrand and Vaibhav Mahant. 2017. Sturgeon and the Cool Kids: Problems with Random Decoys for Top-N Recommender Evaluation. In Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference. AAAI, pp. 639–644. Cited 2 times.

Sushma Channamsetty and Michael D. Ekstrand. 2017. Recommender Response to Diversity and Popularity Bias in User Profiles. Short paper in Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference. AAAI, pp. 657–660. Cited 4 times.

Michael D. Ekstrand, Daniel Kluver, F. Maxwell Harper, and Joseph A. Konstan. 2015. Letting Users Choose Recommender Algorithms: An Experimental Study. In Proceedings of the 9th ACM Conference on Recommender Systems (RecSys ’15). ACM. DOI 10.1145/2792838.2800195. Acceptance rate: 21%. Cited 48 times.

Michael D. Ekstrand, F. Maxwell Harper, Martijn C. Willemsen, and Joseph A. Konstan. 2014. User Perception of Differences in Recommender Algorithms. In Proceedings of the 8th ACM Conference on Recommender Systems (RecSys ’14). ACM. DOI 10.1145/2645710.2645737. Acceptance rate: 23%. Cited 78 times.

Joseph A. Konstan, J.D. Walker, D. Christopher Brooks, Keith Brown, and Michael D. Ekstrand. 2014. Teaching Recommender Systems at Large Scale: Evaluation and Lessons Learned from a Hybrid MOOC. In Proceedings of the First ACM Conference on Learning @ Scale (L@S ’14). ACM. DOI 10.1145/2556325.2566244. Acceptance rate: 37%. Cited 36 times.

Tien T. Nguyen, Daniel Kluver, Ting-Yu Wang, Pik-Mai Hui, Michael D. Ekstrand, Martijn C. Willemsen, and John Riedl. 2013. Rating Support Interfaces to Improve User Experience and Recommender Accuracy. In Proceedings of the 7th ACM Conference on Recommender Systems (RecSys ’13). ACM. DOI 10.1145/2507157.2507188. Acceptance rate: 24%. Cited 28 times.

Daniel Kluver, Tien T. Nguyen, Michael Ekstrand, Shilad Sen, and John Riedl. 2012. How Many Bits per Rating?. In Proceedings of the Sixth ACM Conference on Recommender Systems (RecSys ’12). ACM, pp. 99–106. DOI 10.1145/2365952.2365974. Acceptance rate: 20%. Cited 23 times.

Michael Ekstrand and John Riedl. 2012. When Recommenders Fail: Predicting Recommender Failure for Algorithm Selection and Combination. Short paper in Proceedings of the Sixth ACM Conference on Recommender Systems (RecSys ’12). ACM, pp. 233–236. DOI 10.1145/2365952.2366002. Acceptance rate: 32%. Cited 38 times.

Justin J. Levandoski, Mohamed Sarwat, Mohamed F. Mokbel, and Michael D. Ekstrand. 2012. RecStore: An Extensible And Adaptive Framework for Online Recommender Queries Inside the Database Engine. In Proceedings of the 15th International Conference on Extending Database Technology (EDBT ’12). ACM, pp. 86–96. DOI 10.1145/2247596.2247608. Acceptance rate: 23%. Cited 13 times.

Michael D. Ekstrand, Michael Ludwig, Joseph A. Konstan, and John T. Riedl. 2011. Rethinking The Recommender Research Ecosystem: Reproducibility, Openness, and LensKit. In Proceedings of the Fifth ACM Conference on Recommender Systems (RecSys ’11). ACM, pp. 133–140. DOI 10.1145/2043932.2043958. Acceptance rate: 27% (20% for oral presentation, which this received). Cited 112 times.

Michael Ekstrand, Wei Li, Tovi Grossman, Justin Matejka, and George Fitzmaurice. 2011. Searching for Software Learning Resources Using Application Context. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST ’11). ACM, pp. 195–204. DOI 10.1145/2047196.2047220. Acceptance rate: 25%. Cited 27 times.

Michael D. Ekstrand, Praveen Kannan, James A. Stempter, John T. Butler, Joseph A. Konstan, and John T. Riedl. 2010. Automatically Building Research Reading Lists. In Proceedings of the 4th ACM Conference on Recommender Systems (RecSys ’10). ACM, pp. 159–166. DOI 10.1145/1864708.1864740. Acceptance rate: 19%. Cited 64 times.

Michael D. Ekstrand and John T. Riedl. 2009. rv you’re dumb: Identifying Discarded Work in Wiki Article History. In Proceedings of the 5th International Symposium on Wikis and Open Collaboration (WikiSym ’09). ACM, 10 pp. DOI 10.1145/1641309.1641317. Acceptance rate: 36% (Selected as Best Paper). Cited 23 times.

Workshops, Seminars, Posters, Etc.

These papers have undergone some form of peer review, and are published in workshops, poster proceedings, and similar venues.

Amifa Raj, Connor Wood, Ananda Montoly, and Michael D. Ekstrand. 2020. Comparing Fair Ranking Metrics. Presented at the 3rd FAccTrec Workshop on Offline Evaluation for Recommender Systems. arXiv:2009.01311 [cs.IR].

Ashlee Milton, Michael Green, Adam Keener, Joshua Ames, Michael D. Ekstrand, and Maria Soledad Pera. 2019. StoryTime: Eliciting Preferences from Children for Book Recommendations. Demo recorded in Proceedings of the 13th ACM Conference on Recommender Systems (RecSys ’19). 2 pp. DOI 10.1145/3298689.3347048. NSF PAR 10133610.

Michael D. Ekstrand, Ion Madrazo Azpiazu, Katherine Landau Wright, and Maria Soledad Pera. 2018. Retrieving and Recommending for the Classroom: Stakeholders, Objectives, Resources, and Users. In Proceedings of the ComplexRec 2018 Second Workshop on Recommendation in Complex Scenarios (ComplexRec ’18), at RecSys 2018. Cited 2 times.

Mucun Tian and Michael D. Ekstrand. 2018. Monte Carlo Estimates of Evaluation Metric Error and Bias. Computer Science Faculty Publications and Presentations 148. Boise State University. Presented at the REVEAL 2018 Workshop on Offline Evaluation for Recommender Systems, a workshop at RecSys 2018. DOI 10.18122/cs_facpubs/148/boisestate. NSF PAR 10074452.

Michael D. Ekstrand. 2018. The LKPY Package for Recommender Systems Experiments: Next-Generation Tools and Lessons Learned from the LensKit Project. Computer Science Faculty Publications and Presentations 147. Boise State University. Presented at the REVEAL 2018 Workshop on Offline Evaluation for Recommender Systems, a workshop at RecSys 2018. DOI 10.18122/cs_facpubs/147/boisestate. arXiv:1809.03125v1 [cs.IR]. Cited 4 times.

Maria Soledad Pera, Katherine Wright, and Michael D. Ekstrand. 2018. Recommending Texts to Children with an Expert in the Loop. In Proceedings of the 2nd International Workshop on Children & Recommender Systems (KidRec ’18), at IDC 2018. DOI 10.18122/cs_facpubs/140/boisestate. Cited 1 times.

Rezvan Joshaghani, Michael D. Ekstrand, Bart Knijnenburg, and Hoda Mehrpouyan. 2018. Do Different Groups Have Comparable Privacy Tradeoffs?. At Moving Beyond a ‘One-Size Fits All’ Approach: Exploring Individual Differences in Privacy, a workshop at CHI 2018.

Michael D. Ekstrand and Maria Soledad Pera. 2017. The Demographics of Cool: Popularity and Recommender Performance for Different Groups of Users. In RecSys 2017 Poster Proceedings. CEUR, Workshop Proceedings 1905. Cited 1 times.

Michael D. Ekstrand. 2017. Challenges in Evaluating Recommendations for Children. In Proceedings of the International Workshop on Children & Recommender Systems (KidRec), at RecSys 2017.

Michael D. Ekstrand and Martijn C. Willemsen. 2016. Behaviorism is Not Enough: Better Recommendations through Listening to Users. In Proceedings of the Tenth ACM Conference on Recommender Systems (RecSys ’16). ACM. DOI 10.1145/2959100.2959179. Acceptance rate: 36% (Past, Present, and Future track). Cited 22 times.

Jennifer D. Ekstrand and Michael D. Ekstrand. 2016. First Do No Harm: Considering and Minimizing Harm in Recommender Systems Designed for Engendering Health. In Proceedings of the Workshop on Recommender Systems for Health at RecSys ’16. Cited 5 times.

Michael D. Ekstrand. 2014. Building Open-Source Tools for Reproducible Research and Education. At Sharing, Re-use, and Circulation of Resources in Cooperative Scientific Work, a workshop at CSCW 2014.

Michael D. Ekstrand, Michael Ludwig, Jack Kolb, and John T. Riedl. 2011. LensKit: A Modular Recommender Framework. Demo recorded in Proceedings of the 5th ACM Conference on Recommender Systems (RecSys ’11). ACM, pp. 349-350. DOI 10.1145/2043932.2044001. Cited 22 times.

Other Publications and Presentations

These publications are unreviewed reports, preprints, tutorials, etc.

Asia J. Biega, Fernando Diaz, Michael D. Ekstrand, and Sebastian Kohlmeier. 2020. Overview of the TREC 2019 Fair Ranking Track. In The Twenty-Eighth Text REtrieval Conference (TREC 2019) Proceedings (TREC 2019). arXiv:2003.11650.

Adam Roegiest, Aldo Lipani, Alex Beutel, Alexandra Olteanu, Ana Lucic, Ana-Andreea Stoica, Anubrata Das, Asia Biega, Bart Voorn, Claudia Hauff, Damiano Spina, David Lewis, Douglas W Oard, Emine Yilmaz, Faegheh Hasibi, Gabriella Kazai, Graham McDonald, Hinda Haned, Iadh Ounis, Ilse van der Linden, Jean Garcia-Gathright, Joris Baan, Kamuela N Lau, Krisztian Balog, Maarten de Rijke, Mahmoud Sayed, Maria Panteli, Mark Sanderson, Matthew Lease, Michael D Ekstrand, Preethi Lahoti, and Toshihiro Kamishima. 2019. FACTS-IR: Fairness, Accountability, Confidentiality, Transparency, and Safety in Information Retrieval. SIGIR Forum 53(2) (December 2019), 20–43.

Michael D. Ekstrand, Fernando Diaz, and Robin Burke. 2019. Fairness and Discrimination in Recommendation and Retrieval. Tutorial presented at Proceedings of the 13th ACM Conference on Recommender Systems (RecSys ’19). 2 pp. DOI 10.1145/3298689.3346964.

Michael D. Ekstrand, Fernando Diaz, and Robin Burke. 2019. Fairness and Discrimination in Retrieval and Recommendation. Tutorial presented at Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’19). 2 pp. DOI 10.1145/3331184.3331380. Cited 4 times.

Bettina Berendt, Veronika Bogina, Robin Burke, Michael D. Ekstrand, Alan Hartman, Stylani Kleanthous, Tsvi Kuflik, Bamshad Mobasher, and Janna Otterbacher. 2019. FairUMAP 2019 Chairs’ Welcome Overview. In Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization (UMAP ’19). ACM. DOI 10.1145/3314183.3323842.

Alexandra Olteanu, Jean Garcia-Gathright, Maarten de Rijke, and Michael D. Ekstrand. 2019. Workshop on Fairness, Accountability, Confidentiality, Transparency, and Safety in Information Retrieval (FACTS-IR). In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’19). ACM. DOI 10.1145/3331184.3331644.

Katherine Landau Wright, David McNeill, Michael D. Ekstrand, and Maria Soledad Pera. 2019. Supplementing Classroom Texts with Online Resources. At 2019 American Educational Research Association Conference.

Michael D. Ekstrand and Joseph A. Konstan. 2019. Recommender Systems Notation: Proposed Common Notation for Teaching and Research. Computer Science Faculty Publications and Presentations 177. Boise State University. arXiv:1902.01348 [cs.IR]. DOI 10.18122/cs_facpubs/177/boisestate. Cited 2 times.

Nicola Ferro, Norbert Fuhr, Gregory Grefenstette, Joseph A. Konstan, Pablo Castells, Elizabeth M. Daly, Thierry Declerck, Michael D. Ekstrand, Werner Geyer, Julio Gonzalo, Tsvi Kuflik, Krister Lindén, Bernardo Magnini, Jian-Yun Nie, Raffaele Perego, Bracha Shapira, Ian Soboroff, Nava Tintarev, Karin Verspoor, Martijn C. Willemsen, and Justin Zobel. 2018. From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442). Dagstuhl Manifestos 7(1) (November 2018), 96–139. DOI 10.4230/DagMan.7.1.96. Cited 3 times.

Katherine Landau Wright, Michael D. Ekstrand, and Maria Soledad Pera. 2018. Supplementing Classroom Texts with Online Resources. At 2018 Annual Meeting of the Northwest Rocky Mountain Educational Research Association.

Toshihiro Kamishima, Pierre-Nicolas Schwab, and Michael D. Ekstrand. 2018. 2nd FATREC Workshop: Responsible Recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems (RecSys ’18). ACM. DOI 10.1145/3240323.3240335. Cited 3 times.

Bamshad Mobasher, Robin Burke, Michael D. Ekstrand, and Bettina Berendt. 2018. UMAP 2018 Fairness in User Modeling, Adaptation and Personalization (FairUMAP 2018) Chairs’ Welcome & Organization. In Adjunct Publication of the 26th Conference on User Modeling, Adaptation, and Personalization (UMAP ’18). ACM. DOI 10.1145/3213586.3226200.

Nicola Ferro, Norbert Fuhr, Gregory Grefenstette, Joseph A. Konstan, Pablo Castells, Elizabeth M. Daly, Thierry Declerck, Michael D. Ekstrand, Werner Geyer, Julio Gonzalo, Tsvi Kuflik, Krister Lindén, Bernardo Magnini, Jian-Yun Nie, Raffaele Perego, Bracha Shapira, Ian Soboroff, Nava Tintarev, Karin Verspoor, Martijn C. Willemsen, and Justin Zobel. 2018. The Dagstuhl Perspectives Workshop on Performance Modeling and Prediction. SIGIR Forum 52(1) (June 2018), 91–101. DOI 10.1145/3274784.3274789. Cited 7 times.

Michael D. Ekstrand. 2017. Yak Shaving with Michael Ekstrand. CSR Tales no. 4 (December 2017). PURL https://purl.org/mde/alpaca.

Michael D. Ekstrand and Amit Sharma. 2017. The FATREC Workshop on Responsible Recommendation. In Proceedings of the 11th ACM Conference on Recommender Systems (RecSys ’17). ACM. DOI 10.1145/3109859.3109960. Cited 6 times.

Michael D. Ekstrand. 2014. Towards Recommender Engineering: Tools and Experiments in Recommender Differences. Ph.D thesis, University of Minnesota. HDL 11299/165307. Cited 3 times.

Martijn Willemsen, Dirk Bollen, and Michael Ekstrand. 2011. UCERSTI 2: Second Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces. In Proceedings of the 5th ACM Conference on Recommender Systems (RecSys ’11). ACM, pp. 395–396. DOI 10.1145/2043932.2044020. Cited 4 times.

Research Funding

External Grants

Internal Grants

  • 2017: $19K Boise State College of Education Empathy Grant LITERATE: Locating Informational Texts for Engaging Readers And Teaching Equitably (co-PI; with PI Katherine Wright & co-PI Sole Pera)
  • 2014: $8K Texas State University Research Enhancement Program (competitive internal research grant) Temporal Analysis of Recommender Systems (PI)

Invited Talks

Software

I have built several open-source software packages in the course of my research and other work. Open-source software distribution is a key piece of my research dissemination strategy. My more significant development efforts include:

Service

Ongoing Professional Service, Memberships, and Honors

  • Senior Member of the Association for Computing Machinery
  • Executive committee, ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2020–2022
  • Co-chair, FAccT Network, 2019–present
  • Steering committee, ACM Conference on Recommender Systems (RecSys), 2017–present
  • Steering committee, ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2017–present

Program Committee and Editorial Service

  • Guest editor, 2020 special issue of User Modeling and User-Adapted Interaction on fairness in user modeling.
  • Distinguished Reviewer, ACM Transactions on Interactive Intelligent Systems (TiiS) (2017–present)
  • ACM Conference on Recommender Systems (Senior PC 2019, PC 2014–2017)
  • TheWebConf Track on Behavior Analysis and Personalization (2016–2020)
  • User Modeling and Adaptive Personalization (2019)
  • Workshop on Fairness, Accountability, and Transparency in Machine Learning (FATML) (2017)
  • FLAIRS Special Track on Recommender Systems (2015–2017)
  • SAC Recommender Systems track (2013, 2017)
  • Ad-hoc conference reviews for CHI, CSCW, IUI, UIST, WikiSym, UMAP, ICWSM.
  • Journal reviewer for Communications of the ACM; ACM journals TIST, TOIS, TWEB, TKDD, and TIIS; IEEE journals TDSC and TKDE; Interacting with Computers; UMUAI; Informaiton Retreival Jrnal; ACM Computing Surveys; Artifial Intelligence Review; and others.
  • Grant proposal reviews for NSF (US), NWO (NL), FWF (AT)

Other Professional Service

  • Sponsorship co-chair, ACM FAccT 2021–2022
  • Organized and moderated panel at RecSys 2019 on responsible recommendation
  • Co-organizer, TREC 2019 Track on Fairness in Information Retrieval
  • PR & Publicity Co-chair, 2nd Confernece on Fairness, Accountability, and Transparency (ACM FAT* 2019)
  • General co-chair, ACM RecSys 2018
  • Co-organizer, FATREC Workshop on Responsible Recommendation at RecSys 2017 & 2018
  • Co-organizer, FairUMAP workshop at UMAP 2018 & 2019
  • Track co-chair, 2018 Conference on Fairness, Accountability, and Transparency Systems track
  • Participant in Dagstuhl Perspectives Workshop Towards Cross-Domain Performance Modeling and Prediction: IR/RecSys/NLP
  • Publicity co-chair, ACM RecSys 2016
  • External advisor, CrowdRec (EU Framework Programme collaborative research project, 2014–2016)
  • Proceedings co-chair, ACM CHI 2012–2013
  • Demos co-chair, ACM RecSys 2012

Department and University Service

  • Boise State College of Engineering Curriculum Committee (2019–present)
  • Boise State Ph.D in Computing Steering Committee (2017–present)
  • Boise State CS Dept. Curriculum Committee (2017–present)
  • Boise State CS Dept. Graduate Recruiting Committee (2017)
  • Texas State CS Dept. Undergraduate Committee (2014–2016)
  • Texas State CS Dept. Written Comp Exam Grading (2014–2016)
  • UMN CS Graduate Student Association secretary (2009–2010)

Community Service

  • October 2019 — presented at Idaho Library Association Annual Conference.
  • February 2019 — addressed Idaho State House Judiciary Committee on H.B. 118, regulating pretrial risk assessment algorithms.
  • December 2017 — Boise Public Library panel on preparing for a career in computer science.
  • Judge, 2015 — Travis Elementary School Science Fair.