Making Information Systems Good for People

I gave this talk on March 2, 2018 at Boise State Computer Science’s Senior Seminar.

Abstract

Every day, we interact with countless information systems enhanced by artificial intelligence: Google helps us find what we seek, Amazon and Netflix recommend things for us to buy and watch, Apple News gives us the day’s events, BuzzFeed guides us to related articles, and VISA assesses whether our morning coffee is a fraudulent purchase. These systems deliver immense value, but also have profound influence over how we experience information and the resources and perspectives we see.

In this talk, I will explain the basic principles of modern data-driven artificial intelligence systems, and present some of the work that I and others are doing to measure their human impact and ensure that they are good for the people they affect: that they provide value, minimize harm, and produce outcomes that treat people fairly. I will pay particular attention to recommender systems, the algorithms that suggest products and articles, and how they do or do not meet their users’ needs.

My Related Research

RecSys11

2011

Michael D. Ekstrand, Michael Ludwig, Joseph A. Konstan, and John T. Riedl. 2011. Rethinking The Recommender Research Ecosystem: Reproducibility, Openness, and LensKit. In Proceedings of the Fifth ACM Conference on Recommender Systems (RecSys ’11). ACM, pp. 133–140. DOI 10.1145/2043932.2043958. Acceptance rate: 27% (20% for oral presentation, which this received). Cited 255 times. Cited 195 times.

FAT18-ck

2018

Michael D. Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D. Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018. All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (FAT* 2018). PMLR, Proceedings of Machine Learning Research 81:172–186. Acceptance rate: 24%. Cited 290 times. Cited 212 times.

RecSys12-f

2012

Michael Ekstrand and John Riedl. 2012. When Recommenders Fail: Predicting Recommender Failure for Algorithm Selection and Combination. Short paper in Proceedings of the Sixth ACM Conference on Recommender Systems (RecSys ’12). ACM, pp. 233–236. DOI 10.1145/2365952.2366002. Acceptance rate: 32%. Cited 88 times. Cited 73 times.

RecSys14

2014

Michael D. Ekstrand, F. Maxwell Harper, Martijn C. Willemsen, and Joseph A. Konstan. 2014. User Perception of Differences in Recommender Algorithms. In Proceedings of the 8th ACM Conference on Recommender Systems (RecSys ’14). ACM. DOI 10.1145/2645710.2645737. Acceptance rate: 23%. Cited 284 times. Cited 187 times.

FLAIRS17-s

2017

Michael D. Ekstrand and Vaibhav Mahant. 2017. Sturgeon and the Cool Kids: Problems with Random Decoys for Top-N Recommender Evaluation. In Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference (Recommender Systems track). AAAI, pp. 639–644. No acceptance rate reported. Cited 16 times. Cited 11 times.

FAT18-fp

2018

Michael D. Ekstrand, Rezvan Joshaghani, and Hoda Mehrpouyan. 2018. Privacy for All: Ensuring Fair and Equitable Privacy Protections. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (FAT* 2018). PMLR, Proceedings of Machine Learning Research 81:35–47. Acceptance rate: 24%. Cited 104 times. Cited 78 times.

RecSys16

2016

Michael D. Ekstrand and Martijn C. Willemsen. 2016. Behaviorism is Not Enough: Better Recommendations through Listening to Users. In Proceedings of the Tenth ACM Conference on Recommender Systems (RecSys ’16, Past, Present, and Future track). ACM. DOI 10.1145/2959100.2959179. Acceptance rate: 36%. Cited 142 times. Cited 95 times.

Works Cited

Franklin, Ursula M. 2004. The Real World of Technology. Revised Edition. Toronto, Ont.; Berkeley, CA: House of Anansi Press.
- 1989 CBC Massey Lectures
Resnick, Paul, Neophytos Iacovou, Mitesh Suchak, Peter Bergstrom, and John Riedl. 1994. “GroupLens: An Open Architecture for Collaborative Filtering of Netnews.” In ACM CSCW ’94, 175–86. ACM. doi:10.1145/192844.192905.
Sarwar, Badrul, George Karypis, Joseph Konstan, and John Reidl. 2001. “Item-Based Collaborative Filtering Recommendation Algorithms.” In ACM WWW ’01, 285–95. ACM. doi:10.1145/371920.372071.
Sarwar, Badrul M, George Karypis, Joseph A Konstan, and John T Riedl. 2000. “Application of Dimensionality Reduction in Recommender System – A Case Study.” In WebKDD 2000. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.8381.
ACM Code of Ethics
Rishabh Mehrotra, Ashton Anderson, Fernando Diaz, Amit Sharma, Hanna Wallach, and Emine Yilmaz. 2017. Auditing Search Engines for Differential Satisfaction Across Demographics. In Proceedings of the 26th International Conference on World Wide Web Companion (WWW ’17 Companion).
- Early version in DAT Workshop
Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In Advances in Neural Information Processing Systems 29 (NIPS 2016).
Robyn Speer. 2017. ConceptNet Numberbatch 17.04: better, less-stereotyped word vectors.
Neil Hunt. 2014. 🎞 Quantifying the Value of Better Recommendations.
David Streitfield. 2017. ‘The Internet Is Broken’: @ev Is Trying to Salvage It. New York Times, May 20, 2017.
- Bibblio.org. 2017. Ad-driven media doesn’t care about the truth, and that isn’t going to change
The Conference on Fairness, Accountability, and Transparency