Abstract

Every day, we interact with countless information systems enhanced by artificial intelligence: Google helps us find what we seek, Amazon and Netflix recommend things for us to buy and watch, Apple News gives us the day’s events, BuzzFeed guides us to related articles, and VISA assesses whether our morning coffee is a fraudulent purchase. These systems deliver immense value, but also have profound influence over how we experience information and the resources and perspectives we see.

In this talk, I will explain the basic principles of modern data-driven artificial intelligence systems, and present some of the work that I and others are doing to measure their human impact and ensure that they are good for the people they affect: that they provide value, minimize harm, and produce outcomes that treat people fairly. I will pay particular attention to recommender systems, the algorithms that suggest products and articles, and how they do or do not meet their users’ needs.

My Related Research

Michael D. Ekstrand, Michael Ludwig, Joseph A. Konstan, and John T. Riedl. 2011. Rethinking The Recommender Research Ecosystem: Reproducibility, Openness, and LensKit. In Proceedings of the Fifth ACM Conference on Recommender Systems (RecSys ’11), Oct 24, 2011. ACM, pp. 133–140. DOI 10.1145/2043932.2043958. Acceptance rate: 27% (20% for oral presentation, which this received). Cited 259 times. Cited 198 times.

Michael D. Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D. Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018. All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (FAT* 2018), Feb 23, 2018. PMLR, Proceedings of Machine Learning Research 81:172–186. Acceptance rate: 24%. Cited 324 times. Cited 226 times.

Michael Ekstrand and John Riedl. 2012. When Recommenders Fail: Predicting Recommender Failure for Algorithm Selection and Combination. Short paper in Proceedings of the Sixth ACM Conference on Recommender Systems (RecSys ’12), Sep 10, 2012. ACM, pp. 233–236. DOI 10.1145/2365952.2366002. Acceptance rate: 32%. Cited 90 times. Cited 73 times.

Michael D. Ekstrand, F. Maxwell Harper, Martijn C. Willemsen, and Joseph A. Konstan. 2014. User Perception of Differences in Recommender Algorithms. In Proceedings of the 8th ACM Conference on Recommender Systems (RecSys ’14), Oct 6, 2014. ACM, pp. 161–168. DOI 10.1145/2645710.2645737. Acceptance rate: 23%. Cited 301 times. Cited 194 times.

Michael D. Ekstrand and Vaibhav Mahant. 2017. Sturgeon and the Cool Kids: Problems with Random Decoys for Top-N Recommender Evaluation. In Proceedings of the 30th International Florida Artificial Intelligence Research Society Conference (Recommender Systems track), May 29, 2017. AAAI, pp. 639–644. No acceptance rate reported. Cited 25 times. Cited 11 times.

Michael D. Ekstrand, Rezvan Joshaghani, and Hoda Mehrpouyan. 2018. Privacy for All: Ensuring Fair and Equitable Privacy Protections. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (FAT* 2018), Feb 23, 2018. PMLR, Proceedings of Machine Learning Research 81:35–47. Acceptance rate: 24%. Cited 107 times. Cited 81 times.

Michael D. Ekstrand and Martijn C. Willemsen. 2016. Behaviorism is Not Enough: Better Recommendations through Listening to Users. In Proceedings of the Tenth ACM Conference on Recommender Systems (RecSys ’16, Past, Present, and Future track), Sep 17, 2016. ACM, pp. 221–224. DOI 10.1145/2959100.2959179. Acceptance rate: 36%. Cited 156 times. Cited 102 times.

Works Cited

• Franklin, Ursula M. 2004. The Real World of Technology. Revised Edition. Toronto, Ont.; Berkeley, CA: House of Anansi Press.

  • 1989 CBC Massey Lectures

• Resnick, Paul, Neophytos Iacovou, Mitesh Suchak, Peter Bergstrom, and John Riedl. 1994. “GroupLens: An Open Architecture for Collaborative Filtering of Netnews.” In ACM CSCW ’94, 175–86. ACM. doi:10.1145/192844.192905.

• Sarwar, Badrul, George Karypis, Joseph Konstan, and John Reidl. 2001. “Item-Based Collaborative Filtering Recommendation Algorithms.” In ACM WWW ’01, 285–95. ACM. doi:10.1145/371920.372071.

• Sarwar, Badrul M, George Karypis, Joseph A Konstan, and John T Riedl. 2000. “Application of Dimensionality Reduction in Recommender System – A Case Study.” In WebKDD 2000. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.8381.

• ACM Code of Ethics

• Rishabh Mehrotra, Ashton Anderson, Fernando Diaz, Amit Sharma, Hanna Wallach, and Emine Yilmaz. 2017. Auditing Search Engines for Differential Satisfaction Across Demographics. In Proceedings of the 26th International Conference on World Wide Web Companion (WWW ’17 Companion).

  • Early version in DAT Workshop

• Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In Advances in Neural Information Processing Systems 29 (NIPS 2016).

• Robyn Speer. 2017. ConceptNet Numberbatch 17.04: better, less-stereotyped word vectors.

• Neil Hunt. 2014. 🎞 Quantifying the Value of Better Recommendations.

• David Streitfield. 2017. ‘The Internet Is Broken’: @ev Is Trying to Salvage It. New York Times, May 20, 2017.

  • Bibblio.org. 2017. Ad-driven media doesn’t care about the truth, and that isn’t going to change

• The Conference on Fairness, Accountability, and Transparency