Abstract

Recommender systems, search engines, and other modern information discovery and access technologies have broadened both access to information and the ability to disseminate new information. Every day, people use these tools to locate, collate, and understand information, and they in turn shape their users’ understanding of the information space, and through it the world and society they inhabit. Information discovery has profound possibilities for connecting people with information, products, culture, and people they wouldn’t encounter in other ways, but can also reproduce a range of harms including discrimination, negative or unnecessary stereotypes, information silos, and more.

In this talk, I discuss the goals of information retrieval, access, and discovery, and how those goals translate to specific measurement and evaluation designs. My central thesis is that effective, ethical, and pro-social technologies for information discovery must be grounded both theoretically and empirically in a clear understanding of system goals, and robust operationalizations of those goals into system and evaluation designs.