Abstract

Recommender system evaluation usually focuses on the overall effectiveness of the algorithms, either in terms of measurable accuracy or ability to deliver user satisfaction or improve business metrics. When additional factors are considered, such as the diversity or novelty of the recommendations, the focus typically remains on the algorithm’s overall performance. We examine the relationship of the recommender’s output characteristics — accuracy, popularity (as an inverse of novelty), and diversity — to characteristics of the user’s rating profile. The aims of this analysis are twofold: (1) to probe the conditions under which common algorithms produce more or less diverse or popular recommendations, and (2) to determine if these personalized recommender algorithms reflect a user’s preference for diversity or novelty. We trained recommenders on the MovieLens data and looked for correlation between the user profile and the recommender’s output for both diversity and popularity bias using different metrics. We find that the diversity and popularity of books in users’ profiles has little impact on the recommendations they receive.

Resources

• Copy in the FLAIRS Proceedings
• Author version PDF (provided in compliance with the AAAI Author Agreement)
• Scripts and code to reproduce
• FLAIRS Poster