Letting Users Choose Recommenders

Switching Algorithms

This is a paper about a drop-down menu. I’ll be presenting on it in the first session on Monday at RecSys 2015; the work is joint work with Daniel Kluver, Max Harper, and Joe Konstan at GroupLens.

In our previous paper, we examined what algorithms users say they prefer and the differences they perceive between those algorithms. This paper asks the follow-up question: when users are allowed to select the algorithm they actually use, what do they do?

The new version of MovieLens is now the generally-available MovieLens, rather than a beta. When Max and his team rolled out this version, we added the switch that we promised users in the list comparison study: a menu to pick the recommender that they would use. We logged users’ actions on this menu, and looked for a few things:

The paper has the details, and my talk will have quite a few of them. But here’s the executive summary:

There were 2 other interesting results we observed. First, users who experimented with the menu were more likely to come back to the site. It is important to note that we do not have evidence that this is causal — it is quite likely that more active users are more prone to experiment with the menu.

Second, there were objectively measurable differences in the output of the different algorithms.

Read the paper for more details, and I hope to see you in Vienna.