Tuesday, June 09, 2009

Approaching the limit on recommender systems?

An article at the upcoming UMAP 2009 conference, "I like it... I like it not: Evaluating User Ratings Noise in Recommender Systems" (PDF), looks at inconsistencies when people rates movies.

What is particularly interesting about the article is that the authors argue that "state-of-the-art recommendation algorithms" are nearing the lower bound on accuracy imposed by inconsistencies in ratings, which they and previous work refer to as the "magic barrier".

Natural variability in people's opinions limits how accurate recommender systems can be. According to this paper, we may be almost at that limit already.

Update: For more on this topic, it turns out one of the authors of the paper, Xavier Amatriain, has a blog post, "Netflix Prize: What if there is no Million $ ?", with a good comment thread.

1 comment:

Xavier Amatriain said...

Thanks for the link Greg. You might be interested in the post on my blog where there has been an interesting discussion thread relating our article to the Netflix prize.