When I interviewed at Amazon, I arrived full of ideas. I rattled off a dozen ways I thought I could improve the Amazon website, including an idea about offering book recommendations.
I love books. I loved the idea of a recommender system for books. What fun, discovering new, interesting books. I wanted to make it happen.
In my first week at Amazon, I was disappointed to discover that some at Amazon were already working on a book recommender system. Early users of Amazon.com might remember it. It was called BookMatcher.
BookMatcher users would start by rating 20-30 books, then get recommendations. As you can imagine, few used it. The hurdle of rating 20+ items was too much for most users.
The software that powered BookMatcher was provided by an outside firm. Unfortunately, it didn't work very well. The initial hurdle was just one issue. The recommendations tilted toward bestsellers and away from the tail. The system was slow. Topping it all off, the system was unreliable, falling down under load several times.
This wasn't what Amazon needed. Book recommendations at Amazon needed to work from sparse data, just a few ratings or purchases. It needed to be fast. The system needed to scale to massive numbers of customers and a huge catalog. And it needed to enhance discovery, surfacing books from deep in the catalog that readers wouldn't find on their own.
There had to be a better way.