Thursday, January 18, 2007

The value of recommendation engines

Alex Iskold at Read/WriteWeb writes about "The Art, Science and Business of Recommendation Engines".

Some excerpts:
There are two fundamental activities online -- Search and Browse. When a consumer knows exactly what she is looking for, she searches for it. But when she is not looking for anything specific, she browses.

It is the browsing that holds the golden opportunity for a recommendation system, because the user is not focused on finding a specific thing -- she is open to suggestions.

During browsing, the user's attention (and their money) is up for grabs. By showing the user something compelling, a web site maximizes the likelihood of a transaction. So if a web site can increase the chances of giving users good recommendations, it makes more money.

Amazon is considered a leader in online shopping and particularly recommendations. Over the last decade the company has invested a lot of money and brain power into building a set of smart recommendations that tap into your browsing history, past purchases and purchases of other shoppers -- all to make sure that you buy things.

The Amazon system is phenomenal. It is a genius of collaborative shopping and automation that might not be possible to replicate. This system took a decade for Amazon to build and perfect. It relies on a massive database of items and collective behavior that also "remembers" what you've done years and minutes ago.
I like Alex's description of searching versus browsing.

Search helps when you know what is out there and can easily say what you want. Personalization and recommendations help when you do not know what is out there. Personalization surfaces interesting items you did not know about and may not have found on your own.

And, Alex is right that recommendations can be lucrative. Personalization and recommendations apparently are responsible for 35% of sales at


Steve Flinn said...

Yes -- it seems to me if one thinks most broadly about search and recommendations, then search is just a subset of recommendations -- the results of a search request is a recommendation where the recipient happens to be executing a search. A reasonably high degree of confidence regarding user intensionality can be inferred from a search request, which makes it an "easier" problem than delivering more generalized recommendations.

The process of delivering what we call recommendations must typically (unless within a very limited domain) infer more vague intentionalities on the part of the user, and so is the harder, but ultimately more interesting problem.

Greg Linden said...

That's a great point, Steve.

In general, I think you can think of recommendations as a search problem. We are building an implicit search based on past behavior.

For example, we can build a profile based on past subjects and keywords from a clickstream and then execute searches against loose combinations of those constraints.

Explicit search terms can then supplement the implicit search. And, in fact, that is essentially the structure of the personalized search built by Kaltix and now used in Google Personalized Search.

srikanth thunga said...

there is a gap between the recommendation engines of amazon and web links.

there is money to be spent when you search for a book which makes sure that you research properly before you buy it.

on the other hand, to click on a link, what would you lose? half of the links I click on lead to uninteresting info. this cant be considered for recommendations..

but say, if the user can also say what he likes and what he dislikes, a deadly recommendation engine can be built. findory with would be a deadly combination..

this model works well with movie recommendation or amazon book recommendations because they also rate at the end..

on a completely different note, I think that google cannot do recommendations because most of the time what I search for are the things that I know very little about but after finding it out. My google search terms have everything under the earth.. How can you recommend something to a person who searches for say microsoft, rocky balboa and pineapple pastries :)

For all you know, I might like only pineapple pastries of the 3 but google would assume all 3.. so there are already 2 false positives..

my bet is on a system which combines the power of findory and and not on googles personalized search..

Steve Flinn said...

Greg (and to Srikanth's points too),
Yeah, agree -- search and recommendations are clearly merging domains driving toward the common goal of increasing adaptation to users depending on their inferred intentional state.

One of things we've found in our R&D on recommendations functions is that with access to broader sets of behaviors available in richer behavioral environments (such as with Amazon, or in our adaptive knowledge network site, more effective adaptive recommendations can be delivered than with regard to just the "Plain Old Web (POW)" For example, we have found that how people reference information for future access and organize such information is particularly useful data (much better than just clickstreams or even tagging-type information).

It will be interesting to see how things evolve between highly open systems with access to fewer behavioral cues, and environments that extend POW to enable capture of more types of usage behaviors.