Jim argued that personalization doesn't work and then tried to contrast it with something he called "collective search". Some excerpts:
Ask.com ... aims to tap the collective search habits of its 50 million users to improve the relevancy of Web search.If I could quote from my favorite movie, "You keep using that word. I do not think it means what you think it means."
[Jim Lanzone] said attempts at automated personalization often fail in practice to give users what they want.
Instead, Web search can be improved by understanding the aggregate behavior of different types of users.
This collective approach means users stand to benefit from what users with similar interests have gleaned from previous searches.
"Collective search is something that Ask really believes in," Lanzone said, adding that personalizing what different users see is only a small piece of further improving search.
I admit the term personalization may be poorly defined these days, but it is hard for me to see the distinction between personalized search and changing the search results based on "what users with similar interests have gleaned from previous searches."
In fact, I would think that is the very definition of personalization. Personalization changes what people see based on their past behavior and the past behavior of others.
Perhaps the distinction here is that collective search may change results even for people who have no history? For example, popular search results may get a higher ranking or search results that appear to be related after analyzing what people click on may be handled differently?
Yet even that often still is referred to as personalization. For example, one of Amazon.com's most successful and useful personalization features is similarities ("Customers who bought X also bought"). That feature is targeted to a specific page, not to a user's history, but is still personalization.
Am I missing something more here? Is Jim suggesting collective search includes something that personalization does not? For example, does collective search include explicit sharing of search results across a social network (like Yahoo's struggling MyWeb)? Or something else?
For more details on Jim's interview, see also Tamar Weinberg's notes over at Search Engine Roundtable.
4 comments:
Yet even that often still is referred to as personalization. For example, one of Amazon.com's most successful and useful personalization features is similarities ("Customers who bought X also bought"). That feature is targeted to a specific page, not to a user's history, but is still personalization.
One of the reasons I keep commenting on your blog is that I am also struggling (through no fault of your own) to tease out the distinction, if there even is one.
I do not know the answer, but let me make the following small thought experiment: Starting with your Amazon, tied to the page example, let's suppose that, for the next 24 hours, no one in the world bought the item on a particular page. Suppose, that is, that the history of that item were frozen. Now, suppose furthermore that, in that 24 hours, everyone in the world visits that page. It's just that no one buys anything.
Now, the first question in this thought experiment is: Are the recommendations offered on this page based on that item's history, as it has interacted with (been bought by) people all over the world? Yes.
The second question is: With all the people in the world now visiting this page, over a 24 hour period, are the recommendations that each person sees different? Or are they all the same? I think the answer (please correct me if not!) is that the recommendations are all going to be the same.
So from that perspective, it seems like "personalization", as you have often applied it to users and their own history, is not really happening here. If everyone in the world visits this page over a 24 hour period, everyone is going to see the same recommendations. There will be nothing personal about any of the recommendations. If I happen to like "jaguar" the car and you happen to like "jaguar" the animal, we'll both still see the same recommendations, for that item, on that page.
I think the distinction that Jim Lanzone may be drawing here is the distinction between "item" personalization and "person" personalization. In the Amazon example, the item itself is actually being "personalized", in that the item sees a different set of recommended other items on its page, depending on that items history of interactions with those other items, through the users with whom it comes in contact.
But what is not happening (again, if I have understood all this correctly, which is not necessarily very likely) is personalization of the people visiting this item. Each person sees the exact same recommendations; the recommendations are global, and therefore not "personal".
Just some thoughts. Someone please correct me if I've skipped a logical step somewhere.
To me the real question is should result rankings differ on a per-user basis, a per-group basis, or be the same for all users.
So far, i haven't seen evidence per-user results make much of a difference. Perhaps ask is taking the per-group approach - whether or not this is really different from past "personaliation" technologies based on collaborative filtering is probably more of a quibble about terminology than a meaningful answer.
Personally I still think that a single user can be in multiple "modes". I would prefer to tell Google I am in "medical mode" or "shopping mode" and then see the same results everyone else does.
The really useful thing si if the actual ranking criteria were adjustable. I believe MSN / Live had a demo up that did it awhile back.
Its just a feeling but for a LOT of queries, I'm not so sure that user A's needs for a query are appreciably different than user B, even if they are radically different "kinds" of people. For instance, I have a blog post that is by far my most accessed post because it is #1 for the query "fishbone throat" http://www.google.com/custom?q=fishbone+throat
- the thread has taken a life of its own on and honestly its hard to think of anyone who is googling the topic looking at the result and not finding it useful.
Short story: google as currently configured is pretty damn good.
In my opinion, I think this is a case where Jim is trying to confuse the marketplace and differentiate Ask.com (i.e. make it more relevant).
Also, how would it work where a user with no history is made better by this approach? Surely they have to do *something* to be put into a classification whereby their search could be improved (assuming that underlying technique is in fact what data miners call "clustering"), right? As I see it (and correct me if I'm wrong here), doing optimization without user history to make the experience better already has a name: experiment design. (something which, I'm told, Amazoners are intimately familiar)
How is "the aggregate behavior of different types of users" similar or different than the concept of searching through a 'lens'?
Post a Comment