Tuesday, April 24, 2007

Personalization, Google, and discovery

Gord Hotchkiss, a columnist at the popular Search Engine Land, posts some thoughts on Google Web History and Google's aggressive moves in personalized search.
This allows Google to ... [tap] into your current browsing behavior to try to determine what's on your mind right now ... It helps Google interpret just the kind of site you want to see, given your behavior at the present time.

[Google can] find similar sites you may have never considered, based on the characteristics of the sites you have been visiting ... It's not just providing you a shortcut to sites you are already aware of, it's in making you aware of new sites you never knew existed, ranked and prioritized according to the PageRank algorithm.

The promise of personalization is moving Google to be a true recommendation engine.
Search is about finding. Recommendations are about discovering.

It is hard to find something if you do not know it exists. It is hard to find something if you cannot easily say exactly what you want.

Personalization can help you discover information you would not have found on your own, sites you never knew existed, based on your past behavior and interests. Personalization and recommendations aid discovery.

Update: A few days later, Gord Hotchkiss publishes an interview on personalization with Googlers Marissa Mayer and Sep Kamvar.

Update: In yet another article, Gord Hotchkiss says:
As Sep and his team begin to refine personalization, expect it to be aggressively rolled into multiple aspects of your Google experience.

[Personalization is] the engine that will power the future of Google for the foreseeable future. It will eventually surpass the PageRank algorithm in importance, giving Google the ability to match content to very specific and unique user intent on the fly.

7 comments:

Mike Dierken said...

Good link, and good commentary.
Thanks.

jeremy said...

Personalization can help you discover information you would not have found on your own, sites you never knew existed, based on your past behavior and interests. Personalization and recommendations aid discovery.

I still don't see it, Greg. I understand search/finding. I understand personalization/recommendation. But it seems like you're talking about some sort of recommendation through searching? Where you search for something, and the results you get back are a mixture of what you're actually looking for, and what the system thinks you should be looking for? Is that what you mean?

If so, that goes far beyond all the previous "personalized search" examples, which were essentially "I know that you like cars.. so when you search for 'jaguar', we are not going to show you the animal"

And how is this different from Exploratory Search (my big interest)? In exploratory search, the user has only a vague idea of what they're looking for, and a vague notion of how to ask for it (what query to use). The goal of the system is to provide them tools to help the searcher negotiate their way through a search space that they really do not know much about.

Is that kinda what you are getting at with this notion of inserting personalized recommendations into a search results page? Or am I totally missing the mark?

Greg Linden said...

Hi, Jeremy. I admit I am being pretty vague about what personalization and personalized search means here.

Focusing on personalized search, I primarily mean adding information about you have done in the past to what you are doing right now.

So, for example, a search for "jaguar" may have more results for the car than the norm if you had expressed an interest in cars, especially if you had expressed that interest very recently (such as a search immediately previous for "sports car").

The major difference here is that personalized search breaks the assumption that each search is independent. Someone who searches for [A] then [B] will see different results than someone who searches for just [B]. There almost certainly is valuable information in what you find or did not find in the first search that should change what you see when you follow up with a second search.

jeremy said...

Focusing on personalized search, I primarily mean adding information about you have done in the past to what you are doing right now. [snip] The major difference here is that personalized search breaks the assumption that each search is independent. Someone who searches for [A] then [B] will see different results than someone who searches for just [B]. There almost certainly is valuable information in what you find or did not find in the first search that should change what you see when you follow up with a second search.

Ok, so at the risk of being a pain, how is this different from "search as a dialogue"? Y'know.. the idea that instead of just doing multiple independent searchs you open up a search "session". Everything that happens within that session is cumulative.

So it seems you can break the idea of independent searches [A] and [B], but still not necessarily have or even require "personalization".

In fact, let me submit to you the idea of the Vivisimo/Ask/Quintura interface as another approach that breaks the idea of independent searches. You can think of the first search as query [A], and then when the user clicks on the refinement term, that is query [B]. What the user gets as a result of query [B] is more than if they had just searched using the refinement term selected in step B. They get results "personalized" toward query [A].

And the beautiful thing is that, if you look at the total amount of work done, the person clicking the Vivisimo modification links has done less total work than the "personalization" user. The personalization user has entered two full queries. The Vivisimo user has entered one query and one click.

Anyway, I apologize if this comes across as chip-on-shoulderish. I'm really just trying to clearly delineate some of these ideas. I very much enjoy your comments about these things. The ideas are interesting, no matter what we name them.

Greg Linden said...

Hey, Jeremy. I agree there are similarities between personalized search and search as a dialogue.

However, in most versions of search as a dialogue I have seen, the dialogue is usually explicitly shown and short-term (just the last few items, not a longer history).

So, I think that is a substantial distinction. Personalized search may have a long history of actions (searches and clicks) quietly influencing search results.

I do think you raise a good point that there is overlap betwen some of these approaches. After all, we're all after the same thing, improved relevance and usefulness.

jeremy said...

Ok, so just so I'm clear here.. personalized search defined by the following criteria:

(1) No "extra" work on the part of the (lazy) user. Improvements to search results occur through the user's normal habit of entering text into search boxes.

(2) The current search is not an independent search. It is influenced by the short term of searches that the same user performed in the immediate past, as well as by the long term of one's broader search and click profile.

Correct? Other than that, do you make any sort of distinctions about how the past influences the present?

I ask, because I see a lot of variation at that level, a lot of things that get called "personalization" that sometimes seem to be, sometimes seem not to be. For example, my dissertation advisor favored "user modeling" as the preferred personalization embodiment:
Relevance feedback and personalization: A language modeling perspective

[Read this link above; it's from 2000/2001 and is quite interesting]

However, you seem to favor a collaborative filtering / Amazon recommendation-like type of personalization.

I spoke briefly with Peter Norvig a few months ago, and he seemed to favor a user clustering approach, where the results that you would see were not necessarily because of only the things you did. You would instead be clustered into groups of similarly-behaving users, and your results would be influenced by what they did (almost a sort of user model / Amazon recommendation hybrid, if I understood Norvig correctly).

But you don't care what the implementation is, right? It is all just "personalization", so long as it fits the "long term profile" requirement, correct?

Greg Linden said...

Yep, thanks, Jeremy, that's well stated. I would define personalized search as no extra work, the user's searches are no longer be independent of each other, and different people see different search results depending on their click and search history.

I would also agree that a variety of techniques could support personalized search, though you are right that I tend to argue for approaches that look like collaborative filtering.

I probably should be careful to say this is only my definition. Others may disagree.