Wednesday, May 26, 2004

Why do personalized search?

Personalized search is showing different results to different people on the same search. Rather than just using the keywords provided for a search, everything the search engine knows about a person is brought to bear and impacts the search results.

The goal is to provide more relevant search results. Since different people have different interpretations of what is relevant, at some point, the only way to further improve the quality of search results will be to show different people different results. So, the advantage of personalized search is that it promises to deliver better search results. The difference is likely to be particularly substantial when a search is very ambiguous (e.g. a single word like "desk") or someone has difficulty finding something and refines a search repeatedly.

What are the disadvantages?

First, privacy is an issue. Search engines would build a profile of everything you tell them about your interests, every search you've done, and every search result you ever clicked on. That's a lot of information and, unless handled in the strictest confidence, could make many people nervous.

Second, search results will no longer be consistent for the same search, certainly not for searches by different users and not even for many searches by the same user. This means that e-mailing a search to someone or easily finding something by searching for it again become a bit more difficult.

Third, personalized search is computationally expensive. Caching and many other optimizations become impossible when every search result list is customized in real-time.

Will the advantages outweigh the disadvantages? Big players are betting yes. Google, Microsoft, A9, and Yahoo are all testing personalized search.

4 comments:

Seun Osewa said...

"This means that e-mailing a search to someone or easily finding something by searching for it again become a bit more difficult. "

This calls for better internet bookmark management systems. The ideal solution should be as portable as a9's search history yet as accessible as Internet Explorer's bookmark folder. The various providers should define common standards.

Greg Linden said...

You make a good point on the need for better bookmarking systems. You might be interested in my earlier post on Seruku, Furl, and Microsoft's "Stuff I've Seen". It describes some methods for searching your entire browsing history.

Lisa G said...

Hey, do you know about Eurekster? It's semi-personalized search with a social software bent... As you and your network perform searches, it filteres up results for sites that you actually visited for more than a minute. The thing that is good about it is that the group creates the context. So a search for 'desk' would filter different results if you were in a woodworking group than in an office workers group (where references would be more like 'chained to a desk!').

Aside from that, I wouldn't mind giving up some privacy (do I have any, anyway?) to have search engines feed me more relevant results!

Greg Linden said...

Thanks for the tip. I've seen Eurekster before. I'm not sure I would call it personalized search. It doesn't help you find new information faster. It just ranks higher any search result that you or your friends clicked on in the past. That's closer to A9's search history feature (with a social networking twist) than to search personalization.

I'm pretty skeptical about Eurekster. To me, it sounds like someone looked at the hottest Internet Tech trends -- social networking, web search, and personalization -- and slapped them together without much regard to the quality of the product.