Tuesday, April 24, 2007

Social search and the same old problems

Matthew Hurst pointed to a talk by Yahoo Researcher Prabhakar Raghavan on "The changing face of web search" (PDF). The talk covers good ground, including an overview of some issues in web advertising and of a few types of social applications.

What particularly caught my attention were slides 11-14 on social search. On slide 13, Prabhakar lists challenges in social search:
How do we use these tags for better search?

How do you cope with spam?

What's the ratings and reputation system?
Rephrasing slightly, the challenges are how to we know what user contributed data matters and how do we determine how reliable that information is?

But, wait a second. Those are the same problems we face with traditional web search. Web pages and links between web pages are created by people. We need to determine how reliable those pages and links are and what information in those pages is useful.

Prabhakar says as much two slides earlier, noting that "the wisdom of the crowd can be used to search" but also saying "the principle is not new -- anchor text [is] used in 'standard' search." That is, the wisdom of the crowd, in traditional web search, is in the web pages and web links. In social search, users can create additional snippets of content than web pages. That additional content can also be used in search.

But the problem remains the same. What information is useful? What is the reliability of that information? And, as slide 14 suggests, trying to solve these problems in social search probably looks similar to solving them in traditional search, mostly involving propagating usefulness and trust along the associations and links between the pieces of data.

It makes me wonder how much leverage there is from the idea of social search. Does social search change the nature of the problem? Does the new data somehow help solve what otherwise would be hard problems for traditional search? Or do the same old problems follow those who try to move to social search?

See also my previous post, "Chris Sherman on social search".

1 comment:

jeremy said...

But, wait a second. Those are the same problems we face with traditional web search. Web pages and links between web pages are created by people. We need to determine how reliable those pages and links are and what information in those pages is useful.

One difference I can think of is that with web search, the barrier to entry for creating a spammy link is much higher than in the social tagging realm. Sure, it is much easier today than it was ten years ago to get a small webpage up and running. But it remains easier still to socially tag. Even with things like free web pages, the problem is different. A search engine can see that certain links are coming from a "Geocities" (for example) free hosted page, and automatically discount links from that page. There are no easy/clear filters like that for social tagging.

As a result, while the problems may be abstractly the same, the practicalities and barriers to entry for web pages make it easier; the lack of barriers make social tagging much harder. IMHO.