Thursday, November 13, 2008

Baidu on next generation search

Baidu Chief Scientist William Chang gave an industry day talk at CIKM 2008 on the next generation of search where he predicted an industry-wide move toward personalized web search.

William first described two earlier generations of search, then said the upcoming third generation of search would be the "internet as a matching network". He expected to see an integration of search and recommendations, where recommendations are used to find related concepts, entities, phrases, documents, and sources.

As part of this, he expected to see a renewed interest in the diversity of search results -- he described it as relevance versus user satisfaction -- that appeared to be going down the path of information exploration and search as a dialogue to better help searchers with the task behind the search phrase.

William also briefly mentioned personalized spidering. While he did not elaborate, I would guess he meant software agents gathering, synthesizing, and summarizing information from several deep web sources to satisfy a complicated task.

This talk was one of the ones recorded by videolectures.net and should appear there in a week or so.

Please see also my earlier post, "Rakesh Agrawal at CIKM 2008", that summarizes Rakesh's predictions about a new interest in diversity and discovery in web search.

Please see also my earlier post, "More on computational advertising", which summarizes Andrei Broder's CIKM talk on integrating of advertising and recommendations.

Update: William Chang asked me to add that "this talk reflects my personal views on cutting-edge search, not necessarily the company's."

4 comments:

jeremy said...

As part of this, he expected to see a renewed interest in the diversity of search results -- he described it as relevance versus user satisfaction -- that appeared to be going down the path of information exploration and search as a dialogue to better help searchers with the task behind the search phrase.

I missed this talk, and look forward to it appearing on videolectures.

In the meantime, could you explain what you/he means by relevance vs. user satisfaction? Is it that (personalization == relevance) and (search-as-dialogue == user satisfaction)? Or is something else meant?

Syed Rizvi said...

very interesting article. Can't wait for the talk to appear on videolectures.net

Greg Linden said...

Hi, Jeremy. Oops, sorry, that isn't very clear from my summary, is it?

I think what William meant was that showing the most relevant results for a query (the closest match to the search terms) might not be the best thing for long term satisfaction.

Rather, you might want to show some documents that are not as good of a match to the search terms if that improves diversity and helps more people eventually get to the information they need.

I think this comes up, for example, when there could be multiple intents behind the same query or when the query is so vague it probably will need to be refined. In those cases, it might be more helpful to searchers to give a diverse set of results rather than show the closest matches to the search terms.

Daniel Tunkelang said...

Count me in with the folks excited to see this lecture. It sounds a lot meatier that the talk he gave at the SIGIR industry event a few years ago. And of course I'm predisposed towards approaches that embrace the subjectivity of relevance.