Monday, May 22, 2006

AI and the future of search

Saeed Shah at The Independent (UK) reports that:
Google's ultimate aim is to create a search engine with artificial intelligence to exactly answer any question a user puts to it.

Larry Page, the co-founder and president ... said: "People always make the assumption that we're done with search. That's very far from the case. We're probably only 5 per cent of the way there. We want to create the ultimate search engine that can understand anything ... some people could call that artificial intelligence."

Mr Page ... said it was not possible to predict when Google would achieve this goal, although he pointed out that "a lot of our systems already use learning techniques".
See also my previous posts, "Google and The Happy Searcher", "Search without searching" and "Google and question answering".

[Found on Findory]

Update: An excerpt from a similar article by Richard Wray in the UK Guardian:
"The ultimate search engine would understand everything in the world. It would understand everything that you asked it and give you back the exact right thing instantly," Mr Page [said] ... "You could ask 'what should I ask Larry?' and it would tell you."

Mr Page said one thing that he had learned since Google launched eight years ago was that technology can change faster than expected, and that AI could be a reality within a few years.
One thing you have to say about the Google founders, they certainly are ambitious.

See also more of my earlier posts, "The perfect search", "Perfect search and the clickstream", and "Different visions of the future of search".

[UK Guardian article via Threadwatch]


Anonymous said...

I'm not bullish on the pure AI approach. Pure or Strong AI (ala Searle) only butts heads with Turing, and will not solve it. Yahoo's approach, on the other hand, doesn't even battle with Turing.

Using machine-based approaches *and* the collective intelligence of the human race is the approach that will win out in the end.

Greg Linden said...

Hi, Peter. I don't agree with this description of Google's approach as "machine-based" and Yahoo's approach as human-based.

All of Google's algorithms rely on the collective intelligence of humans. PageRank summarizes what pages people think are important, as indicated by their links. Human opinion and knowledge aggregated by PageRank is then used to surface the human knowledge stored in the documents of the Web. Google News is built on top of the collective intelligence of news editors and news readers. The optimization of Google's advertising relies on the collective behavior of people.

You can't emulate PageRank with people. You can't optimize AdWords ads manually. At large scale, it would take far too much time and be far too expensive to use people directly to manually summarize, filter, and sort out the intelligence from the collective crap.

But, it is a mistake to look at this aggregation and summarization and then think that Google does not seek out the human wisdom stored in the Web. That is, in fact, all they do.

Anonymous said...

I don't mean to say pure emulation by humans is yahoo's approach. no; but, Yahoo's approach ads context and sensitivities that Google's approach does not provide.

I agree with you that Google automated aggregation seeks to collectively make sense of the human choice and what humans deem as important using links as the proxy. On it's face, this is very cool. But, this approach is really not that smart. Machine-only methods (yes, google does do this) cannot make sense of sarcasm, metaphor, word-sense disambiguation and other difficult problems. You say that optimizing adsense is based on human behavior -- this is partly true. the serving-up of these ads are purely based, however, on context, sense, and structure of the content.

Perhaps google does not do the strong AI as I suggested; but, they're approach of pure automated aggregation without a human face to it, is less effective at making sense of the web. Yahoo's approach seeks to make sense of the Web via tagging and allowing users to add context to links and content.

Somewhat related to this post is an interesting look into the future of Google. Here's the url for it.

Greg Linden said...

That's a fun article, Peter! Thanks for mentioning it!

I think the jury is still out on whether additional user-generated content, like tagging or contributed links, is a solution by itself.

Because of the profit motive, I suspect those new forms of user-generated stuff will become dominated by spam, just like all the user-generated content on the Web as a whole, and, like the Web, will require a lot of effort to filter out the crap.

Udani said...

I agree that user-generated content like tagging has worked well for cases where there has been no strong revenue model. If there was a proven revenue model (like Adwords) around Flickr, Wikipedia and other similar approaches then people would have tried to spam it and thereby bring the overall quality down.

Rolando Hernandez said...

I thought you'd like to see this article Search is Not the Answer where Matthew Glotzbach, head of products for Google Enterprise, says "Question answering is the future of enterprise search."

Maybe Matthew read Greg's blog about AI and the future of Search few months ago.

"Question answering" sounds a lot like AI expert systems doesn't it?

Google is clearly moving in that direction, as Greg pointed out.