Google movie shortcut

Gary Price posts details on Google's new movie shortcut. Stick a "movie:" prefix on a keyword search and it'll bring up a list of matching movies and movie reviews.

It appears to search in the movie title, description, and reviews, making it a good way to find movies that match a rough description or to find a movie if you can't remember the exact title.

Here are movie searches for "monty python" and for "hong kong crime".

The movie reviews appear to be spidered from the web, online news sources, and Usenet newsgroups. Google does something similar for the product reviews in Froogle.

Amazon.com can't be happy about this trend. Product and movie reviews at Google are an alternative to the product reviews at Amazon and the movie reviews at Amazon-owned IMDB.

TiVo, Flickr, WSJ, Wikinews

A few recent tidbits worth reading:

George Hotelling summarizes the rumors that Apple might be buying TiVo.

Om Malik summarizes the rumors that Yahoo might be buying Flickr.

Adam Penenberg slams the Wall Street Journal Online, saying that their subscription model dooms them to irrelevance.

Simon Waldman says Wikinews isn't very good. He blames the rapidly changing nature of news. "A news operation starts with a clean slate every day," he says, making it a different and harder problem than Wikipedia.

AOL Local Search

Michael Bazeley writes about AOL's entry into the crowded land of local search.

For part of AOL Local, Topix.net will provide local news, another excellent partnership in Topix's recent deal-making spree. Congrats, Rich!

See also my earlier post, "Local search is hard".

Tivo Suggestions paper

I love TiVo. I can't imagine TV without it.

TiVo has a feature, TiVo Suggestions, that tries to recommend shows you might like. It's a great idea, helping viewers discover new shows they might not know about. Unfortunately, the quality is mediocre.

I've been curious why the recommendations are poor. And now I can find out. Kamal Ali and Wijnand van Stam from TiVo published a paper at KDD 2004, "Tivo: Making Show Recommendations Using a Distributed Collaborative Filtering Architecture" (MS Word doc file).

The paper says:

TiVo uses an item-item (show to show) form of collaborative filtering which obviates the need to keep any persistent memory of each user's viewing preferences at the TiVo server .... It uses k-nearest neighbor with Pearson correlation to make show recommendations ... The collaborative filtering system is augmented on the client by a content-based Bayesian recommendation system to address the cold start problem for new users and shows.

TiVo uses a combination of a form of collaborative filtering and a Bayesian content analysis to make recommendations. The paper only describes the collaborative filtering-like analysis in any detail.

An early clue on the quality problems is here:

We have not performed formal empirical evaluations of its accuracy .... The number one item on our agenda for future work is a thorough empirical evaluation of the quality of suggestions.

They haven't tried to measure the quality of the recommendations. This is not good. Personalization is hard. It is not a solved problem. You can't just throw a particular algorithm at the problem and assume it will work well. Experimentation is crucial. Without a quality measure, they have no way of finding improvements to their recommendations or even knowing if a change is an improvement.

Getting into what kind of improvements could be made, let's start by summarizing their current system. Each TiVo is essentially a low-end PC connected to TiVo's server farm with a slow and only intermittently available pipe (modem over phone line).

At a high level, TiVo Suggestions currently works by sending all the ratings across to their server cluster from each TiVo, generate correlations between the shows, and send back the list of correlations to each TiVo. Because of the slow pipe and the amount of data to be processed, only 1/16 of the correlations are computed each day, so the correlations take up to 16 days to update. Each client TiVo computes new recommendations "at least once a day" using the correlation data.

The algorithm they are currently using, "item-item collaborative filtering", is designed for real-time, high performance applications. Since the recommendations don't need to be computed in real-time, just "at least once a day", it's worth considering whether the recommendations should be computed entirely on the server cluster. If they aren't updating the recommendations in real-time, a real-time algorithm probably is not the best choice.

They compute only 1/16 of the correlations each day, apparently because of computational resource constraints on their cluster, but this creates a bad user experience. Instead, if they really are hardware constrained, sampling or other data reduction techniques could improve performance without reducing quality.

They say they haven't experimented yet with biasing toward more recent data, another way of reducing data and improving the apparently quality of the recommendations.

They are using Pearson correlation as their similarity metric. The choice of similarity metric can have a broad impact on recommendations, biasing the recommendations toward more surprising or unusual recommendations or pushing them into the mainstream.

They made a variety of decisions elsewhere that impact recommendation quality. They combine predictions using a weighted linear average. They have a variety of thresholds in their correlation computation (min-pair, min-single, etc.). They arbitrarily setting the confidence ranges for the content and collaborative filtering-based predictions. Experimentating with other choices here could find improvements.

It's a shame. TiVo is great, but more and more PVRs are coming on the market. Recommendations could be a key differentiator for TiVo.

If my TiVo knows me, knows what I like, and helps me find it, the experience on any other PVR would seem hollow, like I just lost a best friend. With the current quality of TiVo Suggestions, my TiVo feels more like a clueless stranger than a close friend.

[paper via Matt Haughey]

Matching content to audiences

Newspapers are increasingly talking about personalized news. Mary Lou Fulton (VP, Bakersfield Californian) recently said:

What if we thought about it a different way, and thought of our pool of content (local, wires, syndicated content, etc) separately from a single newspaper. We could slice and dice that content differently and come up with many versions of the publication for many audiences.

Mary Lou's comments are similar to what Tom Curley (CEO, Associated Press) proposed a few months ago.

Content is king. Newspapers should exploit new distribution systems to get their content in front of the right audiences.

Yahoo, the long tail, and personalization

Havi Hoffman has some interesting tidbits on personalization in his post on the Yahoo Search Blog.

The mainstream is not the only stream. There's room now for babbling brooks, crooked creeks, and tributaries where trends pick up momentum before they flow downstream. There's an audience for many voices.

We all need long-tail tools that support self-expression, personalized search, recommendations, and trust.

The long tail is a sea of voices, all seeking an audience. Exposing the long tail is about matching voices to audiences, helping people discover the rare and unusual.

Personalization exposes the long tail. It learns your interests, digs deep, and surfaces the gems from the internet's cacophony.

Fantastic to see Yahoo thinking about personalized search and recommendations in the long tail.

Local search is hard

John Battelle posts on competing in the local search and yellow pages market:

I asked [Peter Negulescu (VP, SFGate)] why SFGate isn't an aggressive player in the local online advertising market ... He chided me for my ignorance regarding the 800-pound gorilla of local markets -- the Yellow Pages. "They have like 6-700 local sales people in every major region," he told me. "They visit every merchant in town." In other words, the Chronicle can never compete.

This is the problem with local search. There's millions of itty bitty little merchants, all appearing and disappearing rapidly. It's hard to get accurate information on them. It's expensive to manage the advertising accounts for them. Local search is hard to do right.

John is optimistic that local merchants would come to Yahoo or Google's website to self-manage their information and advertising, but I doubt you could get anything like the coverage you need with a self-service model. Many of these merchants don't even have a web site. They aren't tech savvy.

The Yellow Pages has uses hundreds of sales people because it has to. They need to physically send salespeople out to talk to each merchant. Getting complete coverage requires a massive, expensive sales force. That's why local search is hard.

See also "Down on local search".

VCs and the "me too zone"

Brad Feld (Director, Mobius Venture Capital) writes about trend-following in the VC industry:

Every emerging market hits a point where there is a mad rush of early stage entrepreneurs and VCs piling in. In some cases, it drives rapid innovation; in most it creates near term over-saturation, lots of irrational financings, and plenty of carnage as the laws of Darwin play out over the next couple of years.

I'm afraid we just hit that point with RSS / Blogging. The meme has spread broadly -- which is great. Now we'll watch all gods children pile in to try to get something up and running in this "space" .... It's as if the whole world wakes up one day and starts working on the same types of things.

There is a huge adoption (and innovation) curve ahead of everyone who is doing stuff with RSS / blogging -- and there are plenty of good investments left to make and companies to create -- but the noise and clutter is about to get really loud.

The herding instinct is strong. Brad's right, this one will be fun to watch.

Autolink in Google Toolbar

Gary Price writes about the latest version of Google Toolbar. It includes a feature called Autolink:

With "AutoLink" enabled, web pages will be "enhanced" with additional links if Google thinks additional information might be helpful. For example, say your browsing a web page with numerous addresses on it. AutoLink will turn each of those addresses into direct links to the Google Maps database.

In addition to addresses, it will also add links for ISBNs (books, mostly), package tracking numbers, and vehicle identification numbers. Gary says, "The commercial possibilities are massive."

Useful, but this one makes me a little squeamish. Google is now modifying the web pages displayed. The modifications are useful, sure, but what does it do to Mapquest to have all addresses everywhere pointing to Google Maps? What does it do to Barnes & Noble if all ISBNs point off to Amazon?

Taking this a step further, what if your web browser rewrote all your web pages? Would you be happy if IE pointed all addresses it recognized off to Encarta Maps?

Tuning search to individuals

Interesting tidbit on Yahoo's plans for personalized search from a HBS Working Knowledge article:

Yahoo is able to collect data, such as click patterns, from its users and use the information to individually "tune" searches to users' personal tendencies. Such product improvements, [Yahoo Director Bradley] Horowitz said, will change the nature of search from a one-size-fits-all experience to a more individually oriented one.

Horowitz was part of a panel at the "Search Visionary" session at the Cyberposium conference at Harvard Business School.

Findory Favorites

Findory's newest feature, Favorites, makes it easy for readers to easily see their favorite news and weblog sources.

For example, on the BBC page, clicking the "Add Favorite" button at the upper right will put a link to BBC articles in your "My Favorites" list. You can also mark weblogs such as Boing Boing or Gizmodo as favorites.

Reading news on Findory is easy. Read a few articles. Get personalized recommendations. Click on a few sources. Add a few favorites. Read more articles. Get more personalized recommendations. It's a virtuous cycle. The more you use Findory, the better it gets!

Contrast that with a using a normal feed reader. First, you hunt down a few RSS feeds. Manually cut-and-paste URLs into the feed reader. Hunt down more feeds. Manually add them. Skim all your feeds desperately looking for interesting articles. Realize that many of the feeds are boring and useless. Remove those feeds. Laboriously hunt down more feeds. Realize you now have too many feeds and can't read them all. Remove some feeds. Ugh. You spend more time configuring and customizing than reading news.

Customization and personalization are at opposing ends of a spectrum. Most feed readers are all the way at the extreme of customization, requiring readers to do work -- lots of work -- to be able to read news.

Findory is a different. It's personalized and learns what you need. It requires no effort. It's easy. It all just works.

Yahoo building weblog search?

Steve Rubel and Nathan Weinberg speculate that Yahoo is building a weblog search similar to Technorati and Feedster after seeing evidence of a new Yahoo RSS crawler.

See also my earlier post, "Will Technorati die?"

Implicit vs. explicit sharing

Ross Mayfield (CEO, Socialtext) slams personalization in favor of social software:

The basic problem with Personalization is that tailoring information to you limits social discovery. Users contribute value to the database only for them and the service provider, not for each other ... Nowhere in this mode is sharing, conversations, remixing and socializing information.

By contrast, consider how social software enables people to create their own networks. Groups form, information is shared and implicit and explicit relationships are fostered.

Ross asserts that personalization limits social discovery and doesn't allow users to share data with each other. That's false.

Personalization relies on the strength of the community. It is the shared data between users that allows the system to find relationships and make recommendations.

But there is one major difference. With personalization, sharing is implicit and anonymous. With social software, sharing is explicit and public.

There are two problems with explicit sharing:

1. It requires work. A lot of work. People don't like to do work.
2. The quality of recommendations is low. Why? Chris Anderson said it best: "No matter who you are, someone you don't know has found the coolest stuff." Limiting discovery to what your friends have found cripples the system.

Ross partially acknowledges these issues at the end of his article:

The appeal of personalization is sheer convenience. Today social software fails, with a few exceptions, to deliver the same level of convenience at scale, but give it time.

Unfortunately, Ross, it's not just a matter of time. Most people want something that requires no effort and just works. Social software can't deliver that.

Discovery in the information stream

Rich Skrenta (CEO, Topix.net) has a long post that touches on the incremental web, streaming information based on keywords vs. topic, human vs. automated aggregators, and discovering content in the long tail. An excerpt:

There are 4-8 million active blogs now. At this size, you can still "know" the top bloggers, and find new posts worth reading by clicking around. But when the blogosphere grows 100X or 1000X, the current discovery model will break down. You'll need algorithmic techniques like Topix.net or a Findory to channel the most relevant material from the constant flood of new content.

I think it's worse than Rich says. I think the current discovery model has already broken down.

Even if you monitor just a few tens of sources, you are facing a daily stream of hundreds or thousands of articles. It's a painful, overwhelming task to manually skim it hunting for relevant content. There is precious little discovery in the current model.

First, make sure it works

Todd Bishop at the Seattle PI uncovers some amusing flaws in MSN Search's advertising.

One particular MSN Search ad suggests that you should search for "Pizza in Redmond". Curiously, Todd finds, MSN Search has the worst results by far on that query of any of the major search engines.

MSN is rumored to be spending $150M for this advertising campaign. I'm not sure they're getting their money's worth.

Jeeves loves Mozilla

Ask Jeeves EVP Tuoc Luong says that they are considering making Ask Jeeves Desktop Search open source, partnering with the Mozilla Foundation, and releasing a branded version of the Firefox browser.

I like the idea of making Ask Jeeves Desktop Search open source. With the flood of desktop search applications, releasing the source for Ask's product is a way to spur adoption and continued innovation.

As for a branded version of Firefox, I'm skeptical about how many people would download it, but any users it had probably would tend to come to Ask Jeeves frequently.

Google recently embraced the Mozilla Foundation as well, hosting the default Firefox start page and paying the salaries for two of the lead developers.

Google to help Wikimedia

Google has offered to help Wikimedia -- the non-profit that maintains the free, community-maintained, and very cool Wikipedia encyclopedia -- with hosting and bandwidth.

Apparently, the offer has no strings attached, no requirement for advertising.

This is a generous move by Google. But it is also true that Google has a lot to gain from supporting Wikipedia and other Wikimedia projects.

By using Answers.com for definitions, Google already indirectly includes some Wikipedia content. Google may be planning to include more encyclopedia content, perhaps to counter the Encarta integration into MSN Search, perhaps just to improve the usefulness and relevance of search results.

Like Google's support of Firefox, Google's generosity is not without self-interest. Nevertheless, it's encouraging and gratifying to see Google helping out these tech non-profits.

[via Slashdot]

Social computing at Microsoft Research

Bernard Moon has a fun article on his visit to the Social Computing Group at Microsoft Research.

"We started with this little project ... [where] we looked at our emails, scanned them, and built [clusters] based on who you communicate with." This simple program -- which users don't have to input anything for or do anything with -- dynamically changes over time. The system automatically adds people to users' lists and groups them based on who users email and copy.

The most interesting application to come out this idea is an email program the group dubbed Inner Circle, which lists alphabetically a user's top 40 correspondents ... When an Inner Circle user clicks on a person in that list, the program displays all the conversations the user has had with that person as well as any relevant documents or links that were sent.

Another interesting application ... is Wallop, a group social network and blogging environment ... It includes a level of interactivity that other [blogging] sites can't currently match: You can post comments on just about anything (such as pictures or text), and you can mark an item -- not just the overall profile of a hot guy or girl -- on a person's page as a "favorite."

Fun and very interesting. There's great work going on in Microsoft Research. Unfortunately, it often seems to be stuck there.

As [MSR GM] Lili Cheng said, they are "like a startup group within Microsoft that has to pitch to other groups for them to learn about and consider our ideas." If you put up a hurdle that high, few are going to be able to jump it.

[via Robert Scoble]

All of the world's information

David Kirkpatrick at Fortune interviewed Marissa Mayer (PM, Google). I particularly liked what she had to say about expanding Google's content:

One of our themes now is "more of the information you want." That implies huge index sizes. Now it's eight billion pages, and we'll get really aggressive, growing into more forms of content, like print from books, and video.

Eric [Schmidt] had a great quote the other day. He said, "When we said all of the world's information, we meant all." So that means ... there's very little content we're not interested in.

Watch out, hidden web, Google is looking for you.

CNet's new feed reader

Steve Rubel reports that CNet is about to launch its own feed reader, Newsburst. Welcome to the club, CNet.

There's already integrated feed readers in Firefox, My Yahoo, and My MSN. Ask Jeeves now has Bloglines. CNet will have Newsburst. Oh, wait, there's more, hundreds of other RSS readers, all fairly similar. There's even several newspapers trying to offer their own feed readers.

The products are largely indistinguishable. Launching undifferentiated products into a crowded space is futile. With giants like My MSN, My Yahoo, and Firefox already major players, the world doesn't need yet another feed reader.

What the world needs is innovation. Solve the problems ([1] [2]) with current feed readers. Come up with new ways to read news. Help people find what they need.

Google analyst day

Google had their analyst day today. Long but worthwhile.

It's interesting to see how Google describes themselves (slide #6) and their competitive advantages (slide #7). Eric Schmidt (CEO) and Sergey Brin (co-founder) see continued innovation at Google as being critical to their success, talking at length about their people, how much time they devote to experimentation (slide #12), and how they set up their environment to encourage creativity.

Eric talked about their model for advertising, how auctioning the ad space ensures a fair and competitive price for advertisers, that their advertising engine gets its success from showing fewer but higher quality and more relevant ads, and how Google seeks the long tail (slide #10) of small businesses as advertisers.

Larry Page (co-founder) did a demo of some of Google's new products, including Google Maps (which really is amazing) and Keyhole.

Eric mentioned "end-user personalization" briefly which he described as "a Google that, sort of, knows you, knows more about you." In response to a question, he elaborated by saying:

There are many things we can learn about a person without having them give us a name and a password. And we can use those to provide better results.

It is clear that Google sees personalization as critical to future improvements in relevance.

Feeds drive traffic

Christine Mohan (Director, NYTimes.com) talks about how RSS drives traffic to the newspaper's website

NYTimes.com's RSS feeds generated 4.5 million page-views on the site in January. RSS is an efficient traffic driver that complements other distribution deals we have with top portals, ISPs, and aggregators like Moreover Technologies. RSS users also represent a news-hungry, influential audience for NYTimes.com.

RSS feed are essentially advertisements for the newspaper's content. Newspapers that fail to publish feeds are dropping valuable traffic on the floor.

The RSS explosion

Kevin Laws at VentureBlog posts an excellent article on the future of RSS. A great overview.

Kevin also talks a bit about personalized news and feed readers:

Now that a wider variety of web sites are available in machine readable format, it should be possible to tell your computer things like "tell me when an article about gnosticism appears".

RSS is more evolution than revolution ... [It] has finally pushed machine-to-machine content communication over the tipping point. That certainly allows some interesting and very large opportunities, particularly in search and collaborative filtering.

What makes RSS interesting is that machines can easily read and process news feeds. This is where the future of RSS lies.

Current feed readers merely reformat RSS feeds for display. Future feed readers will rip apart the content, analyze the data, and help you find the information you need.

Home page of the 21st century

Jim Lanzone (SVP, Ask Jeeves) quotes Mark Fletcher (CEO, Bloglines) as saying he wants to build the "homepage of the 21st century."

What is the home page of the 21st century? What do you want from a home page?

I think it should be your view on the world. It should tell you about events that matter to you. It should tell you, this what happened that you need to know about today.

I think it should focus your attention. It should prioritize. It should learn what you want, what you need, and how to best help you find it.

What do you think? What would be your ideal home page?

Google and eBay pursue sellers

Bambi Francisco at CBS Marketwatch looks at competition between Google and eBay:

Will search advertising ultimately become a better mousetrap for sellers? Or to what extent will search advertising take away the potential dollars that were once expected to flow onto eBay's marketplace?

As one eBay seller said to me via e-mail: "Most sellers are like me and didn't really think there was life outside of eBay, but we are getting educated fast." Another seller said he's decided to put up his own Web site for $19.95 per month and buy keywords on Google and Yahoo's Overture.

"Products on Google are not under as much pressure as eBay so you can typically get 5 to 10 percent more for your products on Google," said Scot Wingo of ChannelAdvisor.

See also my earlier post, "Google, small businesses, and eBay".

Update: John Battelle says:

It seems that eBay, which got hammered after its last earnings for not living up to its reputation as a growth machine, is blaming Google for increasing its cost of goods sold. Now that's interesting.

Update: Two years later, a BusinessWeek article reports the eBay "magic is gone ... Shoppers are simply not buying all the inventory anymore. Some items languish without a single bidder. Many shoppers opt for other sites including Amazon.com, use sophisticated search engines such as Google and Yahoo!, or head to store sites directly."

Google Maps

Chris DiBona posts about Google Maps, a new competitor to Mapquest and Yahoo Maps.

The maps are gorgeous and easy to read, the interface simple and uncluttered. There are no advertisements whatsoever. Hard to imagine using Yahoo Maps and Mapquest after seeing Google's offering.

Google Maps is still in the labs. Address searches on Google.com still point to Yahoo Maps and Mapquest. I'm sure that will change soon.

The search war and the browser war

Looking at MSN's revenue numbers, Joe Wilcox at Microsoft Monitor argues that the search war is deeply intertwined with the browser war:

I don't believe the browser as giveaway technology holds true anymore. Paid search creates real revenue opportunity around the browser.

MSN Search ... is ... Internet Explorer's default search engine. I would consider any dip in IE browser market share as a threat to MSN Search -- and one Microsoft shouldn't ignore.

It's a good point. By default, anyone powering up a new computer will use IE. Their web search will be MSN Search. The first web page they see every day will be MSN.com. This guarantees MSN a large customer base. If some computers come with another browser installed or enough people are motivated to install a new browser themselves, then that is a threat to MSN's revenue stream.

But I think Joe understates the importance to Microsoft of the continued dominance of IE. IE was created because Netscape was a threat to the Windows desktop. Networked applications running through a browser reduce the importance of the OS and desktop applications. Loss of IE market share threatens not just MSN but all of Microsoft.

Ask Jeeves buying Bloglines?

Mary Hodder claims Ask Jeeves is buying Bloglines, official announcement apparently coming on Monday Tuesday.

Bloglines is one of the most popular and highly reviewed feed readers. However, there was some question about whether Bloglines could survive independently in an increasingly crowded space, especially as Yahoo, MSN, Mozilla, Apple, and others start integrating feed readers into their products.

[via Steve Rubel]

Update: It's true. The announcement on the official Bloglines weblog, the press release, and press coverage. Kudos and congratulations to Mark Fletcher!

Update: Jim Lanzone (SVP, Ask Jeeves) gives some details. Mark Fletcher (CEO, Bloglines) promises to keep building Bloglines, including integrating the Teoma search engine into Bloglines blog search.

Busy, busy Findory

Findory keeps growing and growing! In the last month, we added:

Findory Neighbors: A new visualization tool for discovering related blogs. Check out the neighborhood for Geeking with Greg or for your blog!

Related articles feeds: Find articles related to your favorite weblogs and news sources using our RSS feeds and inline widget. For example, for Wired magazine, here's the related articles feed and inline widget. I subscribe to the related articles feed for my own blog and use it to discover blog posts I might otherwise have missed.

Personalized search feeds: Another first from Findory! We have personalized feeds for news and blog keyword searches. We highlight the search results that are recommended for you based on your reading history.

Got blogs?: We doubled the size of our weblog crawl. Not on Findory yet? Add your Blog to Findory.

Get big fast: To handle our rapid growth, we brought two new servers online, including a new database server. As usual, we did it without anyone noticing, just as it should be.

Findory's personalization depends on the strength of our community. Thank you, readers of Findory. Together we are building an exciting new way of reading news.

Advertising and the internet

John Markoff and Nat Ives at the New York Times provide some interesting insights into web advertising programs like Google's AdWords and Yahoo's Overture:

"In the past, advertising has been hard to track and hard to make accountable," said [Google VP] Tim Armstrong ... Now, he said, advertising has become a dialogue with the consumer.

Individually focused Internet ads are already siphoning business away from locally oriented classified advertising and yellow pages directories.

"You're seeing advertising move into advertising that people can seek out, and moving away from mass advertising," said [UC Berkeley Adjunct Professor] Peter Sealey.

What is particularly remarkable about the success of AdWords is that it is all done with simple, unobtrusive, text-only ads. The ads are so well targeted that they become useful and informative. Rather than trying to grab your attention with flashy popups or graphics, these ads just try to be relevant and helpful.

See also "Bringing sense to web advertising".

Y!Q contextual search

Yahoo just launched Y!Q, a "contextual search technology that analyzes the contents of the Web page you're viewing and then gives you a list of search results directly related to what you're reading."

From Jeremy Zawodny's post on the Yahoo Search blog:

The fundamental idea was to supplement search queries with context. So instead of having to spend a lot of time searching and assembling all the information you're after, this contextual search technology could incorporate that context (the stuff you were reading at you moment you decided that you wanted to know more) to find the most relevant results.

It appears that Y!Q does keyword extraction from a web page, then runs a search using the extracted keywords. Not a new idea, but interesting to see it launched by Yahoo. If the quality of the search results is high, it can easy way to discover related web pages. But it's quite a challenge to get high quality related pages with this technique.

Jeremy points to Reiner Kraft as the primary researcher behind Y!Q. And Reiner has a recent paper, "Mining Anchor Text for Query Refinement" (PDF), that seems related to the techniques behind Y!Q. The paper describes one of several approaches for doing this kind of keyword extraction.

Social networks vs. personalization

Chris Anderson explains why social networks don't work well for recommendations:

No matter who you are, someone you don't know has found the coolest stuff.

The sad reality is that most of my friends have rotten taste in music (I don't hold it against them), while the music recommendations I actually follow are mostly from people I've never met.

The assumption that there's a correlation between the people I like and the products I like is a flawed one.

Friendships are not a good predictor for recommendations. Your friends aren't clones of you.

For recommendations, reach out to the entire community, including total strangers, to find people with shared interests. There is wisdom in that crowd. All you need to do is find it.

Google and personalized search

Saul Hansell at the IHT has an interesting tidbit buried in his article about Google's Q4 2004 numbers:

[Google CEO Eric] Schmidt said one priority for Google would be the personalization of a search, taking advantage of information about the interests and attributes of each user to present better results.

Google hasn't done much with their personalized search demo since they launched it back in March 2004. I wonder what they're planning?

See also "A real personalized search from Google?"

A beautiful day in the neighborhood

Alex Edelman has the scoop on Findory's nifty new visualization tool, Findory Neighbors. It allows readers to quickly and easily discover related weblogs and news sources.

Here's the neighborhoods for Geeking with Greg, Micro Persuasion, ResourceShelf, and Daily Kos. Try clicking around!

Much thanks to John Battelle for the inspiration for Findory Neighbors.

Update: Comments on Findory Neighbors by Gary Price, John Battelle, Nathan Weinberg, Rob Schluter, Rex Hammock, Niall Kennedy, and Carrick Mundell. Thanks, everyone, for the kind words and great feedback.

Personalizing print newspapers

Aaron Berman at Presstime writes about personalized news in print:

The difference in logistical complexity between producing a customized monthly and a similarly personalized newspaper certainly is enormous, says Douglas Karr, database marketing manager at The Indianapolis Star. But, he adds, the industry will have to embrace some level of personalization if it wants to meet consumer expectations.

"We're still trying to please everybody with one product," Karr says. "I think the day is coming when we'll have to please everybody with their own product."

Given that newspapers haven't even started doing personalization for their online sites -- where personalization is relatively inexpensive -- personalizing individual print newspapers seems unlikely to happen soon. But it's great to see people like Douglas Karr thinking ahead about how to best serve readers.

[via John Burke]

MSN Search officially launches

It's being widely reported that MSN switched to its new, home-grown web search engine. Congratulations to Brady, Erik, Mez, Chris, and the rest of the MSN Search team.

MSN promises their search will offer a "more personalized search experience" than the competition. Interesting that MSN sees personalization as their differentiator.

Chris Payne (VP at MSN Search, ex-Amazon.com) posts about the launch on the MSN Search weblog.