Google Scholar

Shirl Kennedy and Gary Price gush about the launch of Google Scholar, a version of Google focusing on scholarly sources.

Using it a bit, it is very cool. It seems similar to Citeseer. If you haven't tried Citeseer, you should. Among other great features, Citeseer links academic articles by citations and similarity. The new Google Scholar also has a feature to show all article citations. Nice.

You know, Citeseer was created by NEC Labs. One of the authors was Steve Lawrence. Steve Lawrence is now at Google. Hmm...

See also David Krane's post and the New York Times article on Google Scholar.

Update: John Battelle says Google Scholar was built by Anurag Acharya, a former UCSB professor and now a principal engineer at Google. Here's Anurag's old home page and a brief bio from a UW talk he gave back in February 2003.

Update: Danny Sullivan also has a good post on Google Scholar.

Update: Anurag Acharya announces Google Scholar on the Google blog.

Update: 1.5 years later, Microsoft launches Windows Live Academic Search. Apparently, another creator of Citeseer, Lee Giles, was involved with Microsoft's effort.

In my usage, I found myself agreeing with Dare that the breadth of Windows Live Academic is too weak to be competitive. I also found the lack of advanced search annoying.

Top online news websites

CyberJournalist posts the top online news websites for October 2004 (from Nielsen/NetRatings).

Top four are CNN, MSNBC, Yahoo News, and AOL News. Google News is small by comparison at about 1/3 the reach. Interesting to see several newspapers have large online audiences.

Et tu, TiVo?

TiVo will stop skipping past advertisers. As you would expect, reaction to this has been negative. Musing on Technology calls the move "customer-uncentric". BoingBoing says, "Time to build a MythTV."

This comes on top of restrictions on moving recordings made with TiVo and restrictions on recording of pay-per-view shows. Alternatives to TiVo, such as MythTV and Surestream's Beyond TV have no such limitations.

Yet another disappointing business move by TiVo.

See also my earlier post, "Will somebody please fix TiVo Suggestions?"

Update: Scott Johnson (VP of Engineering at Feedster) rails against TiVo and says there's a business opportunity here for someone who builds the PVR that people want.

Update: Interesting new twist on this. PVRblog cites a CBS study that says that people fast forwarding through commercials actually have a higher recall rate than people watching it normally. The counterintuitive result is probably because people fast forwarding are paying close attention to the screen.

RocketNews personalization

RocketNews issues a press release claiming "unique personalization features". Among other things, they say:

RocketNews tracks the articles visitors click to read, and these individual choices are used to determine the relevance and ranking of search results every time the individual returns to RocketNews to view their searches or their personal news portal.

But I wasn't able to see any changes to the RocketNews portal after reading several articles.

RocketNews does appear to keep search history, as do many others (My Yahoo Search, A9, MyJeeves, Findory).

Gary Price has more on the RocketNews announcement.

Yahoo ads in RSS

Yahoo's Overture takes the first steps towards advertising in RSS feeds.

Sigh. I really don't think advertising in RSS is a good idea. I'd rather see excerpts only in RSS feeds and advertising on websites when people clickthrough to read the full article.

But it seems that advertising in RSS is inevitable. I hope it's well targeted and unobtrusive, like Google AdWords, not untargeted and obnoxious, like popup advertising.

Update:Dave Winer comments on advertising in RSS, arguing that excerpt-only feeds are already advertisements (for the site where you have to go to read the full article) and that bloggers should "think carefully about how much advertising you think your readers can endure."

Exactly right. Heavy advertising may generate more short term revenue, but only at a long-term loss in revenue from driving away readers. It's a careful balance. The best advertising is unobnoxious, relevant, and useful.

RSS information overload

Chris Sherman talks about information overload with RSS

The information overload problem comes from sources that update frequently. Sometimes these sources provide must-read information. However, in many cases, new information may consist of a rambling post, a post on a topic of little interest to you, or worst of all (in my opinion) a post that simply points you to another feed that you're already tracking.

and recommends PubSub as the solution (even though Google News Alerts, Yahoo News, and Feedster all offer similar functionality).

The problem with all these solutions is you still have to go through all the effort of explicitly specifying what you want. You need to enter a bunch of keywords, refine them to narrow down to what you actually want, and then maintain them over time. All but the most dedicated information junkies won't bother.

What's really needed is an automated solution. Something that just figures out what you want. With no effort. Something like this.

Mooter CEO Liesl Capper

Red Herring interviews Mooter CEO Liesl Capper. This company certainly has a different view of search. From the interview:

Our entire focus has been on building our algorithms from the human side of the search equation, not the data side. I have spent the last decade studying cognitive styles, and how who you are as a person affects how you deal with information.

[We] use human sciences to predict what a person wants to see. Our information amplifier learns from current interest patterns, and amplifies implicit search interests.

Humanity is facing a subtle but pervasive shift in the way in which we perceive reality and interact with the world and other humans, and that this would be increasingly mediated by technology. In the future we will walk into a room, and an immersive world of information will flow around us, engaging all of our senses.

Well, okay, then.

But, I have to say, it's nice to see people approaching search from such a different perspective. After all, experimentation is the key to innovation.

MSN's long road ahead

US Bancorp analyst Safa Rashtchy says of MSN's new search engine:

This is not a move that will give MSN a big market-share gain; instead, it's a first step to stop the market share loss that MSN Search was experiencing. A new search engine was an absolute strategic necessity, but the hard work comes after the engine is perfected and fully launched to the entire population of MSN. This will include positioning not only the search site, but also MSN itself to customers, as a better web experience than Google.

[via Searchblog]

MSN desktop search leaked

An apparently unauthorized review with several screenshots of the prototype of MSN's desktop search. Supposedly, this will be released by December 2004.

[via msnsearch's weblog]

More on Google in Seattle

Todd Bishop at the Seattle PI writes about Google's new Kirkland office.

See also my earlier post "Google's Kirkland Office".

It's the content itself

Tom Curley (CEO of Associated Press) talked at the Online News Association Conference about online news:

New technologies allow news consumers to get the information they wish when they wish in the forms they wish. "Content will be more important than its container in this next phase," Curley said. "The franchise is not the newspaper; it's not the broadcast; it's not even the Web site. The franchise is the content itself."

He imagined a hypothetical "My Personalized News" of the future, which might include: the latest headlines and photos delivered, delivered to his computer by the AP; video news and ESPN highlights delivered to his set-top box; a list of upcoming earnings reports delivered by The Wall Street Journal to his PDA; and a BusinessWeek analysis delivered as a PDF to his printer. The challenge is therefore, he said, to first "get comfortable with this ice-cold shower of 'disintermediation'" and then for companies to begin "tagging our news for delivery in discrete pieces" while keeping control of their intellectual property and earning money to support their businesses.

"We believe that world needs AP's primary content more than ever," Curley said, "that authoritative voice that we -- and you -- provide, precisely because there are so many new voices and free-flowing content 'atoms' out there."

Content is king. News sources need to take advantage of new, decentralized distribution mechanisms such as RSS and news aggregators, exploiting it to reach a wider audience.

Update: The full text of Tom Curley's keynote is available. I particularly liked this part:

Discrete pieces of content -- stories, photos and video clips -- all categorized and branded, will be disassembled from whatever presentation you create and magically reassembled ... That's the fundamental behind personalization. The content comes to you; you don't have to come to the content.

The value of simplicity

In an InternetNews article, an interesting discussion of simplicity in search interfaces:

Google took the lead in the early days of search because it provided better results in a clean layout. MSN Search includes a bevy of buttons, drop-down menus and sliders to help searchers refine results. But it's not clear to experts that many searchers even care about advanced query techniques, no matter how simplified.

"Users are notoriously very lazy," said usability expert Jakob Nielsen, a principal of Nielsen/Norman Group. "They don't want to go to page two; they don't even want to scroll. The average behavior is to type two or three terms [into the query box], look at what's visible and click on those links."

Nielsen said that Google rose to the top at a time when other search services had piled on content and features in the rush to become portals. "They were very cluttered. Google had the opposite approach -- very lean and cut back and very good at prioritizing. Those are the two reasons for its big success."

Nielsen compared Google to A9, the search service from Amazon.com, which combines Google's search technology with proprietary features, such as personalization, Search Inside the Book and results from different sources appearing in multiple columns."A9 has a variety of extra features, so you would think it must be better than Google. But it's worse."

See also my earlier post "Google wants to keep it simple".

MSN Search's beta blunder?

A brutal article at BusinessWeek:

Nov. 11 was supposed to be a big day for the folks running MSN Search. The new service, albeit a test version, was slated to launch and begin Microsoft's big push into the lucrative and competitive world of Internet search technology. But the day didn't start off too well. The site went down almost as soon as it went up.

It would be one thing if a startup's beta had sputtered. Not too many folks would have noticed. But this is Microsoft, earth's largest software company. And it trumpeted this test launch with a public-relations campaign to ensure that users around the world knew the service was ready for widespread use. So when MSN Search went down, a bit of Microsoft's credibility in the search-engine business went with it.

Ouchie.

Silliness with page rank

Scoble asks Google, "Who has the best geek blog?" and notes that he gets top honors. But, oh mighty Scoble, what happens if you ask who has the "best geeking blog"?

Yes, we have a lot to be proud of. We're both thoroughly geeky.

Of course, the top Google result for "personalized news" is exactly what you would expect. Google knows a good thing when it sees it.

Meeting Topix.net

I just had the pleasure of chatting with Rich Skrenta and meeting the rest of the Topix.net team.

Although Findory.com is not listed on their partners page, we've had a partnership with Topix.net for some time. Topix.net is focused on news classification and local news. Findory is focused on personalization of information. But the two companies have a lot in common.

Google Firefox start page

Sushubh Mittal discusses the new default start page for Firefox, which is hosted by Google, and the benefits for both the Mozilla Foundation and Google.

Of course, the default start page for Internet Explorer is MSN. The major competitor to Internet Explorer is Firefox/Mozilla. And the major competitors to MSN are Google, Yahoo, and AOL.

All very interesting.

Online shopping up substantially

2004 online holiday shopping sales will be up 20-30% over 2003, according to SearchEngineJournal's summary of several analyst reports.

MSN Search and the competition

Many are saying that MSN Search isn't better than Google at this point. But does it have to be? Charlene Li at Forrester explains:

[MSN Search] needs to be "good enough" for MSN to serve its current users, and hopefully, to entice wayward MSN.com and Hotmail users who search with other engines to come back to the roost.

Also, now that MSN has its own search algorithm, it will be able to develop search features like personalization that meets its users' needs. Note that I mean MSN's users, not necessarily the loyal users of Google or Yahoo! Search is fragmented enough that I think each player will carve out their own audience.

There can be many players in this market and, with Microsoft's market power, it's guaranteed to be a big one.

MSN Search launch

MSN Search launched, albeit in beta form. As you would expect, Chris Sherman and John Battelle have good writeups. The MSN press release is also available.

Other than implicit geolocation for local search, no personalization yet. But the MSN Search about page still promises it is coming:

A Personalized Experience. Your Search service should learn from you. What you like, what you read, where you live. Search should deliver results that are more personal and relevant to you.

Does size matter?

In his announcement about Google's index size, Bill Coughran makes a good counterargument against my claim that index size doesn't matter.

His basic point is that there are large classes of searches that return few or no useful results. The bar is pretty low in these cases. If you only return three results for a query, relevance rank isn't going to matter much. If you increase the number of results to five, that's probably helpful if the results you added are at all relevant. So, it's true. If you can increase the number of results in these cases without reducing relevance on other queries, you're helping people find what they need.

Nevertheless, is expanding a general crawl really the right approach? When you already crawl 4B pages, any additional pages you crawl will be deep in the crufty back alleys of the web. These kinds of documents can not only be useless, but can hurt overall search quality if they get surfaced inappropriately. Perhaps it would be more useful to do directed crawls of high quality or specialized data sources? Target specific holes in your coverage?

And there's other ways to be helpful. For example, perhaps the query is just the wrong query. Maybe the searcher needs help with query refinement (replacing search terms) or query expansion (broadening a search with synonymous or related terms).

Again, it all about relevance. If you can improve relevance in a minority of cases by expanding the general crawl without hurting the common cases, you'll improve the overall usefulness of the search engine. But increasing index size isn't the only way to improve relevance.

Update: Interesting post from Danny Sullivan on index size of and depth of the page indexed by the various search engines.