Adam Bosworth on simple web services

Daniel Steinberg reports on Adam Bosworth's talk at the MySQL Users Conference. Adam criticized past efforts on web services as being too complicated and advocated simpler techniques:

Imagine if you can query any data that is available anywhere in the world ... What this requires is a single, simple, open wire format for items. The format needs to be simple for any P programmer to deliver and any JavaScript programmer to consume ... "Complex things tend to break and simple things tend to work."

RSS 2.0 and Atom will be the lingua franca that will be used to consume all data from everywhere. These are simple formats that are sloppily extensible. Anyone who wants to can use these formats to consume content or to author content.

This is our approach at Findory. We designed our RSS feeds as simple, clean, and easy-to-use web services. Programmers can use Findory to get news and weblog content for categories and keyword searches and related content for sources. It's easy and it all just works. Just like the rest of Findory.

[via Dare Obasanjo]

Yahoo My Web and searching web history

Yahoo launched My Web, an extension to the Yahoo Toolbar that lets you save on Yahoo's servers a copy of any web page you've seen. The idea is that it lets you find web pages you saw once before quickly and easily.

This isn't quite like Seruku, which saves a copy of every web page you've seen on your own computer. Or quite like Google Desktop Search, which searches over every web page you've seen recently (using the browser cache on your computer). Or quite like Filangy, which (sort of) stores every page you've seen on their servers.

Because you have to explicitly push a button to save the page, I'm not sure how many people will use Yahoo My Web. Seems likely to me that, when I'm browsing a particular page, I don't really know if I'll want to find it again. It'll be too late when I decide I really should have pushed that little button.

Cute idea though. Fun to see all the innovation lately.

See also Chris Sherman's review of Yahoo My Web.

Update: Tony Gentile posts a detailed review. He also points out that Yahoo My Web probably should be considered more of a bookmarking tool than a step toward Memex.

Press release on Findory growth

Findory just issued a press release on our exponential growth. An excerpt:

Since the site launched in early 2004, Findory's traffic has more than doubled every three months.

With over one million page views a month, tens of thousands of people rely on Findory's personalized front page to keep up with everything from local politics to world cup rugby.

Over one million articles have been read through Findory.com. Findory's traffic has grown nearly thirty times in the last twelve months.

See also my earlier post, "Findory is growing and growing!"

Improving article pages

Steve Outing at Editor & Publisher writes about improving online news article pages:

More and more people bypass news Web sites' home and section pages .... The best approach is to create an article-page template that serves as a sort of secondary home page ... Give them enough choices to guide them to other important content elsewhere on the site.

According to its Web site's editor, Angus Frame, 41% of globeandmail.com visits now begin on non-hub pages ... "Readers who went straight to a story page had no idea how much was available on globeandmail.com," [Frame said]. "They only saw one story and then had no reason to stick around."

Globeandmail.com's managers decided last year to "improve the story-page experience." [Frame said], "We added valuable, informative links to the right-hand side of the story in a fairly wide column. We turned every story into a mini hub .... Literally overnight daily page views increased by more than 25%, from about 2.3 million page views a day to 3.0 million page views a day."

See also my earlier post, "Targeting and online newspapers".

[via Simon Waldman]

Findory redesign

Findory launched a redesign of our site today. We affectionately have been calling it the "fuel" release in honor of the caffeine-charged coding frenzies that took place at Fuel Coffee here in Seattle.

Alex did tremendous work combining suggestions from our readers with new features such as popular sources, creating a clean and attractive new look. Our beta testers have been gushing over it. We hope you like it too!

Trying to Wiki the news

Joanna Glasner at Wired writes about some problems at Wikinews:

Operators of Wikinews are finding their mission rife with frustrations and challenges.

The site, an offshoot of Wikipedia, the volunteer-maintained online encyclopedia, is facing pressures its parent organization rarely had to contend with, such as ferreting out fake posts, incorporating original sources and updating coverage to reflect rapidly changing current events.

"In Wikipedia, the writing style of an encyclopedia is more timeless. You can get it right eventually. It's going to be the same article for many years," said Jimmy Wales, Wikipedia's founder. "With a news story, the actual story has a limited lifespan. If it's not neutral, you've got to fix it quickly."

The Wikinews site follows essentially the same set of rules as the Wikipedia encyclopedia, which allows anyone to create entries or edit and correct other people's work.

Slashdot, another community news site with millions of users, has similar problems with timeliness and quality of their news.

See also my earlier post, "Slash(dot) and Burn".

Text statistics on Amazon.com

Nathan Torkington at the new O'Reilly Radar blog points to a new feature on Amazon that can show you the most frequently used words from and various statistics on many books.

For example, here's the concordance and text statistics for Applied Cryptography.

This is cute, but not very useful. I'm not sure I care that Applied Cryptography averages 1.7 syllables per word or that the most common word in the book is "key".

The "books on related topics" on the same page might be more useful. Amazon says the relationships are determined using their new SIPs data. It would be fun to experiment with using this text analysis data to try to find improvements to the accuracy of Amazon's personalization and recommendations.

Filangy and searching your web history

Gary Price posts on Filangy, a toolbar that keeps track of every web page you've visited and lets you search over your web browsing history.

One curious feature of Filangy is that your history is stored on their servers, not on your PC. The advantage of this is that your history can be combined and accessed from multiple computers. The disadvantage is that the snapshot of the page Filangy indexes may not be the page you actually viewed.

For example, if you go to Amazon's home page, you'll see a personalized page with content picked just for you. When Filangy retrieves the Amazon page from their servers and stores it, it appears that the page they retrieve will be the generic page, not the page you just viewed.

It's a little strange to have a search of your browsing history that searches pages you never actually saw.

About a year ago, I posted about a similar toolbar called Seruku and the Microsoft Research "Stuff I've Seen" project, both of which try to make it easy to find anything you've found before.

Since then, Google launched its desktop search. Unlike the desktop search tools from MSN and Yahoo, Google Desktop Search indexes your web history (at least the cached last few days of it), a very useful feature.

Search over your browsing history brings us closer toward Memex, the memory extender. In the near feature, you will be able to easily recall anything you've seen on your computer. Filangy, Seruku, MSR's "Stuff I've Seen" and Google Desktop Search are bringing us closer to that future.

Google launches search history

Google just launched My Search History. It keeps track of your previous searches and search result clickthroughs, making it easy to find things again that you found in the past.

A9, My Yahoo Search, My Ask Jeeves, and Findory have had this feature for a while. All but Findory require users to sign in to activate the feature. Like Findory and A9 but unlike Yahoo and Ask, Google's search history feature is integrated into the main search on the site.

Keeping search and clickthrough history is a first step toward personalized search. The next big step is to use this data to reorder search results, making the results more relevant to your particular interests and needs.

Personalized search is the future. In his article on Google's new search history feature, Chris Sherman says:

Don't expect Yahoo, Ask Jeeves, MSN or AOL Search to stand still. Personalized search has long been touted as one of the holy grails for the industry ... Beginning today with Google's launch of My Search History, I expect to see major leaps ahead in the arena of personalized search -- and that's a good thing.

None of the search giants personalize search results yet. But, little guys like good old Findory have taken some early first steps, changing web search results in a limited way based on search and clickthrough history. As always, the best way to see the future is to look at what innovative little startups are doing.

[See also Stefanie Olsen's article at CNet]

Update: Charlene Li says personalized search results are the "Holy Grail of search" and quotes Google Director Marissa Mayer for an example of how Google might personalize search results.

Update: Danny Sullivan posts an interesting comparison chart of search history features from A9, Ask, Eurekster, Findory, Furl, Google, and Yahoo.

Fortune on the search war

Fred Vogelstein at Fortune wrote a great article on MSN and the search war with Yahoo and Google. The full article is subscription only, but here are some selected excerpts.

Microsoft has been having difficulties in the search war:

Every month it seems as if Google hires away one of Microsoft's top developers ... As of March, roughly 100 Microsofties had left for its search nemesis ... The Google migration has gotten so bad, says a former Microsoft employee, that when he told his bosses and colleagues he was leaving earlier this year, "the first question out of their mouths was 'You're not going to Google, are you?'"

Trying to build a Google killer ... has turned out to be truly humbling for Microsoft. The effort has taken longer, cost more money, and exposed more big-company problems at Microsoft than anyone imagined.

Google is fleet of foot, but it cannot afford to stumble:

One reason Google has been rolling out so many new or improved products is that [Google CEO Eric] Schmidt understands that innovation is the only sure edge Google has. The moment Google allows itself to slow, Microsoft could overwhelm it.

What will see in the future from the search giants?

"We need to take search way beyond how people think of it today and just have it be naturally available, based on the task they want to do," [said Bill Gates]. For example, if you wanted to look up a factoid while you were writing a document, you might search for it without ever leaving Word.

All three big search engines are scrambling to find ways to make search more personalized. The thinking is that the more a search engine knows about who is searching, the more accurate the results will be.

[Full disclosure: Fred talked with me when he was doing research for this article.]

Google Maps launches in UK

Just when I get back from London, Google launches Google Maps and Google Local in the UK.

Looks cleaner than the alternative, Streetmap.co.uk. It's great to see more options for navigating the confusing twists and turns of London. I wish I had had it last week!

Yahoo News beta

Yahoo News has a beta test of a new look. They made it more powerful and customizable with a focus on reading content from specific sources (e.g. BBC, New York Times).

There are remarkable similarities with some of Findory's features. Findory Favorites allows readers to list their favorite sources; Yahoo News beta has a similar feature called "My Sources".

The source pages are also similar. The new Yahoo News source page for Weekly Standard is here. Findory's Weekly Standard page is here. Findory's is more complete with a greater focus on discovery, but the resemblance is apparent.

Unofficially and off the record, several people at Yahoo have expressed admiration for Findory. I'm not surprised to see these moves by the search giant. I hope that Yahoo will go a step further and target personalized news.

Frankly, it's amazing that our tiny little Seattle startup is at the forefront of news personalization. It's silly to see others so far behind in personalization technology. We would be pleased to see the giants join us to help readers find and discover the news they need.

Personalized news is the future. The sooner we all can make that future a reality, the better for all our readers.

[see also Gary Price]

More My Jeeves

Gary Price has the details on some new features in My Jeeves.

Become.com launches

Become.com, a product information search engine, just launched.

With my first few searches (e.g. "6.1 sound card"), I was underwhelmed, but your mileage may vary.

It's certainly a good idea though. Despite attempts from Amazon.com, ePinions, Consumer Reports, and others, the Web has no authoritative, objective, comprehensive source of product information. Whoever builds that would be providing a useful and valuable service.

Chris Sherman has a detailed review.

Questioning tags

Ubergeek Tim Bray asks:

Are tags useful? Are there any questions you want to ask, or jobs you want to do, where tags are part of the solution, and clearly work better than old-fashioned search?

I really want to believe that tagging is big, a game-changer, but the longer I go on asking this question and not getting an answer, the more nervous I get.

Tags, for those of you not paying attention to all of this, are a social software feature where random people mark various types of content with keywords. Del.icio.us, the popular social bookmarking tool, is one of the early experiments with tagging. More details at Wikipedia.

Update: Danny Sullivan and Gary Price also question tags.

Update: Stephen Green (Principal Investigator of Advanced Search Technologies at Sun) has an interesting discussion of tagging. At one point, he says, "[Tagging is] not really a new way of indexing documents, it's actually an old way that didn't work very well," but he balances that by later saying, "I'm wondering whether we couldn't get some of the benefits of a controlled vocabulary."

Update: John Dvorak at PC Magazine savages tagging, saying that "nobody outside the groupthink community really cares about any of this" and that "the 'folksonomy' notion ... is doomed to failure" because it will succumb to "vandalism and spam." [via Loose Wire]

Update: A couple months later, Tim Bray says, "It's looking like the answers are: Yes, tagging is useful; No, it's not a replacement for full-text search, even partially."

Del.icio.us investors

Del.icio.us apparently has taken a minority investment from a diverse group that includes Union Square Ventures, Amazon.com, Marc Andreessen, BV Capital, Esther Dyson, Seth Goldstein, Josh Koppelman, Howard Morgan, Tim O'Reilly, and Bob Young.

With so many big investors, one has to wonder what the valuation could be for this to be still a minority investment.

See also my earlier post, "Profile of Del.icio.us"

[via Loose Wire]

Update: I should have also cited Del.icio.us Founder Joshua Schachter's post on the delicious-discuss mailing list.

Update: Fred Wilson, a partner at Union Square Ventures, posts his thoughts on their investment.

Google Maps + Craigslist

Paul Rademacher combines Craigslist and Google Maps to create a clever and compelling way to find houses for sale or rent. Well worth a look.

[via Gary Stein and Om Malik]

The future is "my media"

Jeff Weiner (SVP of Search at Yahoo) spoke at the Warton Technology Conference and mentioned personalized information:

The future ... won't belong to either mass or micro players, but rather to consumers who will increasingly tailor their information gathering to their needs and tastes. "The future is going to be 'my media'."

It's an excellent vision, but I think Jeff is overestimating his customers' patience. Users are notoriously lazy. If you ask them to do work, most of them won't do it. From their point of view, you're only of value to them if you save them time.

Customers do want information tailored to their needs and tastes. But, if any work is going to be done, it's going to have to be done by a computer, not a person. People expect you to just make the right thing happen.

[via Search Engine Watch]

Google and question answering

Jonathan Betz at Google announces Google Q&A which returns instant answers to some queries (e.g. "What is the capital of Washington state?").

Ask Jeeves and MSN Search have had this for a while, but one thing about Google's effort is particularly noteworthy. They appear to be extracting the answers from their web crawl. From an InfoWorld article:

Google feeds this service with information from Web sites it considers reliable, but it hasn't established formal relationships with any content provider whose information is being used for this feature, [Google Director Peter] Norvig said.

Google doesn't expect that the owners of the Web sites will complain over the possibility that this new service will steal traffic away from them, Norvig said. On the contrary, being featured at the top of Google's results list will give these Web sites great exposure, and will likely result in increased traffic.

This is a first step down a long road toward understanding the vastness of knowledge stored in the Web. At the end of this path lies a futuristic Oracle of search engines that processes and understands all of human knowledge and responds with a concise and accurate answer to any query.

It will be a long time before we get there -- I'm not sure we will in our lifetimes -- but it's exciting to see progress toward that goal.

See also coverage from Gary Price and Nathan Weinberg.

Feed readers from newspapers

John Gartner at Wired writes about the trend of newspapers offering their own branded feed readers.

Offering their own feed readers feels like a desperate move to me. It's grasping at control of the distribution channel. Given that there's literally hundreds of RSS readers, including offerings from giants like Yahoo, MSN, and Ask Jeeves, I doubt also-ran feed readers from individual newspapers will gain noticeable market share.

See also my earlier post on how newspapers would benefit from embracing new distribution channels.

Blogger is slogging

Adam Penenberg at Wired reports on widespread complaints about the performance and reliability of Google's Blogger service.

Scott Johnson (VP at Feedster) claims that the problem is weblog spam and that Google should focus their effort on eliminating all the bogus weblogs.

Yahoo supports Wikipedia

Yahoo announces that they will "dedicate hardware and resources to support Wikipedia" and use Wikipedia content in Yahoo Search.

Google made a similar offer a couple months ago. Google uses Wikipedia content indirectly through their Answers.com definitions.

[via Yahoo! Search Blog]

Update: Tara Calishain points out that the higher visibility of Wikipedia will increase the incentive for spammers to start mucking up the Wikipedia content.

OJR on Findory

Mark Glaser at OJR reviews a few weblog aggregators including Findory. An excerpt:

[Findory] comes in as perhaps the most high-concept entry. Findory attempts to take the Google News format two steps further -- by offering an interface with just news headlines and blurbs, and another with just blogs. But Findory's most interesting feature is that the weight and placement of stories depends on what you click through to read on repeat visits .... A fascinating experiment.

[via JD Lasica]

Attention XML and finding relevance

Dare Obasanjo (PM at MSN) makes some great points ([1] [2]) about Attention XML and managing information overload. First, on what we'll see in the next generation of feed readers:

The attention problem is the problem that ... eventually readers get overwhelmed by the flood of information hitting their aggregator's inbox. Some have used the analogy "drinking from a firehose" to describe this phenomenon.

Ideally a user should be able to tell a client, "Here are the sites I'm interested in, here are the topics I'm interested in, and now only show me stuff I'd find interesting or important". This is the next frontier of features for RSS/ATOM aggregators.

Current feed readers are too cumbersome for the mainstream. What we need is a relevance rank for weblog posts that helps readers find the news they need.

Dare then turns to Attention XML and whether it helps solve this problem:

After talking to Steve Gillmor I realize another reason I didn't like the attention.xml spec; it ignores all the hard problems and assumes they've been solved. Figuring out what data or what algorithms are useful for determining what items are relevant to a user is hard. Using said data to suggest new items to the user is hard. Coming up with an XML format for describing an arbitrary set of data that could be collected by an RSS aggregator is easy ....

If anyone figures out how to do this right, it is unlikely that it will be made available as an open pool of data. The 'attention.xml' for each user would be demographic data that would be worth its weight in gold to advertisers. If Bloglines could figure out my likes and dislikes right down to what blog posts I'd want to read, I find it hard to imagine why the Bloglines team would make that information available to anyone including the user. For comparison, it's not like Amazon makes my 'attention.xml' for books and CDs available to myself or their competitors.

Unfortunate. I've looked at Attention XML several times myself. Steve Gillmor and I seem to share the same goal, managing information overload. But I agree with Dare that it's unclear how to get this data sharing format adopted or if we even need to get it adopted to move toward our goal.

FeedBurner and RSS advertising

FeedBurner announces a $7M round of funding. Very nice.

When the company first started, seemed to me that they were just a small, free RSS caching service, not that different than the tens of other caching solutions out there.

FeedBurner is still a feed caching service, but they're trying to break through into advertising in RSS feeds and premier (subscriber only) services like more sophisticated tracking and metrics on RSS feeds.

As I've said before, I think there's some serious issues with advertising in RSS feeds. If you're only publishing short excerpts in your feeds, the feeds themselves are ads, advertisements of the content on your website. Adding more ads to the ads will be annoying and will drive people away.

If you're one of the sites that do publish full text feeds, you might never expect people to come to your website to view your content. In that case, advertising is much more reasonable. The key here would be to make the advertising as targeted and relevant as possible.

It'll be interesting to watch FeedBurner. They're right at the center of the growth of RSS.

[via Brad Feld and Matt Marshall]

Update: Two years later, Google acquires FeedBurner for $100M. It appears Google was most attracted to FeedBurner's 430k subscribers (in May 2007) and Feedburner's tracking and metrics services.

While the companies seem like a good match to me, at $230 per FeedBurner subscriber, I might have balked at the price. If Google had hacked up a FeedBurner knock-off and handed out a $100 bill to any FeedBurner subscriber who switched, I suspect Google would have done pretty well.

But, hey, after spending $1.6B on YouTube and $3.1B on DoubleClick (not to mention Microsoft's $6B for aQuantive), I guess $100M is pocket change.

Congrats to the FeedBurner team! You did build a great product and deserve this outcome.

Google Maps + Keyhole

Many are talking about how Google Maps integrated Keyhole (which they acquired about four months ago) to offer nifty satellite images. For example, here's a top-down view of Amazon HQ in Seattle.

Just like others have offered maps before, others have offered satellite and aerial photos. But Google has combined this with a snazzy interface that lets you do things like click and drag the viewpoint around while the images refreshes dynamically.

Chris Sherman at Search Engine Watch has a particularly thorough review of the new service.

Desktop search reviews

Adam Baratz at Ars Technica posts a very detailed review of all the desktop search tools.

[via Findory -> Marc Orchant]

Findory is growing and growing!

This graph shows Findory's traffic growth for the last five quarters. Growing fast!

The graph is hits per quarter on the Findory.com website. The Y-axis starts at zero. Our current traffic is over two million hits per month.

Great to see so many people enjoying personalized news!

Google and competition with eBay

Steven Levy at Newsweek asked Eric Schmidt (CEO, Google) about competing with eBay:

[Eric] just laughed. "It's a perfectly reasonable question, but it doesn't compute here," he says. "If I said at a meeting, 'Are we going to enter eBay's space?' everyone would look at me and say, 'Why? They do a fine job.' The genius of Google is that we find new ways to solve problems that were never solved before."

But no one is expecting Google to launch an also-ran auctions site. Google don't play that game.

What's more likely is that Google's innovation in other areas will impact eBay. For example, some merchants are responding to increased fees on eBay by switching to Google AdWords and marketing and selling their goods directly to customers.

Google is quite aware of this. Eric Schmidt even said that Google is going after small merchants, trying to help them find customers for their products. And Google is doing it with more than just AdWords. Google's Froogle also makes it easier for people to find small sellers.

No one expects Google to launch an eBay look-alike. But it is likely that these two giants will find themselves increasingly pursuing the same customers. The more merchants can be discovered by customers using Google, the less those merchants need to use eBay.

[Newsweek article via John Battelle]

Update: Two years later, a BusinessWeek article reports the eBay "magic is gone ... Shoppers are simply not buying all the inventory anymore. Some items languish without a single bidder. Many shoppers opt for other sites including Amazon.com, use sophisticated search engines such as Google and Yahoo!, or head to store sites directly."

Trends in online news

Mark Glaser at OJR writes about Yahoo News and trends in online news. Some excerpts:

One way to build on the time spent on the site is to make the story pages more rich with links to relevant content on other pages

"People don't start from the front page, they often start at story pages," [Former WSJ Online editor Neil] Budde said. "So if this is the first point of entry into our site, what can we do to expose people there to more of what we have available? I think it's important for any news site."

[Yahoo Director of Personalization] Scott Gatz ... [argues] the key to success for winning over the masses was not bothering to even call them "RSS feeds".

"If you look at how we've integrated RSS into Yahoo News, we're not actually using those three letters very much," Gatz said. "So you look at it, and it says what would you like to add to your political news? Here are some political blogs. Would you like to add CNN or MSNBC onto your news page? The fact that it happens in XML or RSS isn't the important thing. Most of the users don't want to have to figure that out."

See also "Targeting and online newspapers", "XML is for geeks", and "Getting your grandmother to use RSS".

[via John Battelle]

Google Gulp FAQ

Google Gulp is one of the funnier April Fools jokes today. Whatever you do, don't miss the last question in the FAQ:

When will you take Google Gulp out of beta?

Man, if you pressure us, you just drive us away. We'll commit when we're ready, okay? Besides, what's so great about taking things out of beta? It ruins all the romance, the challenge, the possibilities, the right to explore. Carpe diem, ya know? Maybe we're jaded, but we've seen all these other companies leap headlong into 1.0, thinking their product is exactly what they've been dreaming of all their lives, that everything is perfect and hunky-dory -- and the next thing you know some vanilla copycat release from Redmond is kicking their butt, the Board is holding emergency meetings and the CEO is on CNBC blathering sweatily about "a new direction" and "getting back to basics." No thanks, man. We like our freedom.

Sweet. You gotta love these guys. Tellin' it like it is.