Mary Meeker at Web 2.0

Among many other things in her rapid-fire Web 2.0 talk, Mary Meeker emphasized the growing importance of personalization technology.

Mary said that "the best example of personalization is Amazon.com's recommendation engine" and that she expected widespread use of personalization and social filtering in the future.

Also of note was a slide that showed the top e-commerce retailers, most of which are also large offline retailers (like OfficeMax and Staples). Mary pointed out that few had predicted these retailers would appear so high in the rankings. She said traditional media companies should take notice, stop despairing over the threat of online, and focus their efforts on dominating online media.

Update: The slides (PDF) from Mary's talk are now available. [via Paul Kedrosky]

Riya, vaporware, and hard problems

Liz Gannes at GigaOm reports that Riya, a company that promised to revolutionize photo search, has "dramatically changed course". It is now going to be a visual search engine for products.

Apparently, their facial recognition technology never worked very well, so they are shifting to a problem where "the threshold of success is lower."

Riya got a lot of hype about their vaporware facial recognition technology about a year ago. For example, at the time, Michael Arrington said:

Riya leverages potent facial and text recognition technology with an intelligent interface to help people make sense of the thousands of untitled and untagged photos that are building up on their hard drives.

Riya is going to be successful. They have real technology.

It turns out, all Riya had was strong claims about solving hard problems they could not actually solve.

Of course, it is not surprising that Riya couldn't solve the facial recognition problem. It's a very hard problem. What is surprising is that they got such attention for vaporware claims about solving this very hard problem.

I wonder if we are seeing the same thing with the flood of hype and claims about solving natural language search. Powerset, for one example of many, seems to be getting amazing hype for a company with no launched product in a space littered with previous failures.

See also Don Dodge's post, "Riya tries again as Like.com".

See also Danny Sullivan's older post, "Hello Natural Language Search, My Old Over-Hyped Search Friend".

Eric Schmidt at Web 2.0

Eric Schmidt spoke with John Battelle on stage here at the Web 2.0 conference a few hours ago. There were some interesting new tidbits in their interview.

Eric broadly laid out two fundamental shifts that he sees in our near future: online video and software as a service. Online video is a shift toward watching videos on the Web with a focus on discovering related videos and sharing videos with friends.

Software as a service is a twist on the old idea of software moving off the disk and on to the network (a remote cluster of machines). The twist is a focus on reliability from using a large, shared cluster and on sharing and collaborating.

The prediction of an ascendance of online video might justify Google's YouTube acquisition. And, on that point, in response to a question, Eric made the incredible claim that YouTube is not violating copyright; whether objectively true or not, the fact that Google thinks it is true may also explain their willingness to pay an exorbitant $1.65B for YouTube.

Finally, a quick tidbit on hiring. In response to a question about attracting and retaining the best and brightest, Eric said, "People don't work for money. They work for impact." Very true. Money is not always the best motivator.

Update: Other good coverage from ValleyWag and PaidContent. [via Danny Sullivan]

Update: Video of this talk appears to be available.

Google and those TPS reports

Googler Chris Sacca has a fun post full of references to the movie "Office Space".

I remember this stage at Amazon.com. As the company grew, the MBAs flooded in, and the culture started to change. As Chris says:

The leaders of Google have realized, from the earliest days, that the company's own growth would be the biggest challenge.

Nevertheless, the potential big company pitfalls are always looming. As the size grows, I see colleagues, particularly those who join Google from other companies, tempted to carve out fiefdoms and mandate SWOT analyses and extensive Excel spreadsheets littered with three letter acronyms. I have seen a few mid-level bosses evoke the traditions of Japanese management and schedule "pre-meetings" to plan, discuss, and approve what will be planned, discussed and approved at the actual meeting itself. MBA-speak creeps into the parlance and these new managers require the filing of more and more TPS reports.

The good news? Google's culture of letting engineers and product folks build great stuff that users want is still winning the day.

See also my April 2004 post, "Kill Google, Vol. 1", where I said:

The first threat to Google is internal. Google needs to maintain a culture that produces and delivers innovative new products. So far, Google has done this by hiring some of the brightest and most creative researchers in the world.

But, as Google grows, having incredible people isn't enough. Communication becomes difficult in a large organization. Accountability drops, free riding increases. Great prototypes are developed, but never get out the door. People don't know who to contact and how to get things done.

Google is well known for having nearly no management -- the controlled chaos of a research lab -- but, unless Google can adjust its organizational structure to its new size, the firm may find its innovation crushed under its own growth.

See also my May 2006 post, "First, kill all the managers", where I talk about Google's management strategy and Googler Steve Yegge's thoughts on middle management.

Doubling down at Amazon.com

The cover story in next week's issue of BusinessWeek, "Jeff Bezos' Risky Bet", looks at Amazon.com's move into web services.

Some extended excerpts:

Amazon founder and Chief Executive Jeffrey P. Bezos ... is back with yet another new idea. Many people continue to wonder if the world's largest online store will ever fulfill its original promise to revolutionize retailing.

But now Bezos is plotting another new direction for his 12-year-old company ... Judging from an advance look he gave BusinessWeek ... it's so far from Amazon's retail core that you may well wonder if he has finally slipped off the deep end.

Bezos wants Amazon to run your business, at least the messy technical and logistical parts of it, using those same technologies and operations that power his $10 billion online store. In the process, Bezos aims to transform Amazon into a kind of 21st century digital utility.

Amazon is starting to rent out just about everything it uses to run its own business, from rack space in its 10 million square feet of warehouses worldwide to spare computing capacity on its thousands of servers, data storage on its disk drives, and even some of the millions of lines of software code it has written to coordinate all that.

Bezos spent hundreds of millions of dollars to build distribution centers and computer systems in the promise that they eventually would pay off with outsize returns ... Lately profits have fallen, dragged down by spending on new technology projects.

[This] has investors restless and many analysts throwing up their hands wondering if Bezos is merely flailing around for an alternative to his retail operation.

What's more, at the same time Bezos is thinking big thoughts, Amazon's retail business faces new threats ... Other sites are fast becoming preferred first stops on the Web. Google, for one, has replaced retail sites such as Amazon as the place where many people start their shopping.

Amazon's mission to be the place where "customers can find and discover anything they might want to buy online" doesn't especially mesh with the goal to be the prime source of services needed to run an Internet Age business ... Can Bezos manage a company that simultaneously sells the most routine stuff to consumers and the most demanding business services to entrepreneurs and corporations?

"We are willing to go down a bunch of dark passageways," [Jeff] says, "and occasionally we find something that really works." As always, investing in Bezos and his company will require faith that there's light at the end of his newest tunnel.

See also my previous posts on some of Amazon.com's web services, "Amazon launches utility computing service", "Amazon Mechanical Turk?", and "Remote storage on Amazon S3".

See also the "Get Big Fast" post from my Early Amazon series.

[BusinessWeek article via Om Malik]

Update: BusinessWeek also has an interview with Jeff Bezos about the business of offering web services. Jeff says:

We're trying to leverage an existing asset ... We cannot operate our consumer business without these pieces of Web-scale infrastructure.

We have three businesses today ... consumer-facing ... seller-facing ... developer-facing. The first two businesses are already financially meaningful businesses for Amazon.com, and we believe the third one can be too.

[Found via Matt Marshall]

Update: See also the comments in a thread on an earlier post for my thoughts on Amazon's business and their web services. An excerpt:

It may be a failure of imagination on my part, I have a hard time seeing Amazon web services as a profitable venture. I understand the argument that Amazon needs to build a lot of this anyway for internal use, but I think there is a big difference between supporting and servicing internal customers and external customers. I suspect Amazon will find it very expensive to do everything that enterprise-level paying customers will expect of them.

In addition, I find myself confused trying to determine the target market for these web services. Most web services nowadays are free and targeted at hobbyists. Amazon is hoping to extract hundreds of millions from their web services, which means they would need thousands of enterprise level clients. Does this market exist? Other utility computing offerings do not seem to have tapped it, perhaps because substitutes (e.g. leasing servers) work as well or better for many purposes.

Update: Tim O'Reilly, in an interview with Jeff Bezos at Web 2.0, beautifully summarizes the Wall Street reaction to all of this: "But, why?"

A9 goes after advertising

Amazon's A9 appears to be refocusing on web advertising instead of web search. First, they simplified and scaled down their web search effort. Now, they are testing a new web advertising platform called Clickriver.

Clickriver appears to be AdWords for Amazon -- web ads placed on Amazon.com -- and a first step toward an AdSense competitor. From the Clickriver About page:

Clickriver Ads is an advertising program ... that allows businesses to place sponsored links on Amazon.com.

Clickriver Ads will appear on Amazon.com, on search results pages and on many product detail pages.

With Clickriver, a wide range of complementary products and services can be advertised at the precise moment that someone is interested -- as they shop, browse, and search on Amazon.com. For example, ads from banks can display on pages for finance and investment books, such as on the "Road to Wealth" product page. Ads from photo printing services can display next to search results for cameras and tripods. Ads for hotels, car rentals and travel agencies can display on pages for travel books, sunglasses, suitcases, portable DVD players and other travel accessories sold on Amazon.com.

Clickriver Ads was built by A9.com, a search technologies company based in Palo Alto, California. A9.com is a wholly owned subsidiary of Amazon.com.

Back in April, in a post called, "What will become of A9?", I said:

A9 could switch to focusing on AdWords and AdSense-like advertising, building on the success of Amazon Associates.

A9 would compete with Google not on web search, where Google has a strong advantage, but on advertising, where Amazon has its massive catalog and experience selling products to leverage.

Looks like that is exactly what A9 is doing.

[Found via Gary Price, John Battelle, and John Cook]

Update: Om Malik says, "Amazon doesn't really want to be a retailer some days, does it?"

RSS readers are dead, long live RSS readers

Richard MacManus notes the shutdown of Pluck's feed reader and says:

Microsoft is integrating RSS into Outlook next year, Google will probably have Gmail integration soon, and Yahoo has MyYahoo and Yahoo Mail for feeds.

Consumer RSS Readers are rapidly becoming commodities and will soon be next to worthless ... Bloglines and Rojo both got out while the going was still good, via acquisitions.

Consumer RSS Readers are a dead market now.

This day has long been predicted by many including me. Back in Sept 2004, I said:

RSS is already integrated into Firefox and probably will soon be in Safari, IE, and Mozilla. My Yahoo already has a web-based feed reader in beta; Google and MSN may follow soon.

Will independent feed readers survive the entry of these giants?

But all is not lost. As I said in my earlier post, "RSS sucks and information overload", there is hope for those that do more than current generation of feed readers:

The problem is that the current generation of feed readers merely reformat RSS for display. They don't do anything else, no prioritization, no filtering, no help dealing with the flood of information.

The problem [should be] scaling attention. Readers have limited time. They don't want information. They want knowledge. Our job is to help them, to help them focus, prioritize, and find what they need.

Next-generation feed readers should help people find knowledge. Cut through the undifferentiated glut of information and find focus. Cut through the noise and discover knowledge.

See also my previous posts, "Getting your grandmother to use RSS" and "A relevance rank for news and weblogs".

More on Netflix contest

Some interesting tidbits on the Netflix movie recommendations contest lately.

First, 27 days after the start of the contest, there are now 36 entries that beat the performance of Netflix Cinematch. The top entry already has a 5% improvement. It is still a long way from the 10% improvement required to win the grand prize of $1M, but the gap is closing.

Second, there is a fun post in the Netflix contest forums by Benji Smith about the "most hated", "most loved", and "most contentious" popular movies according to the Netflix data. [Found via kottke.org]

Third, Netflix's VP of Recommendation Systems Jim Bennet gave a Sept 2006 talk (PDF) about their Cinematch recommender system. The talk mentions that Cinematch uses an item-to-item algorithm -- the same type of algorithm used by Amazon.com's recommender system -- and includes some nice tidbits such as the characteristics of movies that they can accurately predict. At the end of the talk, Jim provides some justification for why Netflix is spending $1M on this contest, saying that higher quality recommendations are "absolutely critical to retaining users." [via Recommenders06]

See also my original post on the Netflix contest, "Netflix offers $1M prize for improved recs".

Spamming YouTube

In his post, "Gaming YouTube for Fun and Profit", Pete Cashmore looks at how easy it is to get a video featured in YouTube's most popular lists:

YouTube is increasingly being gamed by users.

It's exceptionally easy to rank among the most viewed videos and channels by simply refreshing the page. This widespread gaming might also throw into question YouTube's claim of serving 100 million videos per day.

I grabbed a video called "Sheep" ... and re-uploaded it under the username "themusichall". I realized that I'd lost the audio in the process (converted it to the wrong format), but decided to leave it like that - nobody would voluntarily share a 7 second clip with no audio.

I then set the page to refresh itself over Sunday night and - sure enough - it was among the most viewed clips this morning. Admittedly, 10,000 or so views doesn't get you to the top - it's on the 3rd "Most Viewed" page and ranks 10th in the Comedy category ... [but] it's pretty obvious how you could attain the number one spot.

See also my previous post, "Digg struggles with spam".

See also my earlier post, "YouTube is not Googly".

[Pete Cashmore post found via Matt Marshall]

Update: Just two months later, Mark Cuban looks at the top videos for Dec 2006 on YouTube and says, "That's what Youtube has become. Fake Porn and Commercials ... How long before fake porn just takes over? It was 9 of the top 20 for the week as I write this." Dare Obasanjo adds, "I doubt Google spent $1.62 billion for Youtube just to watch it turn into a haven for fake porn."

Going to Web 2.0

I (and therefore Findory) will be at the Web 2.0 Conference in San Francisco November 7-9.

Web 2.0 looks like a great event. The list of speakers and attendees is impressive. It should be a lot of fun!

Google Custom Search and instant answers

As discussed by many others ([1] [2] [3] [4] [5] [6] [7]), Google announced a new feature rather boringly named "Google Custom Search Engine" that allows anyone to "create a search engine that reflects your knowledge and interests; looks and feels like your own; and ... make money from the traffic you receive."

The actual implementation is not quite as grand as that. As Greg Sterling said, it is basically "industrial-strength Rollyo" where people can limit to or favor search results from specific domains and bias the search results to favor matches to given keywords.

Even so, it is fun and let me easily test an idea I had been kicking around to bias search results to sites that provide high quality answers like Wikipedia. Here is an example inline:

Search for Answers

and here is the same custom search hosted on Google. Try a search like [San Francisco] on this custom search engine and compare to the same search on Google. Medical searches like a search for [arthritis] also seem a bit more useful to me.

This custom search takes the bias Google Web search results seem to have toward Wikipedia a step further. It also favors a few other sites that focus on providing answers. The results are pretty good for questions and requests for factual information, but it obviously is only a twiddle on top of what Google already does, and it does not help for all types of queries (e.g. navigational queries).

By the way, it is not obvious to me that a bias toward Wikipedia or other sources is the right thing in the general case. The existing ranking algorithms already attempt to bias toward authoritative sources. At best, if you like these results better, it might mean that a few particular sources should be considered more authoritative than they already are.

In any case, it is fun to be able to play with this kind of thing so easily and quickly.

Ongoing confusion about Live brand

Mary Jo Foley describes the confusion about the Windows Live brand and how it relates to Windows, Microsoft, and MSN:

Microsoft is actually making some real headway in the way it is developing and distributing services, but almost no one knows it, thanks to the abysmal job the company has done in defining Live and updating the various Microsoft constituencies on its progress.

"Live" is Microsoft shorthand for services. Windows Live is not, as many still assume, a new, hosted version of Windows; it is the set of services extensions to Windows. The same is true of Office Live. (Confusingly, however, CRM Live is a hosted version of Microsoft CRM.)

There's ... been mass confusion ... about exactly what Live is and how Microsoft's Live strategy hangs together ... Analyst Matt Rosoff [said], "It seems like end-users who don't particularly follow Microsoft have never heard of Live or confuse it with the next version of Windows, and customers, partners, and advertisers often express puzzlement over the difference between Windows Live, Live (e.g., Live Search), and MSN."

Microsoft has used "Live" to mean several different things, Rosoff added. "We ... view Windows Live as ... essentially the latest chapter in the long story of MSN. But sometimes the Live brand is also used to describe broader concepts, such as software being delivered as a service or subscription-based models for buying software. I don't think the brand has been as misused as .Net was a few years back, but it's still fairly indistinct."

See also my previous posts, " Is it Windows Live, MSN, or Microsoft?", "Is it Live or MSN?", and "Office Live is not Office Live?".

Update: Three months later, after a disappointing quarterly report, Gartner analyst David Smith says, "Microsoft's Live branding has been tremendously confusing and has hurt the company, and it is very likely contributing to the situation they are in right now. They've created another brand and have not differentiated it." [Found on Findory]

Update: Three months later, Mary Jo Foley makes a Herculean effort to understand how the Live brand is used and writes, "After going out and searching for information on all the Live services I could find, I feel like I'm even more confused about what Windows Live is (and isn't) than I was before."

Geeking with Weird Al

I thought Weird Al Yankovic would never do better at capturing the essence of geek than his song "It's All About the Pentiums" (YouTube video).

But, his new song, "White and Nerdy" (Google Video), also is inspired. One of my favorite parts: "There's no killer app I haven't run / At Pascal, well I'm number one / Do vector calculus just for fun / I ain't got a gat but I got a soldering gun." Yep. As Weird Al says, "How'd I get so white and nerdy?"

If you're a geek and you haven't seen both of these videos, take the time to check them out. Definitely good for a laugh.

Reddit, Digg, and personalized news

Emre Sokullu and Richard MacManus at Read/WriteWeb wrote an article, "Personalized News: A Market Overview", that covers several startups trying to recommend news articles and build personalized front pages for readers.

Some excerpts:

Our guess is that personalized content will become a more popular paradigm in about 1 to 2 years.

Personalized news has a couple of main attractions. Theoretically, if your news is personalized then it's not as vulnerable to gaming as [Digg's] power of masses approach. Plus people are getting busier everyday, so personalized news has a strong appeal as a potential solution for information overload.

The article mostly talks about Reddit. I found this a bit odd, since Reddit strikes me as closer to Digg than a personalized news site, but Reddit does have a "recommended articles" page off their front page.

As the article explains, Reddit appears to use a keyword-based approach like the Bayesian filter Paul Graham developed for spam filtering. I doubt that this simple content-based approach can be made to work well, And, unfortunately, as Emre said, "many [Reddit] users still complain about not receiving relevant news recommendations."

I went back to try Reddit's recommendations again -- I hadn't looked at it in a while -- and, after rating a few articles, it still put up a message that "no links have been recommended for you yet. keep telling reddit what you like and dislike by voting on links, and check back here later for recommended links." I never got to the point where I actually received recommendations. That is a problem. Recommender systems need to work from sparse data in real-time. They need to react immediately and instantly to new data.

After rating ten articles, mostly about Google (on GDrive, MapReduce, Sawzall, BigTable), on Reddit, I finally received one recommendation, "Fun with Javascript". After a few more minutes, I received a few more recommendations, one a political article about stem cells, one on messing with telemarketers, and one on the "Father and Son story of the century". So, yes, I agree, not relevant.

Emre suggests that all personalized news sites are like Reddit. Findory does not use Bayesian analysis over keywords. Instead, Findory uses a form of social filtering where Findory readers anonymously and implicitly share the articles they find and enjoy with other Findory readers.

Oversimplifying a little, Findory works a bit like Digg except that rather than seeing a front page of the generally most popular articles, you see a front page of the articles that are most popular for readers like you. As Emre said, different lists for different people reduces the incentive to game the system by eliminating the winner-takes-all effect.

In general, the power of the masses approach, epitomized by Digg, has two problems with relevance. First, a most popular list is generic and untargeted; it is only relevant as long as your interests match those of the entire community. Second, as power of masses sites reach a mainstream audience, the incentive to spam grows and relevance drops. Personalized news has neither of these issues.

Finally, it is worth noting that personalized news is not limited to crazy little startups. Both Google News and Microsoft's MSN NewsBot have a small widget on their front pages that recommend news stories based on the articles you read.

Findory Q3 2006 traffic

Findory's Q3 traffic numbers are in. The good news is, after a dip, there is a modest increase from the previous quarter. The bad news is that, after exponential growth for two years, Findory's traffic now appears to be flat.


The flat traffic almost certainly is due to lack of resources. Findory remains a self-funded, small startup. That provides a lot of flexibility, but limits the company's ability to address larger projects.

In particular, Findory has not had the resources to address big opportunities such as international expansion, licensing deals, personalized web search, improving our feed reader and customization options, or creating a behavioral advertising network using Findory's personalization.

Findory also has lacked capital for traditional marketing; our advertising budget is not perceivably different than zero. This has limited Findory's ability to reach out to the mainstream audience it seeks.

It is a challenging question what to do next. Growth clearly requires additional funding for Findory. But, those additional resources create some opportunities and come with constraints that limit others. The company is at a choice point that requires some careful thought.

By the way, I have received some advice to stop posting these traffic numbers since they now may be viewed negatively. While I agree that is a concern, I think that it is best to be transparent with this kind of data. I will continue posting Findory's traffic stats every quarter.

See also my previous posts ([1] [2] [3] [4] [5] [6] [7]) about Findory's traffic over the last several quarters.

Eric Schmidt on personalized information

In Google's Q3 2006 earnings call, Google CEO Eric Schmidt emphasized personalization of information in Google's mission and future plans.

Some excerpts:

We believe that people's information and the information they want to receive ... needs to be accessible when and where they want it for them in a very personalized way.

The interesting thing is that this approach to having your information personalized is a benefit not only for the user who can continue to refine and target information ... but also for businesses who want to know they are spending their money in an effective and targeted way.

As we continue to innovate and bring out ... new products, we'll also continue to ... improve the experiences, bringing the most personalized and targeted information to people, which is ultimately our mission.

[We] provide access to the world's information ... [and] organize it in a very personalized and targeted way. That benefit drives the entire cycle of Google, and it's fundamental.

[Conference call transcript link found via Paul Kedrosky]

What should Google do next?

Jessica Mintz from the AP has a fun article asking a few entrepreneurs and analysts, "What should Google do next?"

An excerpt from the article:

Imagine you're Google right now: darling of Wall Street, $10 billion dollars in the bank, the brand-new owner of one of the hottest video-sharing sites on the map.

What would other high-tech entrepreneurs do now if they were running Google?

"There's so much work that needs to be done around search," [Technorati CEO Dave] Sifry said. [We need] "a Google of intention," driven by users' real-time needs.

Nick Denton, publisher of ... Gawker Media, wrote ... "I would get search working, because the results are cluttered with commercial rubbish that ought really to be in the advertising zone."

Henry Copeland, founder of the BlogAds network, wrote ... "Use the collective intelligence of its users to eradicate spam."

There were, of course, a few more outlandish ideas -- buying online auctioneer eBay Inc., for example -- as well as a few geeky ones involving free database design software.

But all in all, the consensus seems to be that Google needs stay focused -- deepen the moat around the core search business, as one analyst said.

I am briefly quoted in the article arguing that Google should go after e-commerce. I do think they should do that, but my complete response about what Google should do, appended below, emphasized core search and advertising more than e-commerce:

I would not stray far from Google's area of strength. Google is good at one thing, using technology to help people find the information they need. This job is far from done; abundant opportunities remain.

Aside from expanding their core search and advertising products with question answering, personalization, machine translation, and query refinement, Google should aggressively expand into e-commerce and look to helping content producers make money.

On e-commerce, Google should be helping people find and discover products they want to buy. Froogle, an early Google effort at shopping metasearch, has gone stagnant. This is an opportunity lost, a market the size of eBay and Amazon combined, and one that may grow to the size of Wal-mart. Google should be the first stop for anyone wanting to buy anything online.

On helping content producers, newspapers and magazines see their audience moving online, but are frustrated by the low revenues earned from an online audience. The market opportunity here is the size of the entire media business. Google already has taken a step toward helping small content producers with AdSense, but revenues and relevance need large gains to be able to support the mainstream media. Google should be expanding and improving its advertising network. Google needs to use techniques such as personalization and geolocation to make advertising more relevant, useful, and lucrative for content producers.

It is possible that Google could speed these initiatives along with large acquisitions -- acquiring Amazon.com, for example, would give a big boost in e-commerce -- but they will not. Instead, Google will use their $10B in cash to hire great people, continue building out their server cluster, and do hundreds of very small acquisitions. For the most part, Google will build instead of buying.

I realize my prediction may not be as exciting as a big merger or completely new initiative, but the right thing to do is not always exciting. I think this is what is mostly likely to build value and what Google is most likely to do.

What do you think? Did I miss a big opportunity for Google? Am I overestimating the value of e-commerce and helping content producers? What do you think Google should do next?

Google A/B tests for all

Google Website Optimizer appears to allow other websites to run A/B tests to optimize the effectiveness of their designs, features, algorithms, and content.

From their help page:

The Website Optimizer allows you to test changes in the website content of your pages in order to determine what will be most effective in getting conversions.

You choose what parts of a page you'd like to test -- headline, image, promo text -- and we'll run an experiment on a portion of your site traffic to determine which content on your site users respond to best.

When we've collected enough data, we'll provide you with reliable reports and a suggested course of action in order to optimize your site for maximum business results.

Much more detailed information in their technical overview.

It is no longer just Amazon and Google who have powerful A/B testing frameworks. Google appears to have just opened up their A/B testing tools to everyone.

[Found via Liam Morrison and Philipp Lenssen]

AdSense will not do behavioral targeting?

In a recent interview, Kim Malone (Director in the AdSense group at Google) said, "Behavioral targeting is not something that Google [AdSense] will do."

Kim could not be more clear in this outright, flat denial, but I simply do not believe it. There is too much to gain. Ads targeted to individual behavior are more relevant, useful, and interesting to users and convert better for advertisers.

If Kim is telling the truth here -- if Google really will not get into behavioral targeting -- I would say this represents a beautiful opportunity for Yahoo, Amazon, and Microsoft to trounce AdSense. Use a little more data, pay a little closer attention to what people want and need, and your ads will be more relevant and less annoying that Google's contextual ads.

While they have been painfully slow to execute, Google's competitors are moving on behavioral targeting.

Microsoft adCenter lets advertisers target ads "to the customers most interested in your products or services." Microsoft adLab, the research arm of adCenter, says their goal is to "change online advertising dramatically in areas such as paid search, behavioral targeting and contextual advertising", claims they have "years of profiling, user behavior, and data mining", and asserts that they "know a heck of a lot more about their audience than Google knows about its own."

Yahoo has been working with Revenue Science "to show text listings on Web pages based on user behavior." Yahoo also recently launched their Panama ad system which, according to Danny Sullivan, will include "audience targeting based on factors that could include demographic information or online behavior" in "future versions."

Amazon Omakase targets ads "based on what the Associate has been successful with in the past; what that user has been interested in; and what the site is about." In my opinion, other than Findory, this is the most dramatic example of fine-grained behavioral targeting out there right now. The Amazon Omakase ads I see on other websites are eerily relevant, targeted to my purchase and clickstream history at Amazon.

It is unbelievable to me that Google is not pursuing behavioral targeting. But, if Google is not pursuing behavioral targeting, it is clear that its competitors cheerfully will.

Talk on Google AdWords and AdSense

Shiva Shivakumar (Director, Google Kirkland) who "led AdSense through beta, launch and hypergrowth" gave a talk at University of Washington Computer Science called "Google Ad Systems" (video available).

The talk is a light introduction to AdWords and AdSense. It mostly covers the history of the products with some brief discussions of the challenges around relevance, scale, optimal auction pricing, and click fraud.

The talk is worthwhile, a good overview of the work involved in building a system like AdSense and AdWords.

I tend to follow this stuff fairly closely, so the only thing that was new to me in the talk was hearing that AdWords is still running on top of a massive MySQL deployment. AdSense, in contrast, is running on top of the Google data infrastructure, GFS and Bigtable. Shiva was not clear about whether the difference was merely due to legacy issues or whether there is something special about the AdWords data access patterns that makes MySQL preferable.

Shiva's talk unfortunately only touches briefly on auction theory and click fraud issues. If you are interested in more details there, you might dive into Jan Pedersen's talk and a related paper I discussed in an earlier post.