Monday, November 17, 2008

Measuring offline ads by their online impact

Googlers Diane Lambert and Daryl Pregibon had a paper at AdKDD 2008, "Online Effects of Offline Ads" (PDF), with a fun look at how far we can get measuring the impact of offline advertising by increases in search queries or website visits.

Some excerpts:
One measure of offline ad effectiveness is an increase in brand related online activity .... As people spend more time on the web, [the] steps toward purchase increasingly include searching for the advertiser's brand or visiting the advertiser's websites, even if the ad campaign was in an offline medium such as print, radio, or TV.

There are two obvious strategies for estimating the [gain] ... We can assume that the past is like the present and use daily outcomes before the campaign ran ... [but] the "before" number of visits ... is not necessarily a good estimate ... if interest in the product is expected to change over time even if no ad campaign is run. For example, if an ad is more likely to be run when product interest is high, then comparing counts-after to counts-before overstates the effect of the campaign.

Alternatively, we could estimate the [gain] ... by the outcome in control DMAs, which are markets in which the ad did not appear ... One problem, though, is that the advertiser may be more likely to advertise in DMAs where the interest in the product is likely to be high.
The paper goes on to detail the technique used and the ability of the technique to detect changes in very noisy traffic data.

This paper is part of a larger group of fun and remarkably successful work that tries to predict offline trends from online behavior. One example that recently received much press attention is Google Flu Trends, which uses searches for flu-related terms to predict real flu outbreaks. Another example is recent work out of Yahoo Research and Cornell to find the physical location of objects of interest from a combination of search terms and geolocation data.

2 comments:

jeremy said...

How far down the long tail is it possible to go with work like this?

My guess is that it would not be very far. There is so much noise that, unless the object of study were in the "head", i.e. common and frequent, it would be drown out by the noise. This limits this type of work to things like the flu (common illness) and large (nation-wide scale) advertising campaigns.. e.g. national elections, automobiles, etc.

Do they address this issue in the paper?

Daniel Tunkelang said...

Similar reaction here--it would seem that this is most useful for evaluating mass market campaigns, where there is enough signal to overcome the noise, rather than the targeted campaigns that I thought were Google's specialty.