Given how many people appear to use Alexa data to make serious business decisions, it seems like Alexa traffic charts are thought of as reliable and accurate. But are they?
As the Alexa site itself warns, the data is not accurate or reliable. An excerpt:
Alexa computes traffic rankings by analyzing the Web usage ... of Alexa Toolbar users.The data is only for Alexa toolbar users, a small and heavily biased sample of Web users.
The Alexa Toolbar works only with the Internet Explorer browser ... The Alexa Toolbar works only on Windows operating systems ... The rate of adoption of Alexa software in different parts of the world may vary widely.
Sites with relatively low traffic will not be accurately ranked by Alexa.
I recently analyzed the Findory.com logs to look at how large this sample might be. On May 17, only 241 hits on the Findory.com website have been from people with the Alexa Toolbar installed. That's less than 0.1% of the total hits on Findory.com in that period, a tiny fraction. And those 241 hits came from only 49 unique visitors.
It is so small a number that it would be trivial to manipulate. Simply installing the Alexa Toolbar and browsing daily through the site could double the page views reported by Alexa. Asking a few dozen regular Findory users to install the Alexa Toolbar could double the reported reach.
I think it would be pretty lame to do that. Trying to manipulate metrics instead of building things that are real always strikes me as a foolish waste of time. However, it does appear many people seem to be spending a lot of time manipulating Alexa numbers, probably because real money flows to those that do.
Clearly, Alexa traffic charts should be used only with careful caveats. Only for large sites, over 10M page views per day, would I consider the data reliable. Otherwise, the tiny, biased sample easily can be manipulated.
Update: Six months later, Jason Calacanis runs an experiment to game Alexa metrics. Interesting.
Update: Eight months later, Peter Norvig writes about problems with Alexa data in his post, "Alexa Toolbar and the Problem of Experiment Design".
3 comments:
Very interesting, Greg. I've often wondered about Alexa metrics myself, and actually didn't know that you could see whether they have the toolbar or not installed. So this certainly answered a few question for me. Thanks.
Thanks for the comparison to your real-life stats, Greg.
It is possible to have your browsing reported to Alexa via the Saerch Status extension in Firefox:
http://www.quirk.biz/searchstatus/
I actually didn't know until I saw a private subdomain make its way onto Alexa. Woops!
Of course, the demographic that not only uses Firefox, but would install an extension is also not a good sample.
Yeah, and webometrics.info uses this Alexa statistics for research purposes. I wrote about my doubts concering Alexa statistics in my swedish blog in April 2005:
http://internetbrus.com/2006/03/17/gar-det-att-lita-pa-webbstatistiken-fran-alexa/
Post a Comment