Statistical Differences: Can You Trust Project Wonderful?
Submitted by Daniel Potter on August 11, 2010 - 01:00
I’m a total statistic junkie. I admit it. I’m one of the stupid people who will refresh my stats a half million times a day and I get very frustrated when Google Analytics lags on me.
So each has a unique viewpoint into your site. But really that’s not the point here. Let’s compare the outputs on my own comic, Walking the Lethe.
Now let’s look at Google Analytics:
Note that unlike PW, there are two axis on the Analytics graph, on the left are the page views and on the right are the visits.
And now AWstats (had to work a little excel magic to get these on the same graph.)
Look at all of these; note the high and the lows. First thing I notice is, without getting into the numbers, is how similar the profiles of these graphs are. All three of these programs are detecting similar peaks and valleys in our traffic. That’s good.
Now lets look at the numbers. This is where some real differences appear. AWstats shows the largest difference with page views that are over three times the number that Analytics produces on the peaks of traffic. PW’s multiplier appears to be less than two. Frustratingly, what holds for the peaks does not hold for the lower numbers as all of the packages get closer each other the lower the number is. In a few places, such as July 8th, PW is actually reporting less page views than Google Analytics, so the relationship between the three is not linear.
So what is “right”? I don’t know. You can track the relative growth of your web page by any of the methods. However there is an additional wrinkle to consider with Project Wonderful stats: we buy ad space based on these numbers. If they are inflated, then we’re not getting what we’re paying for. It is good that the page views of PW are actually closer to Google Analytics’s values that I expected. The rule of thumb that circulates the internet is to divide PW’s numbers by half. In my experience, PW is actually fairly comparable to Google Analytics, but the multiplier seems to vary the most on spikes of traffic.
Now a visitor could look at a single page or a hundred. The only way to truly figure out how popular a site is to look at the number of visitors to the site. Here again Analytics is low, but now AWstats is only twice that while Project Wonderful is now nearly five times the analytics. Worse, PW’s peaks don’t match up with AWstats and Analystics. Look at the third week in particular where the peaks match with the previous week on AW and google but are recorded as lower on PW. This is disappointing. As much as I would like five thousand visitors on my site, (I somehow think I’d be getting more comments), the drastic inflation plus the change in the profile in the graph, points to PW not being a reliable source for determine the number of visitors to a site.
Based my site alone, Project Wonderful’s page views are good enough for me to feel comfortable buying ads biased on them but I don’t trust their visitors statistic. Please feel free to conduct your own analysis and, perhaps, share the results with us in the comments.