Skip to main content

Statistical Differences: Can You Trust Project Wonderful?

I’m a total statistic junkie.  I admit it. I’m one of the stupid people who will refresh my stats a half million times a day and I get very frustrated when Google Analytics lags on me. 

Now everyone says not all statistic solutions are created equal and in particular Project Wonderful’s free and open statistics are looked down on.  So I’ve been wondering, is this actually fair?   So let’s compare the three big ways that people check their statistics on webcomics.  Project Wonderful, Google Analytics and AWstats.  Now each Project Wonderful tracks hits to their ad box, Google Analytics uses javascript and AWstats records each bit your server sends out.  

So each has a unique viewpoint into your site.  But really that’s not the point here.  Let’s compare the outputs on my own comic, Walking the Lethe.

Project Wonderful:

Now let’s look at Google Analytics:

 Note that unlike PW, there are two axis on the Analytics graph, on the left are the page views and on the right are the visits.

My google stats

 

And now AWstats (had to work a little excel magic to get these on the same graph.)

 

Look at all of these; note the high and the lows.  First thing I notice is, without getting into the numbers, is how similar the profiles of these graphs are.  All three of these programs are detecting similar peaks and valleys in our traffic.  That’s good. 

 

Page Views

Now lets look at the numbers.  This is where some real differences appear. AWstats shows the largest difference with page views that are over three times the number that Analytics produces on the peaks of traffic.  PW’s multiplier appears to be less than two.  Frustratingly, what holds for the peaks does not hold for the lower numbers as all of the packages get closer each other the lower the number is.  In a few places, such as July 8th, PW is actually reporting less page views than Google Analytics, so the relationship between the three is not linear. 

So what is “right”?  I don’t know.   You can track the relative growth of your web page by any of the methods.  However there is an additional wrinkle to consider with Project Wonderful stats: we buy ad space based on these numbers.  If they are inflated, then we’re not getting what we’re paying for.  It is good that the page views of PW are actually closer to Google Analytics’s values that I expected.  The rule of thumb that circulates the internet is to divide PW’s numbers by half.  In my experience, PW is actually fairly comparable to Google Analytics, but the multiplier seems to vary the most on spikes of traffic. 

Visitors

Now a visitor could look at a single page or a hundred.  The only way to truly figure out how popular a site is to look at the number of visitors to the site.  Here again Analytics is low, but now AWstats is only twice that while Project Wonderful is now nearly five times the analytics.  Worse, PW’s peaks don’t match up with AWstats and Analystics. Look at the third week in particular where the peaks match with the previous week on AW and google but are recorded as lower on PW.  This is disappointing. As much as I would like five thousand visitors on my site, (I somehow think I’d be getting more comments), the drastic inflation plus the change in the profile in the graph, points to PW not being a reliable source for determine the number of visitors to a site. 

Conclusion

Based my site alone, Project Wonderful’s page views are good enough for me to feel comfortable buying ads biased on them but I don’t trust their visitors statistic. Please feel free to conduct your own analysis and, perhaps, share the results with us in the comments.