We all know that VGChartz is being attacked on various fronts for not having accurate data, and for the numbers to be subject to change at any moment without notice. To gain some insight into the validity of this critisism and satisfy my personal curiosity I started The Weekly adjustments thread. I've been using a spreadsheet to do the simplest calculations I could possibly do on VGChartz hardware adjustments for the three consoles in the three big terretories. So far, all I have done is gather data. Now I thought it could be interesting to look at this data and see what I've learned.

Before I start I must apologize for any inaccurasies, misspellings and other mistakes, as well as the rambling and incoherent nature of this analysis. I will do my best, but suggestions and constructive critisism is very welcome.

Now, onto the numbers!

Sales for the consoles since I started measuring (12 weeks):

The weekly average for each console is:
Wii - 320k
PS3 - 290k
Xbox - 170k

These numbers gives us the base against which we can judge the adjustments.

Taking the sum of the absolute values of each weeks adjustment gives us the following table:

To get the cumulative total adjustment for the period you simply add all adjustments together:

All numbers combined gives us the full picture:

So what does this awful table really mean? For the sake of clarity I'll take an example. The Wii has sold an average of 320k per week. During these 12 weeks it been adjusted both up and down by an average of 33k every week, or 21.34% (Absolute adjustment). The result of these adjustments is, in the end, that it lies on average 17k above the originally reported numbers every week, or 11.23% (Cumulative adjustment).

The result I've come up with is this - The originally reported VGChartz hardware numbers are accurate to within +-10%. During these 12 weeks adjustments have indicated that the Xbox is being overtracked by 9%, with all of those numbers coming from others. The PS3 is being undertracked by 3.5%, and the Wii is being undertracked by 11% in others, resulting in a 6.4% overall undertracking.

When looking at the weekly averages for the last 12 weeks and how they have changed, the picture changes quite a bit:

The PS3 still has been undertracked significantly in other for this period, but overall percentages is quite predictably down quite a bit compared to the total.

That is, I think, not bad at all. VGChartz is pretty accurate with its weekly figures.

What I'm doing here is based on a simple assumption: VGChartz will adjust numbers so that they are accurate. The number of sold consoles in a year is reported, and VGChartz will adjust numbers to reflect those reports. Thus, VGChartz numbers for last year should always be more or less accurate. My goal is therefore to keep track of all adjustments made to previous figures, so that after a while looking at those adjustments can show how accurate the original VGChartz figures were.

Naturally, there are a couple of problems with this analysis. The first problem is that this is all internal VGChartz numbers. I have not gathered data from any other source, so if VGChartz is dead wrong with anything and refuses to adjust it, this analysis cannot catch that. The second problem is that my data is from a relatively short period of time (12 weeks). My numbers will get more accurate over time, but for now the adjustments made every week covers weeks that I do not have data for, which means that the above analysis serves more as a general guideline than anything else. The last problem is that I do not track adjustments made on weekly numbers, but on total cumulative numbers. That means I easily miss subtleties.

I'll try to do another set of numbercrunching when we get to 26 weeks (half a year), but I'm not sure whether or not I'll still be keeping track of this by then. For now, you'll just have to take my figures with a grain of salt.

TL;DR - Pretty accurate (I think)!

VGChartz paid by Microsoft confirmed?

Nice analysis, I thought the Wii was overtracked but it was the other way around.... at least in these weeks :p

Indeed. Microsoft paid VGChartz off sometime last month. We'll see even more impossible weekly figures that will be adjusted down next week. Trust me.

Nice thread killergran. This is really impressive accuracy, imo.

Ignoring 'others' the biggest discrepency was less than 6%

And others is excusable to be a bit less accurate - VGChartz is the only site that even tries to keep track of it all.

so wii undertracked, ps3 undertracked, 360 overtracked? (all in others?)

OT: 10%+- on the initial reported numbers is an amazing feat! Show me anything else that can report these type of numbers on a weekly basis, I dare you.

It will be really interesting to see how this works out. And it will put all the talk about VGChartz accuracy in perspective.

If it will remain in the +- 10% range that will be fantastic!

@markers, yes, that's pretty much it.

No, that's essentially true. And it's one of the weaknesses I talked about. It will be all right in the long run, but in the short run it's kinda... inaccurate. The Wii's numbers were adjusted around +100k or something for the weeks that I covered, the rest was weeks before that. Since I only measure adjustments to the total, I had not discovered this until I put the old average compared to the new average + adjustments. Those figures turned out horribly wrong and it took me an half an hour to figure out what had happened :D

Still, the methodology I use has a solid logical ground, and even though these figures can be slightly off in the beginning, they'll improve massively over time.

No, it really doesn't. I expect the average over these 12 weeks to correspond very well to the yearly average. VGChartz adjusts its data every week. Sometimes big, sometimes small, but every week in any case. My thinking is it will average out eventually, since the period covered will be moved forward every week. I think the quarterly average will look very similar except over christmas.

Excellent work.

VGC is pretty accurate.

VGChartz isn't 100% accurate but neither are the NPD figures since they don't track Walmart, the most popular place of video game shopping.