Sorry I edited my post while you were posting yours.

The article has been updated to include this:

[UPDATE: Someone has just pointed out to me that Brett Walton has accused me of reprinting a 'confidential email conversation' for this article. In the course of my discussion with Brett, I specifically asked him if our emails were on the record. He replied: "I have no issues reproducing the discussion - needs a tidy up of course." As far as I'm concerned, issues like this are symptomatic of why VGChartz cannot be trusted.]

hmmm...yeah...best not to publicly complain.

Any tracking service has to use "guessing" based on past trends.

Take NPD. They alledgedly use 60% of the retailers' data. I assume the other 40% of their figures must be not only extrapolated by a mathematical formula, but also subjected to adjustments based on the demographics a game attracts. For example, you'd expect a title like Ninja Gaiden 2 to sell much better at Gamestop than Walmart. On the other hand, a title like Wii Play may sell better at Walmart. A simple mathematical formula can't embody all of these concepts unless it includes some manual parameters which are input by subjective humans.

Of course, I myself am guessing NPD's methodology which AFAIK is not public either. There could be a lot of guessing there too, and we'd probably never know. Of one thing I'm fairly sure though - NPD doesn't have a magical accurate formula to get that 40% of the market based solely on the other 60%.

All I can say is if I'd used those methods in my degree I'd have failed. To me, a sales tracker tracks sales. It doesn't guess, look at past trends, or anything else. If, for example, the entire population was 1 million retailers of equal size it would survey 100 of them and multiply the figures by 10,000 to produce the overall figures. As you correctly point out, there is not a uniform market, and some retailers have to be weighted. But weighting does not include compiling other people's figures and predictions. The methodology ioi gave shows that this is what VG Chartz does. It compiles a mishmash of sales figures, previous trends and guesswork then changes things when the NPD data appears. I've defended this site because I thought it just used a small and therefore unreliable sample size. The truth appears somewhat different

That method you mentioned can't work in practice, it would give more errors than VGChartz or NPD have in their data... All I'm saying is that any method which aspires to be somewhat accurate must include some subjective factors.

You yourself mentioned that some retailers have to be weighted. Do you agree that those weights will be heavily dependent on the demographics a game attracts? To use the same example, what would happen if you assumed Ninja Gaiden 2 and Wii Play were equally distributed among shops? I'd bet a lot of money that you'd get massive errors on at least one of those games' numbers. It's just not possible to have a reasonable estimate without accounting for the vast differences in sales distribution of different games. The choice is then to either have wildly inaccurate numbers generated by a rigorous process, or more accurate numbers generated by a mixture of objective and subjective methods. I know which one I'd choose (i.e. I wouldn't bother with the former).

I can't comment much on the rest that ioi said, because I'd be guessing too much... What I'm saying is that we shouldn't be surprised that there are subjective factors in tracking sales which may easily be called guessing.

Lot of articles about VG last months;..

I was going to do an editorial on all this but I think this works better:

We get sample data every week for the Americas, if it weren't classified I could post it from last November through last week which is the period for when I've seen it. Its just not enough store variation for some software so there has to be a concerted effort to look at shipment info, how similar games performed on hw at a given time, etc. The real advantage NPD has over us isn't the larger sample size, so much as it is the diversified sample. But still, it is just a larger sample size, and since it covers the majority of the market, but not all of it, and tends to be conservative, we aim to put sell-through figures closer to shipments. NPD isn't perfect, whenever I talk to some of my contacts I hear stories about how NPD changes figures after public release, and spreads out the changes over months, supposedly conflicts between NPD and Microsoft internal figures happened for months on end at Microsoft.

We're not taking other people's figures and predictions since those are flawed and subjective. We have a sample, we weight it for the data that is provided. Most games perform in a similar pattern in spiraling down from week one sales in the sample data, and that is reflected on the site. Adjusting to NPD figures would be silly given that they do not cover the entire market, but if the pattern is different from what we have there is no reason not to recognize it as reflective of 60% of the market in the cases where we have limited data. Its not like we do (Pachter x 2)+(NPD x 10) + (Sample x 3) and divide by 15 to get data or something. Frankly when we have data that is diametrically opposed to what NPD reports we ignore it, when the data is similar we have no reason to change it, it when a game falls in between those two ends because our sample wasn't diversified enough for propper scaling then we go back and weight the sample information differently. That isn't copying NPD, thats us being modest and recognizing that our scaling factors don't always hold steadfast. So if No More Heroes or something sold better nationally than in our sample, it just means we're scaling up the sample, we don't copy NPD, we just use it as a better idea for weighting tough to track games closer to shipments.

If NPD didn't report data we would still be able to provide weekly data from using our sample by constantly looking at the number of stores in a given market, comparing our coverage to that figure, and making the necessary adjustments to weights based on shipment data.

It's great being on a site that allows complete and open discussion about criticism. I'm glad that threads like this can exist. I'm not entirely happy with the way the author has handled himself and others working for that site handled themselves. It's very much as I said. It's a smear campaign against this site to try and win Gaf approval for their own site.

Disclosure: I can say this because I have no affiliation with this game site or any other game site other then being a forum poster. ;)

Edited

This is the big time now, Boss.  You can't say things you don't mean...and mean things you don't say.