His complaints were all completely valid, and it wasn't attacking story at all.  He simply stated the weaknesses of VGChartz.  The weaknesses amount to a lack of resources on the part of VGChartz -- they can't get a lot of sales data, and they aren't quite clear enough on how the data is drawn.  The European data for MGS 4 should obviously have been listed as nothing but estimation.

The conclusion is reasonable: for now, VGC is good for the highly charted games but poor for the uncharted or rarely charted games.  That's fine, we're mostly interested in the highly charted games.

I have no major complaints with how VGChartz gets its data, however it would be nice if there was a little more transperency about the adjustments. My suggestion is to make a stickied thread under either sales or website discussion, every time there is a hardware adjustment, give us a post explaining what caused it (shipment figures, NDP data etc)

Obviously this would take too much effort to do with software.

You know we are better than this. If it is valid criticism the Boss should take it. How many times did he avoid banning VGchartz detractors as long as they provided a slightly valid critic? Questioning VGchartz numbers is ok as long as you can back it up or help improve its accuracy.

WTF? Welcome to the world of journalism! Did he lie? I don't think so. If you can handle journalist, let louie handle such questions ;)

Perhaps if on the methodology page, you explain some of the things we all 'know' but that might not be readily apparent; for example, major-selling software will generally be very accurate.  However, when you get into games that have less than, say, 300k worldwide, you are generally looking at a much larger (potential) percent error than the mega-hits.

This can come into play between the arrival of NPD data and publisher statements; however, I do not think that one or two bad apples in the data is as big a deal as the author of that article does, apparently.

I would suggest, however, that you just add 3 color-codings to the game database:

Yellow = VGC extrapolated data, unconfirmed by publisher reports

Orange = VGC extrapolated data + added data (which can change from game to game, be it NPD, publisher statements, whatever)

Green/Black = VGC extrapolated data confirmed by publisher FY reports or whatever.

This would make it so that there is at least some indication of how 'correct' the number is.  Obviously, some data may never make it to the Green/Black stage, and some other problems definitely exist in this. 

As a whole, however,I feel that it would add to the site's openness which definitely exists, although those who don't crawl the forums may not know it yet, because they may miss a lot of the discussion of the numbers and adjustments.

Well, the site probably shouldn't estimate sales based on absolutely nothing, put out a press release trumpeting that number without even calling it an estimate, change the number as real data comes in, and then tell some random reporter the whole thing. That's a big, embarrassing sequence of dumbness.

But as a responce to botching MGS4, it should be pointed out that VGC nailed Mario Kart Wii launch numbers in America, and then was the first source to predict Wii Fit sales under 1M for launch month in America. VGC was talking about how incredible the MKWii launch was and how the Wii Fit launch missed its goal before anyone else.

Also, the site should start talking up its community and the speed with which it gets numbers instead of its accuracy. Don't try to take on other sources in terms of accuracy, as it will take years to get to the accuracy of benchmark sources. The instant analysis performed by the community, as well as having the only numbers to go by for several days, or several weeks in America's case, plus the huge database of publicly available info, represent different values that the other sources can't compete with VGC over.

There will be people, like Gamasutra, which put a higher demand on accuracy in areas where VGC doesn't have it yet, and that's not the problem of those people. But don't let them reach a total conclusion about the site without getting a mention for the speed of the numbers, the scope of the database, or the quality of the instant analysis provided by the community.

Just a question here since everyone keeps touting "transparency" lately. Exactly how transparent is NDP? Do they publish where they get their data from, their methodologies, their extrapolations and their corrections?

I don't see why ioi "has" to do anything. This article seems to indicate that to be considered acceptable, ioi needs to publish who they get data from, what that data, how they extrapolated it, etc...

So what happens when he does that? People like NDP go to his retail sources and eat them up. Sorry but that sounds like a very horrible idea to me.

People need to learn how to read. Lesson 1 in learning how to read is to actually read the entire article even when, god forbid, you see something you don't agree with (yes, I'm looking at you NJ5). If you actually did read everything, you would have read this final paragraph that pretty much sums up the point of the article: 

"But if I was a writer or analyst trying to extrapolate significant information from the resource, especially regarding those titles which don't chart regularly, given the major discrepancies with other figures shown here, I would not recommend it." (emphasis added by me)

Basically, the author is saying VGChartz doesn't have good enough data to be used by professionals as gospel. And why would it be? The site's run by amateurs. Why would professionals use VGChartz as canon? The site's great for seeing some trends and getting a general idea of sales numbers. That's not a bad thing; it's just the limitation of this site. It's not the end all and be all in sales numbers. 

I tout transparency specifically *because* we don't get it from NPD.  Basically, I think VGC has to be the bigger person in this.

Now, I am not saying that Brett should have to do anything like release his sources, or anything like that.

But, specifically because VGC has a smaller sample size than other sources, it is much more important that we be up-front about its limitations.  As Erik Aston* said, we may not have the accuracy, but we have the speed.  If we are more up-front about the accuracy, possibly through colorcoding or uncertainty measurements, then most of the arguments of the detractors (who don't tend to know how to use the data in the first place) go away.

*By the way, nice responses in the OReilly article.  Especially with Apujunata coming right behind you and legitimizing your methodology by saying using VGC was basically as good as using NPD, although that's not how he probably saw it**

**Apu, since I know you sometimes show up here, Hi!  Nice seeing you around!

@jman8: Did you read what I wrote and quoted? Doesn't seem so.

It's not about disagreeing, I even agree with some of the parts (which, again you can see in my posts). It's about the fact that this article isn't coherent. It keeps contradicting itself.

If there's someone here who didn't read, it's you.