ioi speaks out about ergh "VGC analysts"

[DonFerrari]

Zod95

Ferrari said:

I can relate to the ranking being really incoerent even if we had just 5% of error, when accumulating the error for all items in the ranking it can get misplaced, but is that what really bothers you? And as brett put their top 10 were really close to NPD.

If the error was just about 5%, between VG Chartz numbers and reality, each game of the top 10 couldn't move more than 2 places. That's already a ranking I can look at without thinking "this could be totally different" (as the one we have now that the 1st could very well be 9th and the 9th could very well be 1st). When I say "could very well be", I'm estimating such scenario and similar ones to account for 30% of the cases. Moreover, if the error was just about 5%, I could look at a recent number without thinking "this could be half...or 3 times more than that" (and in this case I'm never out of the 95% confidence range, so I'm not considering extreme cases, as you tried to make it seem). Instead, it would be "this could be a little less (95%) or a little more (105%)". See the difference? It's a hell of a difference.

With this I don't want to criticize the numbers or the methodology used. I think VG Chartz does a good and unique job and I understand that it comes to a point where shortening the confidence ranges can only be achieved by using bigger samples and ultimately this is a Accuracy-vs-Investment trade-off that needs to be balanced. The problem is not there and I can praise VG Chartz's methods and positioning as many times as needed for you to understand that in most of the cases I'm on VG Chartz's side when the rest of the videogaming industry despises it. The problem is in communication, since VG Chartz misleads newcomers (including relevant videogaming stakeholders). Some will dig in until they find how VG Chartz really works and what can they expect from the numbers they see, others won't. VG Chartz still has to learn how to pass a complex message in a simple and accessible way, and there are various alternatives for doing so. Most of them could even be subliminar messages (including the way the numbers are presented) while others could be a simple tutorial eventually with a FAQ linked to a big area at the head of every page saying "Where the numbers come from" (rather than a tiny link at the footer saying "About Us" and that presents VG Chartz's history...somewhere in the middle should be a little and incomplete explanation about the numbers).

What does bother me? This:

1 - People's stubbornness to acknowledge VG Chartz's limitations. What it really does, what it doesn't. What can be actually expect from the numbers, what we should be careful at.

2 - The way VG Chartz presents its results and the lack of willingness to make all the (or at least the major) videogaming stakeholders to understand VG Chartz's value-added contributions towards the industry.

3 - The combination of point 1 and 2 generate rivalry between the sites (like it's happening between IGN and VG Chartz) both at administration and community levels, when it should be the opposite (sites should help each other). After all, we are on the internet and, as the very name tells, it's a net, not a place full of fortifications.

4 - People willing to give insights about VG Chartz being mistreated by the owner of the snd that reaction being further support. ther users. I don't care if there was incorrect behaviour from the other side in the first place. Flamebait shouldn't be fought with flamebait and the site admin should lead by example.

Yep I undersTand and agree with most of your points. And clearness is always Better.