Forums - Gaming Discussion - ioi speaks out about ergh "VGC analysts"

I was just speaking with Pezus through PM about being within 50k  was reasonable and no reason to shout undertracked/overtracked on stuff like that and then i read this. 

From ioi himself. 

Oh man my head hurts reading this thread! I'd love to jump in and correct most of the idiotic comments and "analysis" in here but I don't think it is actually worth doing so since people just seem to want to believe what they want anyway and don't actually listen to logic or reason!

This post - http://gamrconnect.vgchartz.com/post.php?id=5909056 - is a good place for you all to start. After reading that, if you can articulate your concerns in a concise and factual manner I will try to reply to them as best I can...

ioi said:

To be honest I couldn't care less what IGN thinks - they won't be around in 5 years the way they are going so their opinion counts for nothing in my eyes. What does bother me is the ignorance of some people in this very thread who still don't get it!

If we put out a figure saying that a game sold 600k units in a given period then we are not saying that the game sold exactly 600,000 units in that period. It is an estimate, based on much smaller data and extrapolated up. Think of it as a probability - when we publish a figure of 600k we are saying that we are 95% certain it has sold somewhere between 300k and 900k, 85% certain of it being between 400k and 800k and 70% between 500k and 700k and so on.

Take the 600k figure as the midpoint with a decaying range of probability around it, much like a standard probability bell curve:



For people to say something like "VGChartz is wrong" makes no sense. The word "wrong" doesn't make any sense in this context? There is no right or wrong.

If we claim 600k for a game and NPD says 635k then are we "wrong"? If they say 485k are we "wrong"? Not in my eyes, we are within a reasonable and expected margin of error - we'd be in the fat part of that bell curve above - especially when you consider that NPDs data itself will carry a 10-15 error margin. This is why people who say that have overtracked something because we have 790k and NPD has 740k just annoys me! Their figure represents anything between 650k and 850k and our figure represents anything between 700k and 900k so they line up just fine. A 50k difference out of 800k is a tiny and insignificant difference.

You have to start thinking of a data estimate as being a range of possible data not just one hard, absolute figure. I can't emphasise that point enough.

Now if we said a game sold 1.2m and in fact it only sold 50k then that would clearly be outside of any reasonable margin of error and right at one of the extreme ends of that bell curve above - which would clearly be unacceptable.

I'm afraid anyone who comes in with this idea of data being "right" or "wrong" has no clue what they are talking about and has obviously never studied maths (or math as you insist on calling it in the USA) to any decent level (such as Mr Schneider for example).



Around the Network
What about the PS3 having sold more than they shipped? That would be wrong. I wouldn't call the weekly sales wrong. I just don't think of them as right either. Just a first look that is likely to be adjusted.

The only time I call out numbers is where official shipment numbers make the numbers look wrong, or some other OFFICIAL announcement does the same.

The current PS3 situation is an example of this.



I'm not really here!

televandalist Animated Gif on Giphy

Link: Shipment History Since 1995

lol you made a thread bout IOIs post?

i thought that was a given...that this site is not a sales tracker

kowenicki said:
The only time I call out numbers is where official shipment numbers make the numbers look wrong, or some other OFFICIAL announcement does the same.

The current PS3 situation is an example of this.


I think he is more frustrated with people constantly calling overtracked/undertracked every week when the data is usually within a reasonable range of accuracy.  PS3 is wrong and should be adjusted but that is more than a few thousand difference. where as this week we had people arguing over a small difference because they somehow no more through assumptions.  Some people going so far as saying it was off by 500k to 1 million.  It was one thing if people were saying it was off by 50k to 100k but people were saying much more than that as I'm sure you're aware of since i seen you in that thread as well.



Around the Network
TheEspionage said:
kowenicki said:
The only time I call out numbers is where official shipment numbers make the numbers look wrong, or some other OFFICIAL announcement does the same.

The current PS3 situation is an example of this.


I think he is more frustrated with people constantly calling overtracked/undertracked every week when the data is usually within a reasonable range of accuracy.  PS3 is wrong and should be adjusted but that is more than a few thousand difference. where as this week we had people arguing over a small difference because they somehow no more through assumptions.  Some people going so far as saying it was off by 500k to 1 million.  It was one thing if people were saying it was off by 50k to 100k but people were saying much more than that as I'm sure you're aware of since i seen you in that thread as well.

I agree with you.  Shouting over/undertracked by huge numbers with zero evidence is silly.

Their hopes may be correct, but their hopes alone arent evidence.



I'm not really here!

televandalist Animated Gif on Giphy

Link: Shipment History Since 1995

I post this time and time again.

http://www.vgchartz.com/article/82746/editorial-why-it-is-so-easy-to-blame-vgchartz/

More people need to actually read things about the site and the sites numbers.

_____________________________________

What VGChartz offers is timely data that isn't meant to be 100% accurate but be in the right range. We don't compete with the likes of NPD, GFK or ChartTrack; we offer a service that is totally different. One that is not based on comprehensive and direct retail tracking, but rather uses modern and alternative methods to quickly arrive at estimates, combined with a database of historical sales - constantly adjusted and tweaked to be as accurate as possible.

However, even with the greatest of diligence, our data is still just an estimate so whenever quoting a figure from VGChartz it should be listed as an estimate and readers should be made aware that there is a margin of error associated. With this in mind, VGChartz data is fine for most applications - from a year-on-year genre analysis to first-day estimates for a major title to a ballpark estimate of total sales to date for a given game. It just requires the user to have a little common sense and realise that an estimate is not exact but better than having no information and intended to point you in the right direction.

_____________________________________

Its really not all that difficult to understand. Constantly crying and clammering for adjustments wont get anything sorted. Its like going for a meal thats been payed for you by the restaurant yet sitting there complaining that the presentation isnt perfect and refusing to eat it until its fixed.

People need to understand that without actual solid evidence, they are pointlessly trying to get the numbers changed (most likely due to a bias). Find evidence that supports your claim, from other sales trackers or publishers/developers. Dont just say "UNDERTRACKED! Why? Because I said so!"

No wonder ioi gets annoyed and pissed off at some people.



 

Is ioi responding to stuff at the moment? I'd like to know his thoughts on including digital vouchers in the sales of Wii Party U (and NSMBU) in Japan; and whether it's going to become the norm for sales tracking from now on.

Any "error" that represents less than a 5% difference to an "official" source of data is not an "error" as far as I'm concerned - it is within the accepted range of figures that our data point represents. Also, I'd argue in principal that there is no such thing in this area of research as an actual number, however official you want to believe that it is.

In the same way, someone who feels that one game or platform "beat" another because it sold 320,000 units according to our data compared to 310,000 units for the competitor is just being idiotic given data ranges that those two figures represent.

Around the Network
Kresnik said:
Is ioi responding to stuff at the moment? I'd like to know his thoughts on including digital vouchers in the sales of Wii Party U (and NSMBU) in Japan; and whether it's going to become the norm for sales tracking from now on.


If that answers your question ^^^