By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Gaming Discussion - ioi speaks out about ergh "VGC analysts"

Carl2291 said:

 

Its really not all that difficult to understand. Constantly crying and clammering for adjustments wont get anything sorted. Its like going for a meal thats been payed for you by the restaurant yet sitting there complaining that the presentation isnt perfect and refusing to eat it until its fixed.

People need to understand that without actual solid evidence, they are pointlessly trying to get the numbers changed (most likely due to a bias). Find evidence that supports your claim, from other sales trackers or publishers/developers. Dont just say "UNDERTRACKED! Why? Because I said so!"

No wonder ioi gets annoyed and pissed off at some people.

I said something similar yesterday but you said it better lol http://gamrconnect.vgchartz.com/post.php?id=5923529



Around the Network

Everyone is giving factual evidence, don't know what you are talking about.



Nintendo and PC gamer

ioi said:
Any "error" that represents less than a 5% difference to an "official" source of data is not an "error" as far as I'm concerned - it is within the accepted range of figures that our data point represents. Also, I'd argue in principal that there is no such thing in this area of research as an actual number, however official you want to believe that it is.

In the same way, someone who feels that one game or platform "beat" another because it sold 320,000 units according to our data compared to 310,000 units for the competitor is just being idiotic given data ranges that those two figures represent.


I agree with what you say and I agree with Carl's post.  But thats doesnt alter some obvious facts.

5% on 100k is nothing as a tracking error.

5% on 80m is huge.

You simply cant hide behind statistical error on huge numbers.

A glaring example would be more sold through than shipped (or the same)... or ridiculous supply.  These are errors.  They must be.

A shipped number is official as it gets.  Its an accounting procedure that is audited independently, to report falsely would be accounting fraud.

Saying it is within statistical limits and ignoring the fact that it is an impossibility that certain numbers are correct is crazy imo.  That being said, just lumping a number somewhere to make the numers look right is also problematic, I see that too.



I'm not really here!


ioi said:
Any "error" that represents less than a 5% difference to an "official" source of data is not an "error" as far as I'm concerned - it is within the accepted range of figures that our data point represents.


The counter to that would be pointing out the significance of the 5%.

Weekly sales? 5% of 100,000 to 300,000 units is nothing worth spilling milk over. Lifetime sales? 5% of 80 Million is a totally different prospect. Surely there has to be leway when the numbers are so large and the evidence is in front of you.

Edit - kow beat me to it.



                            

kowenicki said:

I agree with what you say and I agree with Carl's post.  But thats doesnt alter some obvious facts.

5% on 100k is nothing as a tracking error.

5% on 80m is huge.

You simply cant hide behind statistical error on huge numbers.

A glaring example would be more sold through than shipped (or the same)... or ridiculous supply.  These are errors.  They must be.

A shipped number is official as it gets.  Its an accounting procedure that is audited independently, to report falsely would be accounting fraud.

Saying it is within statistical limits and ignoring the fact that it is an impossibility that certain numbers are correct is crazy imo.  That being said, just lumping a number somewhere to make the numers look right is also problematic, I see that too.


Again it all depends on what your hoping to get out of it.

If someone casually said to me "how many PS3s have been sold worldwide" and I said "about 80 million", do you think they'd care if that figure was actually 78.7m or 80.9m or whatever? Does it really make the machine any more or less successful either way or change your perception or analysis of how well it has done? It is just nitpicking to sit here arguing over a few hundred thousand sales when in reality nobody knows exactly how many have been sold.

To answer you and Carl directly - our lifetime hardware data is obviously a special case as we get shipment data and can make regular adjustments to our formulas (to represent changing trends as buying habits change over time) and correct older data to stay within a certain range of the shipment data. There is obviously increased emphasis on the lifetime hardware data as it means so much to everyone in the "console wars" and as such it is regularly adjusted to be as accurate as possible but if shipment data wasn't released then we would probably be looking a +/-5% range - which would be acceptable for most normal uses. Let's be honest here - the only reason people care is because the PS3 and 360 sales are quite close, if there was a 20-30m gap then nobody would be moaning about 200k here and there

As for official shipment data - none of us know exact stock levels, exact date ranges used and so on. Manufacturers have been known to play tricks in the past with their numbers to make them look better in one way or another so even these figures shouldn't be taken as gospel. I try to use shipment data as a guide to make sure we are in the right area but with stock being anything from 100k to maybe 2-3m at different points in time and with discrepancies in actual dates used etc then the shipment data is only useful to a certain point. With all of that said - what is the specific PS3 situation that you are refering to, I will take a look into it.

To make a general point - we don't just go in and start changing numbers to match someone else. I don't trust what some "insider" posts on Twitter and neither should anyone else. Similarly, most news sites that publish or leak data usually have no idea what they are actually talking about and often mis-report figures or don't compare the correct time ranges and so on. We have our own data, our own contacts, sources, insiders and I prefer to trust our info as I know where it is coming from and exactly what it represents.

It is quite infuriating when someone insists that our data is wrong because so-and-so said such-and-such when we have 100 times as much information on the subject than they ever will!



Around the Network
Carl2291 said:

The counter to that would be pointing out the significance of the 5%.

Weekly sales? 5% of 100,000 to 300,000 units is nothing worth spilling milk over. Lifetime sales? 5% of 80 Million is a totally different prospect. Surely there has to be leway when the numbers are so large and the evidence is in front of you.

Edit - kow beat me to it.

Another aspect is that a 5% error in Worldwide sales could be a massive error in a specific region.  For example between the time they first went up and now, the rest of the world numbers for the PS4 went up by 24.5k.  That is pretty small for the overall weekly numbers, but represents a 128% increase for the region.



I thought it was already established that VGC numbers were estimates and not an absolute number from every retailer on Earth.



    I was surprised to see ioi's comment earlier.. I usually miss all of his posts here. :/



                
       ---Member of the official Squeezol Fanclub---

Figgycal said:

I thought it was already established that VGC numbers were estimates and not an absolute number from every retailer on Earth.

The problem is, some people seem to think that NPD, or Chart-Track, or Media Create, or Famitsu, etc, are an absolute number from every retailer within their respective regions, rather than using the exact same basic methodology as VGChartz (although certain finer details, especially the specific sources, would vary).

 

That being said, ioi's comment about "within 5%" is a little concerning, if my memory of statistics is accurate (I'm a pure mathematician, always had some trouble with statistics) - if you have estimated data with a 5% margin of error, and then repeat the "experiment" 100 times, then the margin of error of the final estimate should decrease to far less than 5%. In other words, sales numbers after 2 years should have significantly less than a 5% margin of error if each week's sales number has a margin of error of 5%. In real world context, I'd expect it to reduce less significantly, maybe to 2-3%, given that magnitudes decrease with time (so only the first few data points significantly affect the sum).

What annoys me, though, is people trying to assert that NPD numbers are "right" because they're what publishers use... ignoring the fact that they're the one publishers use because people at NPD have convinced the publishers to use them, just like any other service.



Carl2291 said:

I post this time and time again.

http://www.vgchartz.com/article/82746/editorial-why-it-is-so-easy-to-blame-vgchartz/

More people need to actually read things about the site and the sites numbers.

_____________________________________

What VGChartz offers is timely data that isn't meant to be 100% accurate but be in the right range. We don't compete with the likes of NPD, GFK or ChartTrack; we offer a service that is totally different. One that is not based on comprehensive and direct retail tracking, but rather uses modern and alternative methods to quickly arrive at estimates, combined with a database of historical sales - constantly adjusted and tweaked to be as accurate as possible.

 

In the past representatives of VGC claimed to work with retailers and have actual access to real life store sales samples, but I wander what happened since then.

My guess is it took too much time to keep those relations active and instead they left the strategy on trying to have their own hard data and instead decided to solely rely on all these alternative methods to estimate sales data (polling, leaderboards, surveys, past sales data, official shipments etc), which IMO works quite well on a consistant basis.