By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Gaming Discussion - ioi speaks out about ergh "VGC analysts"

When something is broken, it's not going to get fixed whilst you ignore the problem or make excuses for it. The sales numbers on this site are abominably poor, particularly around the holidays. Sorry, but that is a fact.

ioi can say that the figures are generally within 5% and therefore acceptable, but that is rarely true for the figures provided. Sure, if you look at the cumulative sales (which are regularly adjusted on this site every time the quarterly results get posted by the respective company) now that the Wii, 360 and PS3 have all sold 80million plus, the results may well fall within the 5% error ioi uses in his argument. However, if you look at the numbers on a weekly basis, VGChartz has a very poor track record. If the site was any good at forecasting, we wouldn't be seeing these large adjustments (relative to the quarterly sales). Either the sample being used is unreliable, too small to be representative or both.

ioi deserves a lot of praise for making this site the success it is with a large (and I presume growing) following but the site does need to get better at the sales tracking/forecasting. If it did, I'm sure Sony, Microsoft, Nintendo and many of the big publishers would take it more seriously and ioi would probably be able to reap greater rewards for the extra work that would be involved.



Around the Network
DonFerrari said:
Zkuq said:

Cumulative numbers will be quite a bit off if you round the numbers at every turn.

Not if you make a balance in the adjustments... some up, some down, and you can always adjust when you have more data, like they do now... but in math, its wrong to put numbers of significance above your margin of error...

Lets say you have a scale (analog) that measure up to 1 pound differences... So you would measure someone 150Lbs or 151... would be acceptable to use 150.5 +- 0.5 as well... but would be totally wrong to use 150.56738 Kg because you have no way to accertain that imprecision... but I don't know much about statistics methods and if it is acceptable to use this many significance numbers.

First, I must admit that I don't know a whole lot about statistics. That said, rounding the numbers to an arbitrary direction every time doesn't sound like a sound method, nor does it sound like a convenient one. Who could ever tell which direction to round the numbers each time? A good rule of thumb, and the only rule I know I can trust, is to round the final results only and never round any intermediate results. I suppose you could round intermediate results in some cases but it requires special care so that no accuracy is lost for the final result.



What I don't understand is why ioi is on here downplaying NPD numbers, as they have FAR more data than VGC does. I also don't remember people adding up their numbers and realizing that they were 1M overtracked on anything when compared to shipment data. And to top it off, once NPD comes out, usually we get adjustments within a week or so to at least be inline with their numbers.

It's like Pachter said, this site is freaking awesome when it comes to older data, just not the greatest with the newer data. Not saying they are always wrong with new data, of course, but there's only so much you can do without incredible amounts of data.



kowenicki said:
The only time I call out numbers is where official shipment numbers make the numbers look wrong, or some other OFFICIAL announcement does the same.

The current PS3 situation is an example of this.

You keep going on about this overtracking of the PS3.  What makes you think it's overtracked?



Torillian said:

While I understand where you're coming from Brett, I think the discrepancy between what readers think is accurate and what you'd call accurate is partially on how the numbers are reported. Is it common in statistics to report numbers to such an apparently high precision even though you know the error to be higher than that? In a scientific journal the standard is to report the estimated error in the results and to round the number to the first significant digit which with a 5% error would be probably be the third digit so 282,965 would become 283,000. Is there any particularly issue with reporting the numbers this way with the estimated error included versus what you do now?


good point.



I'm not really here!


Around the Network

I agree with him but games like God of War which has sold over 5.2 million in 2012 according to Sony PR and its 2013 and this game still hasn't sold above 4.5 million according to vgchartz and the same goes to Uncharted 3 and a few other games. And I really have respect for what ioi has done with this website its absolutely great but then again ioi gets bashed from sites like IGN for not listing their sources. And with all due respect how does he know whether IGN will be around in the next 5 years? IGN has been around a lot longer than VGC and is far more popular than VGC has even been.



thismeintiel said:
kowenicki said:
The only time I call out numbers is where official shipment numbers make the numbers look wrong, or some other OFFICIAL announcement does the same.

The current PS3 situation is an example of this.

You keep going on about this overtracking of the PS3.  What makes you think it's overtracked?

80m shipped and 80m sold at EXACTLY the same time. November, when shipments are ramped up to cater for increased demand. 

As a comparison.  360 reported 80m shipped by 15th October... apparently it took until 7th December to sell through those.

I'm sorry but that is ridiculous.

Zero supply in November for PS3 but almost 2 months supply for 360 in October?  Really?

 



I'm not really here!


Zkuq said:
DonFerrari said:
Zkuq said:

Cumulative numbers will be quite a bit off if you round the numbers at every turn.

Not if you make a balance in the adjustments... some up, some down, and you can always adjust when you have more data, like they do now... but in math, its wrong to put numbers of significance above your margin of error...

Lets say you have a scale (analog) that measure up to 1 pound differences... So you would measure someone 150Lbs or 151... would be acceptable to use 150.5 +- 0.5 as well... but would be totally wrong to use 150.56738 Kg because you have no way to accertain that imprecision... but I don't know much about statistics methods and if it is acceptable to use this many significance numbers.

First, I must admit that I don't know a whole lot about statistics. That said, rounding the numbers to an arbitrary direction every time doesn't sound like a sound method, nor does it sound like a convenient one. Who could ever tell which direction to round the numbers each time? A good rule of thumb, and the only rule I know I can trust, is to round the final results only and never round any intermediate results. I suppose you could round intermediate results in some cases but it requires special care so that no accuracy is lost for the final result.

Not arbitrary... what i'm saying is... instead of using 238,738 consoles sold W1 use 240k (added 1,262) so next week lets say it sell 132,000 so use 130k (took 2k) (inbalance of 738), and so on... that is because they are not pulling that exact number on statistic they are guessing a range lets say from 200k to 250k and by their method 238,738 is the most representative number for them so they use it...

But if they are 5 or 10% uncertain of it they shouldn't use precision to the last number because the imprecision is of 11,936 to 23,873 so using 240 or even 250k would be better description of the numbers... hope that helped. But in the end that doesn't make much difference.



duduspace11 "Well, since we are estimating costs, Pokemon Red/Blue did cost Nintendo about $50m to make back in 1996"

http://gamrconnect.vgchartz.com/post.php?id=8808363

Mr Puggsly: "Hehe, I said good profit. You said big profit. Frankly, not losing money is what I meant by good. Don't get hung up on semantics"

http://gamrconnect.vgchartz.com/post.php?id=9008994

Azzanation: "PS5 wouldn't sold out at launch without scalpers."

kowenicki said:
thismeintiel said:

You keep going on about this overtracking of the PS3.  What makes you think it's overtracked?

80m shipped and 80m sold at EXACTLY the same time. November, when shipments are ramped up to cater for increased demand. 

As a comparison.  360 reported 80m shipped by 15th October... apparently it took until 7th December to sell through those.

I'm sorry but that is ridiculous.

Zero supply in November for PS3 but almost 2 months supply for 360 in October?  Really?

 

Actually, we have it at 79.8M for the week announced over 80M shipped.  Depends on how much over 80M is, really.  If it's 80.2M-80.3M, that puts 400K-500K on shelves.  Which I agree seems a little low, unless they shipped more the following week(s) for Black Friday.  But I can see where it could be a few 100Ks overtracked.



Yakuzaice said:
Carl2291 said:

The counter to that would be pointing out the significance of the 5%.

Weekly sales? 5% of 100,000 to 300,000 units is nothing worth spilling milk over. Lifetime sales? 5% of 80 Million is a totally different prospect. Surely there has to be leway when the numbers are so large and the evidence is in front of you.

Edit - kow beat me to it.

Another aspect is that a 5% error in Worldwide sales could be a massive error in a specific region.  For example between the time they first went up and now, the rest of the world numbers for the PS4 went up by 24.5k.  That is pretty small for the overall weekly numbers, but represents a 128% increase for the region.

This. This is all I was pointing out. The 19k in "others" simply didn't make sense, and I want to know why ioi agreed in the end.