By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Website Topics - We're a month behind in the numbers and ioi has been gone for over two weeks, what gives?

Blood_Tears said:
Why doesn't somebody just email him then? He's given his email address publicly before in the forums when hiring for jobs in the past.

bwalton@vgchartz.com


You RARELY ever get a reply, trust me.



Around the Network

Yeah it's definitely been frustrating/disappointing. Some ppl are saying who cares? But this site is called vgchartz bc it's basis is sales. So it makes sense for it's users to expect sales data.



Wait.... doesn't this guy have an office and staff?

Or is it actually just one guy making up numbers.

What a joke.



Oh noez, ioi's been kidnapped by evil NeoGAF mutant ninja trolls!



Stwike him, Centuwion. Stwike him vewy wuffly! (Pontius Pilate, "Life of Brian")
A fart without stink is like a sky without stars.
TGS, Third Grade Shooter: brand new genre invented by Kevin Butler exclusively for Natal WiiToo Kinect. PEW! PEW-PEW-PEW! 
 


Jabbamk1 said:
VGChartz has been known to just make numbers up. For example. VGChartz had 15,000 sales for Dynasty Warriors 8 Empires on PS4 in the USA 2 weeks before the game released because they forgot to update the delayed release date. That shows they made the number up based on a prediction model and no actual retail tracking. Therefore VGChartz can not be compared to NPD, IDG, Gfk or Media Create who base their tracking using POS data, not predicted data or made up data.

If they have X as their preorder numbers from their direct sources, and they have Y as their sales numbers from their direct sources, they'll have some combination mX+nY as part of their prediction for total sales. And if they forget to update their release dates, then Y=0, and they end up with just mX... which won't be zero. And that is how they can have it show sales numbers prior to release, especially if it's working on only partial data (remember, I haven't accounted for things like correlations - the value of Y would affect m, and the value of X would affect n, for instance).

And of course, if they were merely guessing numbers, they wouldn't have guessed 15,000 for DW8E on PS4, given that even the PS3 version of the original (and the original always sells better than the expansions) only barely broke that number, and the PS4 version of the first expansion did significantly less. So if they were using a method of guessing, that number just doesn't make sense as a guess. But if it's coming from a statistical model combining sales sources, preorders, and trend data, then it makes more sense - very inaccurate data (in this case, their sales source data being zero) fed into the model produces nonsensical results.

Of course, it goes beyond that. They have retail sources A, B, and C, which represent some fraction of the entire retail market. But then, A, B, and C, and the other sources in the market, have different sales patterns to others. So they have to use predictive techniques to estimate what the others would be, using the information they have. And that means that they end up with correlative elements that have human influences, like judging of subgenre. If they notice that their sources do better in musou games, but worse in FPS, than other sources, then they will scale down their musou numbers and scale up their FPS numbers, relative to the regular prediction.

All of the tracking companies do this. And you claim NPD tracks 95% of the market directly. I'm going to challenge you on the assertion, since I can find no source for that claim. I would bet that the assertion comes from a misunderstanding of something NPD claimed at some point - 5% is an important number in standard statistics, and I'm betting that NPD said that their data is 95% accurate - that is, that the official numbers given to their clients (publicly leaked information, you'll notice, doesn't include this sort of information) statistically have a minimised chance of being more than 5% away from the real number... and those official numbers would include error estimates.

What I *have* been able to find is a kind of chinese whispers in values. In 2008, a Gamasutra article speculated a 60-65% retail coverage in the US by NPD, and it was clearly a speculated value, based on wording. Later mentions of NPD's coverage cite 60-65%, literally as "60-65%", without reference to it being speculation, as though it were the official number. My bet is that the 95% number has arisen in the same way - someone proposed that, with (as I recall) Walmart being added to NPD's list of sources, they "probably have something like 95% coverage"... which lost the "probably" and "something like" qualifiers when repeated.

It would be highly inefficient for NPD to collect 95% of all data. No company requires that much accuracy (with 95% of data, with sensible extrapolation, you'd get within something like 0.01% of the real number something like 95% of the time - note the uncertain figures, I'm not wasting my time working out the real numbers). They need ballpark figures (say, within 5%, or maybe within 1%), and NPD are a business, they aren't going to waste resources. So they'll collect a reasonable sample... and that can mean inaccurate numbers, sometimes.

A Sega rep has actually stated that NPD are routinely as much as 30% off in their numbers (see the 4guys1up podcast for 1/1/2010) - they didn't have Walmart at the time, but adding Walmart wouldn't have magically fixed the inaccuracies). In short, what you believe is going on, simply isn't.



Around the Network
Aielyn said:
Jabbamk1 said:
VGChartz has been known to just make numbers up. For example. VGChartz had 15,000 sales for Dynasty Warriors 8 Empires on PS4 in the USA 2 weeks before the game released because they forgot to update the delayed release date. That shows they made the number up based on a prediction model and no actual retail tracking. Therefore VGChartz can not be compared to NPD, IDG, Gfk or Media Create who base their tracking using POS data, not predicted data or made up data.

Post about NPD/VGChartz

 

 

I understand what predictive modelling is. 

 

But basing it on zero actual POS data makes the whole thing inaccurate and a bunch of guesses. At that point it becomes wildly inaccurate. 

 

Compared to NPD who use actual POS data from retailers. 

 

Your argument about NPD is wrong as well, I understand that all numbers are going to be estimates, but to say VGChartz is anywhere near as accurate as NPD is ludicrous.  Using the whole "well all numbers are estimates" argument is pointless as that's blatently obvious. But when NPD are basing their estimate from actual data direct from retailers, are quoted by every major organisation and are seen as industry leading then what you're doing by saying they must be inaccurate makes no sense. 

I can guarantee the 90-95% number is valid. 



Its ok, I'll guess the numbers for us for the week of March 6!

XB1: OVAR 99 MILLION!

PS4: More than XB1

3ds: 2,213

Wii U: 6

Everything else: 9



Jabbamk1 said:
Aielyn said:
Jabbamk1 said:
VGChartz has been known to just make numbers up. For example. VGChartz had 15,000 sales for Dynasty Warriors 8 Empires on PS4 in the USA 2 weeks before the game released because they forgot to update the delayed release date. That shows they made the number up based on a prediction model and no actual retail tracking. Therefore VGChartz can not be compared to NPD, IDG, Gfk or Media Create who base their tracking using POS data, not predicted data or made up data.

Post about NPD/VGChartz

 

 

I understand what predictive modelling is. 

 

But basing it on zero actual POS data makes the whole thing inaccurate and a bunch of guesses. At that point it becomes wildly inaccurate. 

 

Compared to NPD who use actual POS data from retailers. 

 

Your argument about NPD is wrong as well, I understand that all numbers are going to be estimates, but to say VGChartz is anywhere near as accurate as NPD is ludicrous.  Using the whole "well all numbers are estimates" argument is pointless as that's blatently obvious. But when NPD are basing their estimate from actual data direct from retailers, are quoted by every major organisation and are seen as industry leading then what you're doing by saying they must be inaccurate makes no sense. 

I can guarantee the 90-95% number is valid. 


Do you have a source for that?

What aielyn said is true, however that doesn't mean it's not a big mistake to release numbers for an unreleased game.



Jabbamk1 said:
I understand what predictive modelling is.

 

But basing it on zero actual POS data makes the whole thing inaccurate and a bunch of guesses. At that point it becomes wildly inaccurate.

 

Compared to NPD who use actual POS data from retailers.

 

Your argument about NPD is wrong as well, I understand that all numbers are going to be estimates, but to say VGChartz is anywhere near as accurate as NPD is ludicrous.  Using the whole "well all numbers are estimates" argument is pointless as that's blatently obvious. But when NPD are basing their estimate from actual data direct from retailers, are quoted by every major organisation and are seen as industry leading then what you're doing by saying they must be inaccurate makes no sense.

I can guarantee the 90-95% number is valid.

I said nothing about predictive modelling. This is statistical extrapolation, which is a different thing. VGChartz does have POS sales numbers. That POS data does not cover the entire market, however, and thus some sort of extrapolation must be done. VGChartz does this by factoring in other sources of data, including pre-order data, data from other tracking companies, and data from game companies themselves.

The result is that their numbers tend to be more accurate for more popular titles than for less popular titles - this is unsurprising, and most would consider it a minor problem at worst.

You keep making the claim that VGChartz doesn't use "actual data direct from retailers". Since VGChartz themselves say otherwise, I'm going to challenge you to provide evidence for your claim... especially since, as I pointed out, the strange numbers that happen when they forget to update release dates suggest that the absence of retail data alters the numbers. That is, the resulting numbers don't look anything like the "typical" numbers that such titles generally obtain. If they were "a bunch of guesses" when this doesn't happen, then the numbers when the date is wrong would still look just like the other guesses, and a naive prediction based on guesses would probably use past performance of titles in the same franchise to predict future performance. That this isn't what happens when the date is wrong suggests that there's a whole lot more nuance to their extrapolation than "they're just guessing".

And as Zuhyc said, regarding the "guarantee" that NPD covers 90-95% of the market... I'm going to need a source or a justification for that. After all, we all recall that big announcement when NPD started tracking Walmart. If NPD were at, say, 70% coverage, there would be no benefit to them to add Walmart to their set of sources. 70% is more than sufficient to get a proper sample and to permit highly-accurate extrapolation. That they needed to add Walmart suggests that there were some major holes in their data, and the fact that a Sega rep has explicitly said that NPD aren't as accurate as people think means that the actual coverage isn't nearly as big as people think...

So, on that point, I'm going to challenge you to put up or shut up.