By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Website Topics - Ruh-roh: Simon Carless of Gamasutra/Game Set Watch analyzes VGChartz...

TheSource said:

I was going to do an editorial on all this but I think this works better:

We get sample data every week for the Americas, if it weren't classified I could post it from last November through last week which is the period for when I've seen it. Its just not enough store variation for some software so there has to be a concerted effort to look at shipment info, how similar games performed on hw at a given time, etc. The real advantage NPD has over us isn't the larger sample size, so much as it is the diversified sample. But still, it is just a larger sample size, and since it covers the majority of the market, but not all of it, and tends to be conservative, we aim to put sell-through figures closer to shipments. NPD isn't perfect, whenever I talk to some of my contacts I hear stories about how NPD changes figures after public release, and spreads out the changes over months, supposedly conflicts between NPD and Microsoft internal figures happened for months on end at Microsoft.

We're not taking other people's figures and predictions since those are flawed and subjective. We have a sample, we weight it for the data that is provided. Most games perform in a similar pattern in spiraling down from week one sales in the sample data, and that is reflected on the site. Adjusting to NPD figures would be silly given that they do not cover the entire market, but if the pattern is different from what we have there is no reason not to recognize it as reflective of 60% of the market in the cases where we have limited data. Its not like we do (Pachter x 2)+(NPD x 10) + (Sample x 3) and divide by 15 to get data or something. Frankly when we have data that is diametrically opposed to what NPD reports we ignore it, when the data is similar we have no reason to change it, it when a game falls in between those two ends because our sample wasn't diversified enough for propper scaling then we go back and weight the sample information differently. That isn't copying NPD, thats us being modest and recognizing that our scaling factors don't always hold steadfast. So if No More Heroes or something sold better nationally than in our sample, it just means we're scaling up the sample, we don't copy NPD, we just use it as a better idea for weighting tough to track games closer to shipments.

If NPD didn't report data we would still be able to provide weekly data from using our sample by constantly looking at the number of stores in a given market, comparing our coverage to that figure, and making the necessary adjustments to weights based on shipment data.

 

 

^ There's far more information here about VGChartz's method, than information publically available anywhere about NPD. How's that for transparency?

 



My Mario Kart Wii friend code: 2707-1866-0957

Around the Network
ioi said:
I didn't say anything I didn't mean? I said he had no permission to reproduce anything without my consent and now he is blatantly lying and saying i gave him permission.

Same with most of his article which is based on lies and misinformation.

You said you would appreciate a copy.  To a journalist.  You can delete my comments in this thread....but all I'm saying is that people who write for a living get data however they can (just like you)...and the lines between legal and illegal or appropriate/irresponsible get blurred.  I think it's best to use this experience as a lesson learned and move on.  It's just Gamasutra.

 



Someone took your quotes out of context and made you look bad ??? Well what goes aroud comes around...

 



 

 

 

El Duderino said:

Someone took your quotes out of context and made you look bad ??? Well what goes aroud comes around...

 

YMMD ;)

 



El Duderino said:

Someone took your quotes out of context and made you look bad ??? Well what goes aroud comes around...

 

The Dude no longer Abides...

RIP.

 



Around the Network

To be fair, Mr. Carless isn't trying to argue VGC to the masses of the mainstream...He's talking to developers and publishers.

The fact that VGC can be so blatantly wrong at times is not a very good thing. It might not do good for your companies stock if the Wall Street Journal quoted VGC for a major game flopping, selling 300,000 units first week as a big-name title, have a stockholder reaction, all of the while your actual numbers were much higher, and better.

The market that VGC is in isn't to be taken lightly. It's a multi-billion dollar industry. There's a reason that NPD charges what they do - they have to hire actual, real industry analysts that cost money.

VGC is a great site, but they/we need to invest more money, and resources into increasing the actual base for numbers that are obtained. Gathering 2-3% of American retail stores is NOT enough to build a worthwhile model. It's been proven every month that VGC usually has quite a few software titles off by 20%+, and it has not got better.

It can get better, but again, it' not going to come from who works at VGC, it will come via a bigger sample size. Even with my XBLA charts, my sample size is around 20% of the XBLA market, and I can still be wrong (and far too often for what I'd prefer).

I agree his wording isn't the greatest, but again...If your a shareholder of a major gaming company that has stock...And wander onto the site to see that your big new game has inappropriately flopped, it can cause investor panic, or issues. Simon is an industry professional. Regardless of your opinions, the fact is, VGC is looked down on because it's inaccurate, and always has been. This isn't Brett's fault per-se. He is doing the best he can with quality models of extrapolation, but when your sample size is so small (and very focused to a certain type of retail store), then it's not easy to get a good model of prediction,and accuracy.



Back from the dead, I'm afraid.

What's YMDD?

Anyhoo, since you've been somewhat more transparent with the data now (Telling us how you derived numbers), it's nice to know how you came at the conclusion for the MGS4 first day sales, but I think that a disclaimer at the bottom of a post (news or otherwise) would greatly help people who don't frequent the site, and don't know about your extrapolation methods, not call the site false, or estimating, etc.



ioi said:
"If the Wall Street Journal quoted VGC for a major game flopping, selling 300,000 units first week as a big-name title, have a stockholder reaction, all of the while your actual numbers were much higher, and better."

When has that ever happened?

I think he was moreso saying if it ever happened.



mrstickball,

I have to say, working in an industry that's a lot bigger than the games industry, that sometimes you make decisions based on estimated guesses. I wish I had something like VGChartz in what I do. That wouldn't mean I wouldn't hire some high-payed consultants anymore, but it would mean I could do more myself AND have a better view on what the consultants are doing for their money.
Sometimes an investment decision has to be made in a very short time, so you don't even have the time to wait on 3rd party figures.

Now, I'm not saying that VGChartz should be the main vg data supplier for the vg industry, but you can believe me that a lot of people in other businesses would often be happy with a free market analysis of 3% coverage.



DMeisterJ said:
What's YMDD?

Anyhoo, since you've been somewhat more transparent with the data now (Telling us how you derived numbers), it's nice to know how you came at the conclusion for the MGS4 first day sales, but I think that a disclaimer at the bottom of a post (news or otherwise) would greatly help people who don't frequent the site, and don't know about your extrapolation methods, not call the site false, or estimating, etc.

YMMD means You made my day, and he did. Men ioi can be a child sometimes and not even can get a joke on his expense.

(A side note to the guy banning arround here today, I know right now the ban button under my post is looking really tempting, at least breath deep before hitting ;))