By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Gaming - ioi speaks out about ergh "VGC analysts"

Ok, I just found the methodology link in the giant footer and I've been looking for something like it for a while. Call me stupid or whatever if you like. It's an ok link, but I really think you should expand it to go into more depth and include more of what we've been discussing in this thread. Assume a basic understanding of statistics, but include all of the relevant details about sampling methods. Then you can link to a separate FAQ page and probably include a link to this editorial or something similar http://www.vgchartz.com/article/82746/editorial-why-it-is-so-easy-to-blame-vgchartz/ intended for some of the stupider questions.

I really think including the Methodology page more prominently in the site design would not be overkill and would help a lot, since it's an important element of the site. I would actually include it somewhere in the header, but simply bolding the link at the bottom of the charts would make it more noticeable.



Around the Network
MaskedBandit2 said:
ioi said:
MaskedBandit2 said:

You cannot take such a tiny portion of a market and balloon it out to a precise and exact number to the nearest unit and rank them.

Let me put the ball in your court then. Tell me how you would do it. Here is some hypothetical raw data for three games:

Game 1

Week 1 - 984
Week 2 - 411
Week 3 - 219



Game 2

Week 1 - 477
Week 2 - 341
Week 3 - 356

 

Game 3

Week 1 - 147
Week 2 - 115
Week 3 - 99

Now assume that the data comes from a sample that represents 0.5% of the total population. Can you demonstrate to me how you would present the weekly chart for those 3 games for those 3 weeks in a way that you feel would make you happy.

This is really tough to say because of how small the sample is.  On one hand, something like game 1 shows a large, clear decline, and so a general estimation showing a decline I think would be fine.  Just for easy calculations for show, scaled by 200, for game 1, week 1 published as ~200k, week 2 published as ~80k, and week 3 published as ~40k.  The numbers imply a reasonable level of doubt while still giving actual information about the current state of the title.  Game 2 on the other hand, the difference between week 2 and week 3 is so small, I have to wonder if there is just some statisical noise involved.  For me, I think reporting both as ~70k would be a better bet than trying to discern a trend between the weeks. 

I don't think it would be wise to publish the actual starting data, but that could also help show where the published numbers are coming from to also allow the user to make reasonable judgment about the publishings as well. 

Like Torillian stated, reporting the 95% CI in addition could also help solve the problem of misrepresentation and show that there is a range of values to consider because of the small starting data. 

And of course, the origin of the starting data needs to be explained, as I mentioned before.  Not only is it small, but does is it actually representative of the whole market?  Where and how is this data being gathered? 

Just let it go already. You should just assume the numbers aren't %100 accurate,end of story. No tracking are rating service is %100 accurate/perfect. ioi can run this site however he wants. He's not obligated to explain anything 2 you. If your such an expert and think you can do a better job of tracking numbers,then why not come up with your own website. You should just be thankful this is a free service. If you have vgchartz pro and your paying for these numbers,then all of your points and questions are %100 valid. These numbers aren't misleading,because %99 of people are smart enough to know there not %100 accurate. There's no possible way they can be,unless you have a person tracking every sale,in every store in the world. 



Lawlight said:


I reason neogaf takes issue with vgchartz is that vgchartz numbers can be way off sometimes. The 3DS being overtracked by 0.4M in Nov in the US in a recent example. Pokemon selling 5.5M in 1 day when Nintendo shipped 4M in 2 days is another.

And years ago a developer (forgot which one) said that people shouldn't take VGC seriously because VGC numbers of their games were way off. I want to say it was Factor 5/Free Radical or something. That was the begining of when I started to stop taking SW sales to mean anything.



Ponyless said:
Lawlight said:
ioi said:

The only reason there has ever been any questions raised over VGChartz numbers is because of the relationship we have with GAF and the misinformation that they spread back in 2005 - 2007 when VGChartz was first launched. This is a personal issue that stems from the fact that I used to post on GAF and fell out with some of the users on there when I pointed out that a lot of the existing data was flawed and wanted to do a better job by starting VGChartz. Had none of that happened and VGChartz just came out of the blue then none of this would exist. The backlash is a direct result of the enormous rise in popularity that the site had when it first launched since it offered something that was truly unique and filled a gap in the market. The users of GAF and other related sites couldn't get their head around it.

When a representative from Nielson, Bloomberg etc gets in touch to work with us they don't bring up any issues and treat us as they treat any other data partner. It is only on the realms of internet forums, N4G etc that these issues seem to exist. Believe me, I have spent a number of years trying to explain and defend the site but at the end of the day there is no point. Each week our latest chart gets posted on at least 50 different forums, we get more than 4 million unique users each month and have more than 30 professional data partners who subscribe to and use our data - I don't have the time or indeed the inclination to keep trying to convince people who are too narrow-minded to understand and will probably never get it! Things are going just fine as it is.


I reason neogaf takes issue with vgchartz is that vgchartz numbers can be way off sometimes. The 3DS being overtracked by 0.4M in Nov in the US in a recent example. Pokemon selling 5.5M in 1 day when Nintendo shipped 4M in 2 days is another.

Considering VGChartz has been banned before 2008, there must be some really good fortune tellers over there since they were able to predict that we would overtrack the 3DS 5 -6 years from now.



I would think that the numbers were way worse back then.

Zod95 said:
Ponyless said:
Zod95 said:
Ponyless said:

Considering VGChartz has been banned before 2008, there must be some really good fortune tellers over there since they were able to predict that we would overtrack the 3DS 5 -6 years from now.

I guess those were just recent examples. I think the main point was "vgchartz numbers can be way off sometimes". But what do you mean by "VGChartz has been banned before 2008"?

Yea i know, i was just joking. Anyways, the earliest mention i found of Vgchartz being banned on Gaf was Jan 2009, and the person said it has been banned there for years, so i took it as before 2008

No, I was asking about VGChartz being banned. What does it actually mean? How can a site be banned by other site?

You cannot use vgchartz as a source for arguments on neogaf. If you make a thread using vgchartz as a source you get banned.



Around the Network
Lawlight said:
Ponyless said:

Considering VGChartz has been banned before 2008, there must be some really good fortune tellers over there since they were able to predict that we would overtrack the 3DS 5 -6 years from now.



I would think that the numbers were way worse back then.


Its got nothing to do with the numbers. Vgchartz has been banned on GAF since its inception. This is because of a fallout between ioi and GAF.

Please stop making as if this has something to do with the numbers...



Intel Core i7 3770K [3.5GHz]|MSI Big Bang Z77 Mpower|Corsair Vengeance DDR3-1866 2 x 4GB|MSI GeForce GTX 560 ti Twin Frozr 2|OCZ Vertex 4 128GB|Corsair HX750|Cooler Master CM 690II Advanced|

I'm behind ioi on this one.

The only thing that Vgchartz perhaps does wrong is not make it absolutely clear what the numbers really represent. Users need to find certain info pages to get this detail. But really the site creators probably didnt want to clutter the weekly sales charts with messages and disclaimers etc. when all that info is in the about us page and the methodology page. There has also been numerous amounts of posts from ioi over the years giving explanations.

For me personally i always had an idea of what Vgchartz was trying to achieve with the site. Not once did i treat the numbers as absolute fact. And over the years i became to understand this more clearly.

Thats why even when the numbers are way off...i don't really see that as a major issue anymore.



Intel Core i7 3770K [3.5GHz]|MSI Big Bang Z77 Mpower|Corsair Vengeance DDR3-1866 2 x 4GB|MSI GeForce GTX 560 ti Twin Frozr 2|OCZ Vertex 4 128GB|Corsair HX750|Cooler Master CM 690II Advanced|

Shinobi-san said:
Lawlight said:
Ponyless said:

Considering VGChartz has been banned before 2008, there must be some really good fortune tellers over there since they were able to predict that we would overtrack the 3DS 5 -6 years from now.



I would think that the numbers were way worse back then.


Its got nothing to do with the numbers. Vgchartz has been banned on GAF since its inception. This is because of a fallout between ioi and GAF.

Please stop making as if this has something to do with the numbers...


Well, he did get banned from neogaf for a good reason. And the numbers provided by VGChartz don't do anything to help it's credibility over there. We see that everytime there's an NPD topic.

Also, next time someone tells me that official numbers are shipped and not sold and we should therefore use vgchartz's numbers instead, I'll post this:

http://www.neogaf.com/forum/showpost.php?p=3963064&postcount=53



Shinobi-san said:

Its got nothing to do with the numbers. Vgchartz has been banned on GAF since its inception. This is because of a fallout between ioi and GAF.

Please stop making as if this has something to do with the numbers...

I might be misremembering because it has been years, but didn't ioi start out posting his estimates on GAF before making VGC?  It definitely has to do with the numbers.

It does seem like certainty in the numbers has decayed over the years.  4-5 years ago the official line seemed to always be that the numbers were within 10% of NPD and the like.



Lawlight said:
Shinobi-san said:


Its got nothing to do with the numbers. Vgchartz has been banned on GAF since its inception. This is because of a fallout between ioi and GAF.

Please stop making as if this has something to do with the numbers...


Well, he did get banned from neogaf for a good reason. And the numbers provided by VGChartz don't do anything to help it's credibility over there. We see that everytime there's an NPD topic.

Also, next time someone tells me that official numbers are shipped and not sold and we should therefore use vgchartz's numbers instead, I'll post this:

http://www.neogaf.com/forum/showpost.php?p=3963064&postcount=53

Yes he did get banned from GAF. However what you said initially was incorrect...saying that the reason why GAF banned any mention of VGC was because its numbers wernt correct. Thats not really true. Its because ioi is the site founder. Like ioi said if Vgc popped up out of the blue, i doubt it would be completely banned from GAF. It would be accepted for what it is.  GAF and ioi did have some sort of falling out to the point where it became personal. That is why, Vgc is banned from GAF. It doesnt really matter if our numbers were more correct...it would still be banned from GAF.

Also when you say "the numbers provided by Vgcharts dont do anything to help its credibility over there" then i think you also kinda miss the point of what vgchartz is. You would also see why its completely irrational for Vgc to be banned entirely from another site.



Intel Core i7 3770K [3.5GHz]|MSI Big Bang Z77 Mpower|Corsair Vengeance DDR3-1866 2 x 4GB|MSI GeForce GTX 560 ti Twin Frozr 2|OCZ Vertex 4 128GB|Corsair HX750|Cooler Master CM 690II Advanced|