By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Website Topics - Ruh-roh: Simon Carless of Gamasutra/Game Set Watch analyzes VGChartz...

Lol at the guy who said a 10-20% error is a mortal flaw in methodology. Look at Famitsu and Media Create numbers sometime.



My Mario Kart Wii friend code: 2707-1866-0957

Around the Network
theRepublic said:
shio said:
steven787 said:
Accuracy of 10 or even 20 percent error isn't a big deal when you are discussing Qualitative properties of sales. Of course publishers and retailers want more accurate data, but as far as using sales to predict future trends in gaming (what type of games sell, what console is "winning") broader estimate work just fine. I don't see what the problem is or why the established game websites are so against the idea of VGChartz.

a 10% error, let alone 20%,  is a mortal flaw in methodology. Any professional outlet that often possesses that terrible margin of error wouldn't last long in the business world.

The article didn't say much I didn't already know, and I agree completely with it.

 

To Brett Walton: Please give credit to your sources, that's the way it should be.

 

Read this post from naznatips:

"That said, all data trackers provide nothing but estimates. ALL. Famitsu and Media Create differ as much as 40% on a given week, and each one tracks more of the Japanese market than NPD tracks of the NA market. Pretending any sales data gives anything more than a basic idea of sales trends is laughable."

Famitsu and Media Create are profesional tracking services.  I guess they and the NPD are all mortally flawed by your standards.

This is not science, and it is not a labratory.  There are different standards in the real world.

 

Shio, your missing the point.  If you want more accurate data, you are (1) not going to get it and (2) setting your standards unecesarily high.  We are just here having fun, talking about general trends, etc.

Common anti-social thinking.  People are having fun, oh my god, I need to stop it.

 

VGChartz is free.

VGChartz is open about its errors and goes out of its way to compare their results to non-free trackers.

VGChartz is fun.

Next the false logic in this thread is really nagging me:

Non-free trackers aren't necesarily better.  One of the first things you learn in (a real) statistics class is that a larger sample can mess up extrapolation.



I would cite regulation, but I know you will simply ignore it.

Bah, a couple of points the article make are alright, but the tone of it is what bothers me:

-It seems that his intention is to put a bad mark on this site (to and/or for a certain group of audience) while at the same time (trying to) mask it with having balance criticisms, which turns the article into a piece of contradictory statements.
-He pretends to understand the purpose of Vgchartz, and states that he understands it, yet writes a contradict piece in his conclusion.

Basically, I hate his pretense as well as his bias.

That said though, this article brings up some interesting issues. It seems that many people, some vgchartz users included (and myself included), do not know/understand the site's purpose, or do understand it but do not know what direction we should be taking.

The core of the matter is wanting more and more people to look at vgchartz data, though. As for vgchartz itself, it has been improving, so I don't see how it has stopped improving now.

 

EDIT: Btw, why does neogaf hate us so much?



I'm an ALIEN!!!! - officially identified as by Konnichiwa

Of course... My English is still... horrible - appreciation and thanks to FJ-Warez  

Brawl FC: 0301-9911-8154

tiachopvutru said:

EDIT: Btw, why does neogaf hate us so much?

 

They're bored? They're jealous? They look down on non-GAF people? They're idiots?

 



My Mario Kart Wii friend code: 2707-1866-0957

ioi said:

@apujanata - there is no issue with constructive criticism of the site, of our numbers or even of me - I welcome and will listen to all of it and where possible offer a response. All members should feel able to speak openly and freely about any issue - that is one of the core values I build this site on. All opinions are welcome.

However, personal insults and saying that I behave like a child are another matter and go against forum rules (nevermind the fact this is my site!!) so Ben is lucky he is still here.

It's not a case of this is the real world, accept it. I have been accepting criticism and opposing views for over 3 years now on this site - I am used to it believe me. I've also said that I don't necessarily disagree with the article overall - vgchartz has a time and a place and it is upto industry people to decide whether they want to use us or not and to understand the limitations of our data. My issue comes with the tone of the article, the twisting of my words to make it look as though I said that data is just "made up" on a regular basis - which is definitely not what I said - and the way in which he disguised his intentions in our initial email conversation. As I said originally, his approach has been deeply unprofessional and all I have done is tried to clear up the factual inaccuracies and unsupported assumptions he made in his article. I don't see what is childish about that.

Well, offensive post (name calling) is unprofessional, but if you ban him from calling you once, that measure is a little bit too harsh, IMO.

You have to realize that once you gave information to journalist, it is their prerogatives to interpret them however they like. Sometime their opinion is favorable to you (like Robert P. article), but sometime it happen the other way around (unfavorable, like the Simon C. article). That is why normal people hate journalist, especially the TV ones.

I am not saying that Simon C. is in the wrong, or Robert P. is in the right, but that is the way journalist work (as far as I know). Hope this experience will make you more cautious when corresponding with journalist in the future.

BTW, I believe I hadn't congratulate you on getting VGC to it's state (big, full of activity and member). Just try to avoid making another mistake like MGS4 in the future (I believe speed is good, but too speedy can cause severe inaccuracy). Try to find a better balance between speed and accuracy, and put more information/disclaimer (like estimated) in the article itself, especially when it is for public consumption.

Good luck, and keep improving (by getting more data sample). And don't forget your promise to keep your site free of charge forever.



Around the Network

The one thing that stood out to me was, he mentioned that he got a quite impressive answer.

" I asked Brett Walton his methodology, and he gave me the following, quite impressive answer:"

Including:
"
- Sampled direct sell-through data
- plenty of statistical analysis, regression calculations, market projections"

But he didn't seem to understand the answer or refuses to, because:

"It's a reasonable figure, of course, because the VGChartz folks are smart people. But it's not a real figure. It's a educated guesstimate"

So, VGC guesstimates, but he would 'extrapolate' (ie do a fancier guessestimate)

"But if I was a writer or analyst trying to extrapolate significant information from the resource, especially regarding those titles which don't chart regularly, given the major discrepancies with other figures shown here, I would not recommend it."



Torturing the numbers.  Hear them scream.

omoneru said:
@Just_Ben

I think you really should not post here anymore.

- because you apparently don't have a minimum amount of respect to those who have been working their asses off to provide FREE ESTIMATES for YOUR fun, have maintained the community where you can hang around, have tried hard to establish some credibility among skeptics, and have trusted a journalist that presented good faith and were unfortunately deceived by him.

You should spend at least ten years in some industry that gets a lot of public attention and should learn how hard it is to combat rogue journalism like this. The last thing this site needs is your little condescending lecture on journalism.

I hope you got my point because I don't have time to dissect your marginally-interesting response with a smiley mark. Mass Effect is waiting for me.

So long.

 

And you think I don't know something like that? You don't think I don't know how it is to put a hard work and give it others for free? Well I must have dreamed the writing off my League_Bot than, supporting this site (most likley more than you will ever do) with my personal experience. I didn't got a dime for writing him and running him and maintaining him. I would never ask for something like than. Do you really think I should go? Well ask people playing in those leagues and think again, my guess is, they think diffrent.

Oh well, it is only marginally intresting I guess, off course. I would say, do something similar for this site, and I decide to get your point then.



If I were to offer constructive criticism to ioi and the team, about the only thing I would like to see would be some just some basic 90% confidence intervals for the numbers presented, or if that is not possible (I'm no statistician) then just some basic verbal indication of the margin of error based on historical performance of VGC vs. other sources. That would IMO increase the credibility of the numbers tremendously. Those who use the numbers professionally have to judge the margin of error themselves now, so I'd imagine they would love if it was done for them by those who actually compile the data in the first place.

On a completely another note, I wholeheartedly agree with ioi and his idea of what VGC is and should be. And to all who wish for increased market coverage, you have to remember that with increased coverage comes increased workload as ioi has more and more numbers to crunch and check and adjust and extrapolate, which works directly against what the site is about, i.e. speedy delivery of good-enough-for-most numbers.



This post never happend

 



@Just_Ben

Well, if you recall, I helped translate the press release into Japanese last year (Nobody on the managerial side thanked me for that. But that's just fine by me because I'm not gonna do it anymore).

But sure you're certainly more valuable to this site than I am.

I don't even post here often. I don't have an incentive because sales or console war doesn't interest me at all. I don't give a flying fuck if a game sells or not. MS,Nintendo,Sony can go bankrupt tomorrow for all I care.

I'm only marginally interested in people's impressions on particular games and business strategy in a broader sense, but there are plenty of other places for me to discuss such matters. Besides, I don't like to socialize too much with random strangers on the Internet. So never mind my prior claim about stop posting.

But your I-know-better-than-you-I-told-you-so-grow-up-kid attitude did absolutely nothing helpful to the site managers who undeservedly suffered from this PR trickery. It doesn't matter who point this out to you. If I haven't done it, somebody else would have.

If you want to warn someone about professionalism, next time try to provide something with substance - instead of throwing some pejorative terms to honest people during a hard time.



No, it's not going to stop  'Til you wise up
No, it's not going to stop  So just ... give up
- Aimee Mann