By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Gaming - Review numbers are ridiculous. I now pledge I will not use them at all.

The_vagabond7 on 13 February 2009
famousringo said:
Words Of Wisdom said:
famousringo said:
Words Of Wisdom said:
famousringo said:

IGN recently gave a 0.8 to one piece of shovelware. I'm sure few other reviewers bothered to look at it, and there are many other shovelware games which even IGN didn't want to waste paid labour and bandwidth on.

Why was it a 0.8?  Why not a 0.9 or a 0.7?  Why not a 1.3?  What's the difference between all those scores?

 

Would it be too obvious if I answered that it was because the reviewer thought it was better than a 0.7 and worse than a 0.9?

And the reason why VGC is using a "meaningless" numerical system is because they did a poll of VGC users, and a plurality of voters asked for a 100-point scale, more than doubling the votes cast for any other grading method. People want the clarity of numbers over letters, and they want enough granularity in the scale that they can distinguish between a game which barely made it to the 80s and a game which almost made it to the 90s.

Letters are no more inherently meaningful than numbers.  Choosing between them is essentially choosing between the useless mainstream scale or the useless nonmainstream scale.  Yay.

You get no clarity from either system because the actual numbers have no meaning.  There is no truly objective difference between an A and B nor a 9.1 and a 9.4.  It doesn't exist.  People argue about scores all the time because scores mean different things to different people. 

Those scales work in the educational system because I can objectively give you a 90% on a test because you got 2 out of 20 questions wrong.  Video games lack that quality (or it's just that no one has ever come up with a solid enough rating system).

 

What does objectivity have to do with it? It's a matter of ranking, not objectivity. You can deliver a list of pros and cons for two games along with an explanation of why you list those attributes as good or bad, but you won't answer the the most important question people want to know between these two games: Which one is better?

That's what the number is for, and why the number is more clear when you have more granularity in the scale. That's why people want to see 8.0 and 8.9 instead of just two 8s. Just because there's no objectivity in any of these numbers doesn't mean that knowing which game ranks better in the subjective opinion of a group or individual is worthless information. The only real problem is that people think an aggregation of subjective numbers alchemically makes those numbers objective.


The immature sense that ranking is objective is absurdist though. Why is Tales of Symphonia DOTNW objectively one point out of a hundred better than Infinite Undiscovery? Why is Killzone 2 Objectively a 92 out of a hundred? According to metacritic. There is no objecitve way to rank one game as being better than another, and no point to doing so other than to fuel fanboy wars. I like No More Heroes about a bajillion times more then Uncharted Drakes Fortune. I love Disgaea 3 waaay more than I like Fallout 3. According to Metacritic, I'm an idiot and it should be the reverse. So either I am playing the games wrong, or there is no point in ranking one better than the other because it is inherently subjective. There numbers, the rankings, the whatever you want to call them don't do anything that the body of the text can't do except incite flame wars over arbitrary numbers.



You can find me on facebook as Markus Van Rijn, if you friend me just mention you're from VGchartz and who you are here.

Around the Network
YesWiiCan on 13 February 2009

I think everyone is give all the Wii games -1 from the real score like X play just because it's on Wii.



Phoenix_Wiight on 13 February 2009

reviews were ridiculous even after last gen when some tony hawk game got a 10/10 on gamespot.

because you know, tony hawk games have everything essential to a masterpiece. an amazing storyline, stellar soundtrack, etc etc.



rubido on 13 February 2009

Edge has an article on this subject: http://www.edge-online.com/features/is-metacritic-damaging-games-industry

EDIT: Only thing I dislike about this article is that it doesn't mention a correlation between metacritic scores and reviewer scores trying to match each other. I think this is as bad as publishers going by metacritic scores to decide on what to invest in and take out the innovation from games.



famousringo on 13 February 2009
The_vagabond7 said:
famousringo said:

 

What does objectivity have to do with it? It's a matter of ranking, not objectivity. You can deliver a list of pros and cons for two games along with an explanation of why you list those attributes as good or bad, but you won't answer the the most important question people want to know between these two games: Which one is better?

That's what the number is for, and why the number is more clear when you have more granularity in the scale. That's why people want to see 8.0 and 8.9 instead of just two 8s. Just because there's no objectivity in any of these numbers doesn't mean that knowing which game ranks better in the subjective opinion of a group or individual is worthless information. The only real problem is that people think an aggregation of subjective numbers alchemically makes those numbers objective.

The immature sense that ranking is objective is absurdist though. Why is Tales of Symphonia DOTNW objectively one point out of a hundred better than Infinite Undiscovery? Why is Killzone 2 Objectively a 92 out of a hundred? According to metacritic. There is no objecitve way to rank one game as being better than another, and no point to doing so other than to fuel fanboy wars. I like No More Heroes about a bajillion times more then Uncharted Drakes Fortune. I love Disgaea 3 waaay more than I like Fallout 3. According to Metacritic, I'm an idiot and it should be the reverse. So either I am playing the games wrong, or there is no point in ranking one better than the other because it is inherently subjective. There numbers, the rankings, the whatever you want to call them don't do anything that the body of the text can't do except incite flame wars over arbitrary numbers.

Just to be clear, I am making no argument that review numbers or aggregate review numbers are in any way objective. Like I said above, there's nothing objective about review numbers or aggregate review numbers.

But I don't think that information is worthless just because it's subjective, either. Look at the number as a summary for the sake of clarity. A quick, easily digestible nugget of information on one person's opinion of a game. Some people will only accept information in small, ADD-resistant bites. So review scores are going to stay because people want them to stay.

Like I said, the only real problem is that some people think an aggregation of subjective numbers somehow makes those numbers objective.



"The worst part about these reviews is they are [subjective]--and their scores often depend on how drunk you got the media at a Street Fighter event."  — Mona Hamilton, Capcom Senior VP of Marketing
*Image indefinitely borrowed from BrainBoxLtd without his consent.

Around the Network
BenVTrigger on 13 February 2009

@ Phoenix Wiight

actually remember a 10 / 10 is perfectly reasonalbe for a tony hawk game AS LONG AS it accomplishes what is stellar for a genre. Is Tony Hawk trying to be an Epic RPG or FPS? No but if it captures an awesome skating experience it deserves a 10 / 10 just as much as say God Of War, Final Fantasy, or any other game for that matter. Remember not everyone likes every genre so you must rank it in comparison to games similar to it.



richardhutnik on 13 February 2009
rubido said:

Edge has an article on this subject: http://www.edge-online.com/features/is-metacritic-damaging-games-industry

EDIT: Only thing I dislike about this article is that it doesn't mention a correlation between metacritic scores and reviewer scores trying to match each other. I think this is as bad as publishers going by metacritic scores to decide on what to invest in and take out the innovation from games.

Ahh, that may of been the article I read, that I found out about how the industry uses it to make business decisions.

 

 



rubido on 13 February 2009
richardhutnik said:
rubido said:

Edge has an article on this subject: http://www.edge-online.com/features/is-metacritic-damaging-games-industry

EDIT: Only thing I dislike about this article is that it doesn't mention a correlation between metacritic scores and reviewer scores trying to match each other. I think this is as bad as publishers going by metacritic scores to decide on what to invest in and take out the innovation from games.

Ahh, that may of been the article I read, that I found out about how the industry uses it to make business decisions.

 

 

 

This article was published yesterday. It might have been another one.



im_sneaky on 13 February 2009
Soriku said:
Agree. I buy whatever games I want without heeding review scores (just look at ToS; DotNW scores...6s? lol not even.)

Reviews are a POS. Sometimes nice to look at, but that's it.

Your right, a 6 is way too high for that game. Numbers should stay, readers should just READ the review and understand the plus and minus.



 

 

richardhutnik on 13 February 2009
rubido said:
richardhutnik said:
rubido said:

Edge has an article on this subject: http://www.edge-online.com/features/is-metacritic-damaging-games-industry

EDIT: Only thing I dislike about this article is that it doesn't mention a correlation between metacritic scores and reviewer scores trying to match each other. I think this is as bad as publishers going by metacritic scores to decide on what to invest in and take out the innovation from games.

Ahh, that may of been the article I read, that I found out about how the industry uses it to make business decisions.

This article was published yesterday. It might have been another one.

If that is the case, then it is someone else who must have observed this also.  I recall reading it a year or two ago, likely before I found this site.