By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Gaming - Review Inflation

The main factors I believe are (in no particular order):

1 - commercialization of the industry with influence of publishers/hype on reviewers
2 - lack of real clarity for reviewing principles in videogame industry vs say film (with films, no matter how much something like Transformers 2 makes, it's still gonna get ripped critically)
3 - heavy focus on genre titles results in higher number of good games that aren't innovative but are decent as they are built on readily available solid tech and use a 'paint by numbers' to put in accepted gameplay mechanics
4 - rising cost of development means less stinkers as those that publish stinkers will lose money and exit the game (this is why Wii sees more shovelware I believe, as well as it's higher install base)
5 - for PS3/360 PSN and Live offer a less reviewed and obvious route for smaller titles, some of which are very good, others of which are average at best

Still, there are always the wacky outlier reviews keeping things interesting - and infuriating those who track metacritic averages as the true sign of a game's worth.




Try to be reasonable... its easier than you think...

Around the Network
r505Matt said:
forest-spirit said:
It's because of the many flaws (imo) in the scoring system that I simply can't trust scores and the only good information I get is from reading the review. Drop the scores and force people to actually read the reviews (yes, I'm that evil...)


Other people are saying this too, and it's not that I disagree, but I don't agree either. Unfortunately, to rely on a critic's score, you need to know the critic well. Observe how he/she rates other games. Just looking at a number will not help at all.

Metacritic stuff is just awful, don't even look at that.

My suggestion, find a few critics you trust, and really observe/analyze how they rate, what they think of as good or bad in a game. Some things will be small quirky things, but maybe you have the same small quirky likes/dislikes.

I agree with that suggestion.



TWRoO said:
I was diheartened when the community here chose the 100 point scale for this website's reviews.... although the poll did I think have 2 different ones that are essentially the same scoring system (0.0, 0.1, 0.3 through to 9.9, 10.0.... and out of 100 are the same thing)

Out of 5 or less if a score is somehow vitally needed.

I like this method also, and only in single digit increments.

5=Amazing
4=Great
3=Good/Average
2=Bad
1=Unplayable

This allows the reviewer to express themselves through the written portion of the review and not use the numbering as a crutch, at least it should help in theory. This way also, 5 doesn't have to mean perfect. It just describes the cream of the crop, or upper echelon of games.



Tag: Became a freaking mod and a complete douche, coincidentally, at the same time.



Onyxmeth said:
TWRoO said:
I was diheartened when the community here chose the 100 point scale for this website's reviews.... although the poll did I think have 2 different ones that are essentially the same scoring system (0.0, 0.1, 0.3 through to 9.9, 10.0.... and out of 100 are the same thing)

Out of 5 or less if a score is somehow vitally needed.

I like this method also, and only in single digit increments.

5=Amazing
4=Great
3=Good/Average
2=Bad
1=Unplayable

This allows the reviewer to express themselves through the written portion of the review and not use the numbering as a crutch, at least it should help in theory. This way also, 5 doesn't have to mean perfect. It just describes the cream of the crop, or upper echelon of games.

I quite like the EGSU method my old school used for grading classwork too.

E = Excellent
G = Good
S = Satisfactory
U = Unsatisfactory

(for games an S should be given for games that are worthy for big fans of that kind of thing, so for instance I myself will be buying F1 2009 for the Wii even if it scores as low as 50%, which in most current review scores seems to equate to terrible... of course for international language reasons it's probably best to use 1-4 rather than letters)

Or I do think a 7 tier score works ok as well:

Outstanding
Excellent
Very Good
Good
Average
Poor
Abyssimal


Whichever way a scoring system is used though, it should encourage the viewer to actually become a reader and check out the review content, because it could always be the case that the reviewer scored low or high because he has a different tolerance of certain flaws... perhaps a bad frame-rate would ruin a game for you but the reviwer doesn't mind as long as the stills are pretty.

Of course in some cases reading the review often leads most people to believe the reviewer is a total wanker... perhaps this is why review inflation is happening? reviewers are simply scoring high because people rarely question high scores so they are not encouraged to read the total drivel that the reviewer wrote?



Dropping a scoring system could also be very negative. A lot of gamers get interested in a game if they see that it is getting high scores.

Something we saw with the last Need for speed game. Saw a lot of comments (not especially on VGCHARTZ) that people were not interesting in the game because the one before Shift sucked but when Shift got some scores and quite high people got interested in it.



 

Around the Network
Onyxmeth said:
TWRoO said:
I was diheartened when the community here chose the 100 point scale for this website's reviews.... although the poll did I think have 2 different ones that are essentially the same scoring system (0.0, 0.1, 0.3 through to 9.9, 10.0.... and out of 100 are the same thing)

Out of 5 or less if a score is somehow vitally needed.

I like this method also, and only in single digit increments.

5=Amazing
4=Great
3=Good/Average
2=Bad
1=Unplayable

This allows the reviewer to express themselves through the written portion of the review and not use the numbering as a crutch, at least it should help in theory. This way also, 5 doesn't have to mean perfect. It just describes the cream of the crop, or upper echelon of games.


A clear cut 1-5 rating is both good and bad. The good, it's clear cut, so it's a very stable way to review something. You can also have as much range as you want (3.45 if you really need it) so thats not an issue either.

The bad thing though, is that if you actually use only signel digits, well, that's kind of silly. To have only 5 degrees is VERY limiting. If you made a list of 20 games you considered as 5s, I'd bet that you would be able to pick out games in that list that are better/worse than others within that list. So they aren't all the SAME 5, they're all different 5s, which makes it VERY hard to trust that sort of system.

The bigger issue though is that it's TOO concrete. Reviewing is all relative, and that should be reflected in the system as well. To say the quality of average games today is the same as the quality of average games 15 years ago is silly. That's why reviews are raising up, since we all remember those games (those of us that aren't new to gaming, or incredibly young), and that influences reviews mildly enough to need the wiggle room that a non-concrete system gives.

 



A key factor in perceived review inflation is that the worst games rarely get reviewed at all, while the best games get reviewed by everybody.

Of course the average looks high when the bottom quintile falls into a black hole and the second lowest quintile gets locked in a cellar.



"The worst part about these reviews is they are [subjective]--and their scores often depend on how drunk you got the media at a Street Fighter event."  — Mona Hamilton, Capcom Senior VP of Marketing
*Image indefinitely borrowed from BrainBoxLtd without his consent.

Onyxmeth said:
TWRoO said:
I was diheartened when the community here chose the 100 point scale for this website's reviews.... although the poll did I think have 2 different ones that are essentially the same scoring system (0.0, 0.1, 0.3 through to 9.9, 10.0.... and out of 100 are the same thing)

Out of 5 or less if a score is somehow vitally needed.

I like this method also, and only in single digit increments.

5=Amazing
4=Great
3=Good/Average
2=Bad
1=Unplayable

This allows the reviewer to express themselves through the written portion of the review and not use the numbering as a crutch, at least it should help in theory. This way also, 5 doesn't have to mean perfect. It just describes the cream of the crop, or upper echelon of games.

I agree.  The more the score can be fine grained the more difficult it becomes to be consistent, and you get all the 'how could that be 4.6 if that's 4.2' type arguements.

I hate the score mentality but understand its growth and focus today - but really, something is either:

a - absolutely amazing and top notch

b - very good

c - good

d - average

e - below average

And so a 1, 2, 3, 4 or 5 works.  The text then should make the difference.  I'd prefer stars to scores because scores imply numbers and add the temptation to fine grain them (the "if we have 4 then why not 4.2" syndrome) and letters are too algined to A, B, C with C feeling poor immediately.  Any great game is four stars, the tiny percentage of amazing titles get five stars, three stars means good if you like that genre and anything below three stars is average to bad (unless you really love the genre or feel forgiving).

 

 



Try to be reasonable... its easier than you think...

Lostplanet22 said:
Dropping a scoring system could also be very negative. A lot of gamers get interested in a game if they see that it is getting high scores.

Something we saw with the last Need for speed game. Saw a lot of comments (not especially on VGCHARTZ) that people were not interesting in the game because the one before Shift sucked but when Shift got some scores and quite high people got interested in it.

But if reviews in general didn't have scores it would encurage those people to actually read some reviews themselves rather than trust a random number that someone else has associated with their liking of the game.

And those that are not encouraged to read about games would end up picking games up based on word-of mouth like people used to do anyhow.... which certainly isn't a bad thing as it would cut down all that 1st week hype crap where people buy games basically because they recieved a few 90+ scores a couple of weeks before release.



Lostplanet22 said:
Dropping a scoring system could also be very negative. A lot of gamers get interested in a game if they see that it is getting high scores.

Something we saw with the last Need for speed game. Saw a lot of comments (not especially on VGCHARTZ) that people were not interesting in the game because the one before Shift sucked but when Shift got some scores and quite high people got interested in it.

I don't think scoring has anything to do with it.

Here's the PS3 sales of Undercover and Shift. They're flat.

Here's the 360 sales of Undercover and Shift. Shift is trending a couple hundred thousand below Undercover in the same timeframe.

Here's the PSP sales of Undercover and Shift. Shift has only been released in America thus far, so I stuck with only that region for both games. Shift is trending WAY below Undercover.

The reviews are all heavily skewed in Shift's favor across all three platforms, by anywhere from 10-30 points difference. It has done nothing positive for sales thus far, and on 2/3 of the platforms, sales have dropped.



Tag: Became a freaking mod and a complete douche, coincidentally, at the same time.