By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Gaming - Review Inflation

Kenryoku_Maxis on 04 November 2009
tombi123 said:
Probably because they receive tons of hate mail from fanboys if they score a game too low.

This.  Review sites inflate scores for their games based on what type of game it is, not based on the actual content of the game.  They're trying to generate clicks to their site and more readers, not an accurate score.  Some reviewers are overly harsh to try to generate 'controversy' while others try to give inflated positive scores to most games to make fans feel like every game is good.

Its a flawed system to its core that was only somewhat accurate 10 years ago when people were scoring games on the PSX/N64 through Magazines and the early internet.  Once the reviewers found they could make money off the system...there went any credibility with the system.  It just became a 'system' of placating the public and a job.



Six upcoming games you should look into:

 

  

Around the Network
TWRoO on 04 November 2009
r505Matt said:
Onyxmeth said:
TWRoO said:
I was diheartened when the community here chose the 100 point scale for this website's reviews.... although the poll did I think have 2 different ones that are essentially the same scoring system (0.0, 0.1, 0.3 through to 9.9, 10.0.... and out of 100 are the same thing)

Out of 5 or less if a score is somehow vitally needed.

I like this method also, and only in single digit increments.

5=Amazing
4=Great
3=Good/Average
2=Bad
1=Unplayable

This allows the reviewer to express themselves through the written portion of the review and not use the numbering as a crutch, at least it should help in theory. This way also, 5 doesn't have to mean perfect. It just describes the cream of the crop, or upper echelon of games.


A clear cut 1-5 rating is both good and bad. The good, it's clear cut, so it's a very stable way to review something. You can also have as much range as you want (3.45 if you really need it) so thats not an issue either.

The bad thing though, is that if you actually use only signel digits, well, that's kind of silly. To have only 5 degrees is VERY limiting. If you made a list of 20 games you considered as 5s, I'd bet that you would be able to pick out games in that list that are better/worse than others within that list. So they aren't all the SAME 5, they're all different 5s, which makes it VERY hard to trust that sort of system.

The bigger issue though is that it's TOO concrete. Reviewing is all relative, and that should be reflected in the system as well. To say the quality of average games today is the same as the quality of average games 15 years ago is silly. That's why reviews are raising up, since we all remember those games (those of us that aren't new to gaming, or incredibly young), and that influences reviews mildly enough to need the wiggle room that a non-concrete system gives.

 

The whole point was to cut down on the range... we are not talking about whether the highest mark is 5, 10 or 100 here as it makes no difference if there are the same number of tiers. scoring out of 100% is exactly the same as scoring out of 10 to the nearest 1.d.p.

That's the point.... making it more limited is a good thing. reviews are an opinion, there is no accuracy in an opinion so there is no need for accuracy in a score. My opinion says Mario Kart Wii should be rated above Super Mario Galaxy, but so what, they are both (to me) 5* games, so lets just leave it at that so that there are fewer arguments about which games scored what.

Quality of how well a game is made is nothing to do with how good the game is... All the money and time in the world can be put into a game and it still may not be enjoyable. Review scores shouldn't need wiggle room.



Onyxmeth on 04 November 2009
r505Matt said:
Onyxmeth said:
TWRoO said:
I was diheartened when the community here chose the 100 point scale for this website's reviews.... although the poll did I think have 2 different ones that are essentially the same scoring system (0.0, 0.1, 0.3 through to 9.9, 10.0.... and out of 100 are the same thing)

Out of 5 or less if a score is somehow vitally needed.

I like this method also, and only in single digit increments.

5=Amazing
4=Great
3=Good/Average
2=Bad
1=Unplayable

This allows the reviewer to express themselves through the written portion of the review and not use the numbering as a crutch, at least it should help in theory. This way also, 5 doesn't have to mean perfect. It just describes the cream of the crop, or upper echelon of games.


A clear cut 1-5 rating is both good and bad. The good, it's clear cut, so it's a very stable way to review something. You can also have as much range as you want (3.45 if you really need it) so thats not an issue either.

The bad thing though, is that if you actually use only signel digits, well, that's kind of silly. To have only 5 degrees is VERY limiting. If you made a list of 20 games you considered as 5s, I'd bet that you would be able to pick out games in that list that are better/worse than others within that list. So they aren't all the SAME 5, they're all different 5s, which makes it VERY hard to trust that sort of system.

The bigger issue though is that it's TOO concrete. Reviewing is all relative, and that should be reflected in the system as well. To say the quality of average games today is the same as the quality of average games 15 years ago is silly. That's why reviews are raising up, since we all remember those games (those of us that aren't new to gaming, or incredibly young), and that influences reviews mildly enough to need the wiggle room that a non-concrete system gives.

 

A 1-5 works perfectly, because to differentiate between what's better or worse out of multiple titles of the same score, you READ THE REVIEW. That is what has gone horribly wrong with scoring games out of 100. It forces reviewers to think too hard about the numbers, which encourages inflation. They'll remember a title they gave a 91 to in graphics and realize this title is better so they give it a 93, so on and so forth. Giving a vague number allows the reviewer to use the numbering system as a generalization of their feelings towards the game, and not a scientific formula that needs to be examined to it's minute decimals.

You're effectively agreeing with the current trend that the review systems continue to go up because the quality of gaming is going up. That's fine and all. However, as games progressively get "better" I hope you enjoy picking apart the average games that get a 90, from the good games that get a 92, to the great games that get a 95, to the excellent games that get a 97, to the classic games that all get 100. That's where we're heading.



Tag: Became a freaking mod and a complete douche, coincidentally, at the same time.



Kenryoku_Maxis on 04 November 2009
Onyxmeth said:
r505Matt said:
Onyxmeth said:
TWRoO said:
I was diheartened when the community here chose the 100 point scale for this website's reviews.... although the poll did I think have 2 different ones that are essentially the same scoring system (0.0, 0.1, 0.3 through to 9.9, 10.0.... and out of 100 are the same thing)

Out of 5 or less if a score is somehow vitally needed.

I like this method also, and only in single digit increments.

5=Amazing
4=Great
3=Good/Average
2=Bad
1=Unplayable

This allows the reviewer to express themselves through the written portion of the review and not use the numbering as a crutch, at least it should help in theory. This way also, 5 doesn't have to mean perfect. It just describes the cream of the crop, or upper echelon of games.


A clear cut 1-5 rating is both good and bad. The good, it's clear cut, so it's a very stable way to review something. You can also have as much range as you want (3.45 if you really need it) so thats not an issue either.

The bad thing though, is that if you actually use only signel digits, well, that's kind of silly. To have only 5 degrees is VERY limiting. If you made a list of 20 games you considered as 5s, I'd bet that you would be able to pick out games in that list that are better/worse than others within that list. So they aren't all the SAME 5, they're all different 5s, which makes it VERY hard to trust that sort of system.

The bigger issue though is that it's TOO concrete. Reviewing is all relative, and that should be reflected in the system as well. To say the quality of average games today is the same as the quality of average games 15 years ago is silly. That's why reviews are raising up, since we all remember those games (those of us that aren't new to gaming, or incredibly young), and that influences reviews mildly enough to need the wiggle room that a non-concrete system gives.

 

A 1-5 works perfectly, because to differentiate between what's better or worse out of multiple titles of the same score, you READ THE REVIEW. That is what has gone horribly wrong with scoring games out of 100. It forces reviewers to think too hard about the numbers, which encourages inflation. They'll remember a title they gave a 91 to in graphics and realize this title is better so they give it a 93, so on and so forth. Giving a vague number allows the reviewer to use the numbering system as a generalization of their feelings towards the game, and not a scientific formula that needs to be examined to it's minute decimals.

You're effectively agreeing with the current trend that the review systems continue to go up because the quality of gaming is going up. That's fine and all. However, as games progressively get "better" I hope you enjoy picking apart the average games that get a 90, from the good games that get a 92, to the great games that get a 95, to the excellent games that get a 97, to the classic games that all get 100. That's where we're heading.

A 1-5 rating system is horribly limiting, the same as a 1-10 system.  You'll encounter the same exact fan outcry as you see today, but compounded because various games keep getting '3's while others get '4's.  And then only one game a year gets a '5' (lol GTAIV).

I would agree that sites should drop the rating system.  But they won't, because its what brings the majority of their readers back.  Our very forum being spammed with 'IGN gives [X] a [X.X]'!!!! is testament to this.  Probably 25% of people read the actual review and probably less than that actually take context from it.  And from what I've been reading on these review sites, it kind of looks like that's what they count on.



Six upcoming games you should look into:

 

  

Smeags on 04 November 2009

I once had a gaming blog that sometimes reviewed games. Our scoring system was this:





It was simple, yet got the point across. Plus, it made readers actually live up to their name and read the written review.



3DS Rec Thread ~ Wii U Rec Thread ~ Xbox One Rec Thread

Around the Network
Onyxmeth on 04 November 2009
Kenryoku_Maxis said:

A 1-5 rating system is horribly limiting, the same as a 1-10 system.  You'll encounter the same exact fan outcry as you see today, but compounded because various games keep getting '3's while others get '4's.  And then only one game a year gets a '5' (lol GTAIV).

I would agree that sites should drop the rating system.  But they won't, because its what brings the majority of their readers back.  Our very forum being spammed with 'IGN gives [X] a [X.X]'!!!! is testament to this.  Probably 25% of people read the actual review and probably less than that actually take context from it.  And from what I've been reading on these review sites, it kind of looks like that's what they count on.

Fan outcry doesn't matter. Placating to that nonsense is partially why it's such a mess in the first place. If everyone were to go to a 1-5 system, fans would read it. Trust me. They will read it, and then adjust and adapt to it because they have no alternative. Subsequently, it would correct many of the problems game reviews have. What do you think forum folk are going to do, stop paying attention to reviews? Yeah right.

The 5 wouldn't have to be mainly for just the top title of the year though. It could be for any game the reviewer feels is a top notch game within it's genre, so essentially you could judge Nintendogs as a 5 for pet simulations, and Halo 3 as a 5 for FPS, and then let the readers get the context of the worth of each title from the review itself and not the numbers. This could also help to cut down on genre bias, but not necessarily so.



Tag: Became a freaking mod and a complete douche, coincidentally, at the same time.



Lostplanet22 on 04 November 2009
Onyxmeth said:
Lostplanet22 said:
Dropping a scoring system could also be very negative. A lot of gamers get interested in a game if they see that it is getting high scores.

Something we saw with the last Need for speed game. Saw a lot of comments (not especially on VGCHARTZ) that people were not interesting in the game because the one before Shift sucked but when Shift got some scores and quite high people got interested in it.

I don't think scoring has anything to do with it.

Here's the PS3 sales of Undercover and Shift. They're flat.

Here's the 360 sales of Undercover and Shift. Shift is trending a couple hundred thousand below Undercover in the same timeframe.

Here's the PSP sales of Undercover and Shift. Shift has only been released in America thus far, so I stuck with only that region for both games. Shift is trending WAY below Undercover.

The reviews are all heavily skewed in Shift's favor across all three platforms, by anywhere from 10-30 points difference. It has done nothing positive for sales thus far, and on 2/3 of the platforms, sales have dropped.

Was not really talking about the sales considering sales could be worse if it had also bad reviews don't want to discuss it but games like batman and okami where games that were out of my radar until I saw many threads a la '(Site's name) scores 9/10 for Okami/batman etc.   If a game that I was not interested in getting high scores I mostly will check reviews out.  Numbers don't matter for me in games I am really interested in or sequels of games that I love....Then the reviews are important for me like Halo ODST.



 

O-D-C on 04 November 2009

my personal scale

10 - Classic
9 - Very Good
8 - Good
7 - Above Average
6 - Slighlty Above Average
5 - Average
4 - Slightly below average
3 - Bad
2 - Very Bad
1 - Abysmal



Onyxmeth on 04 November 2009
Lostplanet22 said:
Onyxmeth said:
Lostplanet22 said:
Dropping a scoring system could also be very negative. A lot of gamers get interested in a game if they see that it is getting high scores.

Something we saw with the last Need for speed game. Saw a lot of comments (not especially on VGCHARTZ) that people were not interesting in the game because the one before Shift sucked but when Shift got some scores and quite high people got interested in it.

I don't think scoring has anything to do with it.

Here's the PS3 sales of Undercover and Shift. They're flat.

Here's the 360 sales of Undercover and Shift. Shift is trending a couple hundred thousand below Undercover in the same timeframe.

Here's the PSP sales of Undercover and Shift. Shift has only been released in America thus far, so I stuck with only that region for both games. Shift is trending WAY below Undercover.

The reviews are all heavily skewed in Shift's favor across all three platforms, by anywhere from 10-30 points difference. It has done nothing positive for sales thus far, and on 2/3 of the platforms, sales have dropped.

Was not really talking about the sales considering sales could be worse if it had also bad reviews don't want to discuss it but games like batman and okami where games that were out of my radar until I saw many threads a la '(Site's name) scores 9/10 for Okami/batman etc.   If a game that I was not interested in getting high scores I mostly will check reviews out.  Numbers don't matter for me in games I am really interested in or sequels of games that I love....Then the reviews are important for me like Halo ODST.

You said a lot of gamers get interested in games because of high review scores. You then gave the example as Need for Speed: Shift, not me. I merely showed you whatever portion of gamers actually base their purchases in this way, in the case of this game are practically irrelevant to the big picture, thus there is no big negative to dropping a scoring system on reviews.



Tag: Became a freaking mod and a complete douche, coincidentally, at the same time.



Kenryoku_Maxis on 04 November 2009
Onyxmeth said:
Kenryoku_Maxis said:
 

A 1-5 rating system is horribly limiting, the same as a 1-10 system.  You'll encounter the same exact fan outcry as you see today, but compounded because various games keep getting '3's while others get '4's.  And then only one game a year gets a '5' (lol GTAIV).

I would agree that sites should drop the rating system.  But they won't, because its what brings the majority of their readers back.  Our very forum being spammed with 'IGN gives [X] a [X.X]'!!!! is testament to this.  Probably 25% of people read the actual review and probably less than that actually take context from it.  And from what I've been reading on these review sites, it kind of looks like that's what they count on.

Fan outcry doesn't matter. Placating to that nonsense is partially why it's such a mess in the first place. If everyone were to go to a 1-5 system, fans would read it. Trust me. They will read it, and then adjust and adapt to it because they have no alternative. Subsequently, it would correct many of the problems game reviews have. What do you think forum folk are going to do, stop paying attention to reviews? Yeah right.

The 5 wouldn't have to be mainly for just the top title of the year though. It could be for any game the reviewer feels is a top notch game within it's genre, so essentially you could judge Nintendogs as a 5 for pet simulations, and Halo 3 as a 5 for FPS, and then let the readers get the context of the worth of each title from the review itself and not the numbers. This could also help to cut down on genre bias, but not necessarily so.

That's the problem I'm worried about.  Certain reviewers, like Famitsu, review games as you described, as being 'the best in their genre' or 'best on their system'.  While most western reviewers are taking a different approach and reviewing games based on a multiple point scale including graphics, presentation and a completely gray area of 'gameplay'.  When you throw these things into a pot, most Wii/DS games get downscored next to 360/PS3 games because they don't have as good graphics and are more harshly judged in gameplay.

Putting that into a 5 point scale would be a madhouse is all I'm saying...generating a ton of 2s and 3s for DS and Wii games and a lot of 4s and 5s for 360/PS3 games.  Its essentially what we have now with the difference being DS/Wii games range between 7.0-8.9 while 360/PS3 games avg between 8.0-9.5.

But who knows...maybe it could fix everything.



Six upcoming games you should look into: