By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Gaming Discussion - Understanding Review Scores - Quit the Console Review War Crap

VAMatt said:

I miss the days of walking into the store, having no idea what game you're going to leave with ,looking at pictures and reading the backs of boxes, and leaving with a new game to experience on your own, without the taint of other people's opinions.

I hated those days :).

Seriously though I had to save up 3 birthdays and Christmas to buy a PS3. The only thing I could consistently ask for is WoW, because my mother thought $15 a month was easier to pull of than $60 every 4-6 months.

Of course now I do exactly that. Buy games without the taint of other people's opinions. Though my friends are always like buy this, buy this, buy this, but that has always been a thing. And I mostly ignore them haha.



Around the Network
Wyrdness said:

- This same argument applies to score numbers so you've argued against reviews in general here, reviews are the to detail why they came to their verdict not just to put a number down and be done all you're telling us here is that you can't be bothered to read a few reviews or go through a summary that's not a problem with a verdict system that's a problem with how you use reviews.

No, I'm not arguing against reviews in general, I'm arguing against reviews without a score.

Gradient scores allow a quick comparison of games (of the same genre), which helps with the preselection.
After the preselection, I read a few reviews of the most promising candidates.

If I want to buy a new action-adventure and 100 of the 300 new action-adventures are in the "good category", I don't want to read 100 reviews (or 200 or 300 for different opinions. Instead I want to sort these 100 good games by score and then read the reviews of the top 10.

And if I want to gift a bad game to my friend for the lulz, I want to gift a REAL bad one... without reading through hundreds of reviews. 

So a sorting option by score is also welcome for bad games.



Conina said:

No, I'm not arguing against reviews in general, I'm arguing against reviews without a score.

Gradient scores allow a quick comparison of games (of the same genre), which helps with the preselection.
After the preselection, I read a few reviews of the most promising candidates.

If I want to buy a new action-adventure and 100 of the 300 new action-adventures are in the "good category", I don't want to read 100 reviews (or 200 or 300 for different opinions. Instead I want to sort these 100 good games by score and then read the reviews of the top 10.

And if I want to gift a bad game to my friend for the lulz, I want to gift a REAL bad one... without reading through hundreds of reviews. 

So a sorting option by score is also welcome for bad games.

Your argument applies to reviews in general so that point you tried to make is against reviews as a whole regardless of what grading system is used.

- Quick comparisons can be done with a verdict as well, just read the final summary and pros and cons.

- If you're looking for the top 10 as you say they'd be easy to find as they would be the games people are hyping up the most, social media and all would do that for you.

- You don't need to read hundreds of reviews only one or two will suffice if you're looking for a bad game, again the summary can be used for a quick read.



I don't think there's a problem in using critic scores to help decide which games to buy on day one, to buy on sale, to play through on a subscription service, etc. I think a bigger problem is some think games that score in the 70's - 75's are not worth playing.




PotentHerbs said:

I don't think there's a problem in using critic scores to help decide which games to buy on day one, to buy on sale, to play through on a subscription service, etc. I think a bigger problem is some think games that score in the 70's - 75's are not worth playing.


Yeah, and this stems from the complete lack of agreement on what a score should signify, not only from consumers but from publications themselves.

 



Around the Network

Gradiant scales and verdicts both works, movie reviews websites do really a good job with both systems

They work because being a movie critic is (or was) a PROFESSION. Overall there was clear guidelines and frameworks to use when picking if something is bad or good, and decades of practice and culture refined the practices until there is a more or less agreement of what makes a movie good or bad and to scale if in simple systems (generally of 5 stars). Besides, cinema is a college study field, there is enough theory to help you identify elements that could constitute a good or bad movie 

Of course the modern state or cinema lead to a proliferation of journalists and non professional critics, which overall decrease the quality of scores, but as I said, there is enough documented text to help and most of those reviews just repeated whatever the true important and influential reviews says

Gaming reviews were originally made from nerds who doesn't have anything else to do with their lifes and created gaming websites, or from few magazines dedicated to gaming news and advertising. This fact holds true for this day. Worst, they seemed to be created specifically to help selling games, that's why the scores are very rarely bad (less than 7). Games need a lot of time and thought to test, a game with like 100 hours will of course be sent to reviewers weeks before the release. If the community start to bomb your game you will just find another nerd to review the game for you

Gaming reviews are, more than anything, bought. They are bought nor for money, but to making those websites afloat with content. If you don't get a gaming code before release your review will maybe take a few weeks to be released and you will just lose readers and clicks because somebody else got their code and released the review before you. So better give this mediocre game an 8 score and keep getting codes from studios 

The problem is then, hardly the system, but the quality of gaming reviewers and the dynamics between reviews and studios. Changing the system won't change the root of the problem. 



To say it bluntly.

Arguing on reviews and comparing games based on scores is pointless. Unfortunately this is what most people seem to do on the internet (or maybe it's just the most vocal ones).

At the end scores matter VERY LITTLE.

Arguments is what matters in a review. The quality of the argumentation is what determines the reputation of a reviewer and the credibility of a review.

Reviews ARE ultimately well argumented informed opinions so of course you are allowed to agree or disagree based on the arguments presented and how they are presented.

Reviews ARE NOT objective quality assessments. Objectivity, as the word it self suggest, presumes a relation or a comparison with an actual object (a metric). An objective metric to compare different creative operas (games, movies, paintings, music, etc.) simply does not exist. And no, an aggregator is not a surrogate for objectivity: the sum of objective measurements is an objective measurement but the sum different opinions is still an expression of opinions.

Real world example:

- Let's say I'll ask you to tell me how large is the screen of the device you're using. Your answer would be like 9 inches. That is would be objective, since you relate the size of the screen with an actual object (1 inch). Let's say I'll ask 100 people the same question. At the end I would be capable of measuring the average size of their screens objectively, since all the measurements are related to an object (1 inch).

- Now let's say I'll ask: how large is your screen from 1 to 10 ? The answer would be like 7/10. That would definitely not be objective, as much as you could argument your score you would be biased depending on what devices you were accustomed to use and to what meaning you would attribute to the score itself (ex. does 10/10 mean the screen is comfortably big or is it too big?). Let's say I'll ask 100 people the same question and calculate the average score. At the end what does the score tell me? Absolutely nothing, it's actually less meaningful the a single score since it would be detached to the argumentation that lead to the value.

At the end the only effective use of score aggregators is to give a quick glance to understand the most popular opinion about a game.

The fixation for review scores and supposed objectivity, that form what I see is peculiar to gaming (as compared to other media), imo is actually a great detriment to actual constructive criticism and one big detractor for me to consider gaming a mature art form as compared to other media.

Edit: And this is of course not to devalue the work of professional reviews, on the contrary I respect a lot their work (btw. even my own brother worked as a reviewer for some time). Their job is to be inquisitive, they have to be informed and they should present their opinion in an accessible and well argumented way. It's up to them to decide how much they want to try to be detached from personal biases (the important thing is to specify your reasoning).

Last edited by freebs2 - on 15 March 2022

Doctor_MG said:

Reviewers are professionals (typically) who are aiming to provide an objective measurement of a subjective experience. IMO, a professional reviewers opinion matters to me more than most other random people's opinions. However, they are still having a subjective experience so we may not see eye to eye on things. What my preference is, I find a reviewer I generally agree with (my views line up with Dunkey's in just about everything but RPG's) and I look to their reviews as a better barometer for whether or not I'll like the game.

With regards to things like Metacritic, I think when you have an average of 89, or 92, or whatever...at that point, if you don't agree, I think it's wise to admit that you're probably in the minority or biased to some extent. I don't believe in games being overrated on a universal level, at least for the time. The reverse is true as well. A game that scores a 51 metacritic is more likely a bad game and your enjoyment is either an outlier difference or bias (such as an old game you grew up with).

Basically, I don't think that game ratings are completely subjective. However, experiences still vary and a review or multitude of reviews doesn't invalidate your personal and individual feelings about the game. Just don't take that personal feeling and try to make it universal (i.e. underrated, overrated, objectively awful, etc).

You also have to admit that, in their quest for objectivity, reviewers as a whole tend to flock toward metrics that are often irrelevant for the vast majority of players. That’s why review scores are often inconsistently low with some of the most heavily loved games of all time.

Pokémon Go is a great example of a game that became a cultural phenomenon and is one of the highest grossing games of all time, generating money on what really amounts to a whole lot of good will since the game is free and isn’t pinchy at all. Sure, some reviewers scored it high, but the meta score is low.



I describe myself as a little dose of toxic masculinity.

Scores should be one of 3 options; buy, price drop, avoid



Jumpin said:

You also have to admit that, in their quest for objectivity, reviewers as a whole tend to flock toward metrics that are often irrelevant for the vast majority of players. That’s why review scores are often inconsistently low with some of the most heavily loved games of all time.

Pokémon Go is a great example of a game that became a cultural phenomenon and is one of the highest grossing games of all time, generating money on what really amounts to a whole lot of good will since the game is free and isn’t pinchy at all. Sure, some reviewers scored it high, but the meta score is low.

I think the difference here is that you are equating time spent or money spent with quality. Though these may be correlated, they are different concepts.

In the case of Pokemon, it is an international sensation and has been for many MANY years now. Pokemon Go being a cultural phenomenon is less about the game itself and it's mechanics (specifically early on) and more the fact that it is Pokemon. We can easily see this by looking at the popularity of Niantic's previous game before Pokemon and even the Harry Potter and Pikmin themed versions (which were all MUCH less successful). A reviewer is providing an objective measurement of a subjective experience based on graphical presentation, narrative, gameplay mechanics, etc. But we know that quality doesn't always equate to sales (Persona 5 vs. Pokemon Sword and Shield). This doesn't mean that the reviewers are "wrong" as so much more goes into the popularity of a game than just it's general gameplay. Targeted demographic, cost, amount of platforms it's on, marketing, etc. all of these factors play a huge role but none of these are typically considered when looking into a review (aside from maybe cost occasionally).