By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Gaming Discussion - Understanding Review Scores - Quit the Console Review War Crap

Doctor_MG said:

Reviewers are professionals (typically) who are aiming to provide an objective measurement of a subjective experience. IMO, a professional reviewers opinion matters to me more than most other random people's opinions. However, they are still having a subjective experience so we may not see eye to eye on things. What my preference is, I find a reviewer I generally agree with (my views line up with Dunkey's in just about everything but RPG's) and I look to their reviews as a better barometer for whether or not I'll like the game.

With regards to things like Metacritic, I think when you have an average of 89, or 92, or whatever...at that point, if you don't agree, I think it's wise to admit that you're probably in the minority or biased to some extent. I don't believe in games being overrated on a universal level, at least for the time. The reverse is true as well. A game that scores a 51 metacritic is more likely a bad game and your enjoyment is either an outlier difference or bias (such as an old game you grew up with).

Basically, I don't think that game ratings are completely subjective. However, experiences still vary and a review or multitude of reviews doesn't invalidate your personal and individual feelings about the game. Just don't take that personal feeling and try to make it universal (i.e. underrated, overrated, objectively awful, etc).

You also have to admit that, in their quest for objectivity, reviewers as a whole tend to flock toward metrics that are often irrelevant for the vast majority of players. That’s why review scores are often inconsistently low with some of the most heavily loved games of all time.

Pokémon Go is a great example of a game that became a cultural phenomenon and is one of the highest grossing games of all time, generating money on what really amounts to a whole lot of good will since the game is free and isn’t pinchy at all. Sure, some reviewers scored it high, but the meta score is low.



I describe myself as a little dose of toxic masculinity.

Around the Network

Scores should be one of 3 options; buy, price drop, avoid



Jumpin said:

You also have to admit that, in their quest for objectivity, reviewers as a whole tend to flock toward metrics that are often irrelevant for the vast majority of players. That’s why review scores are often inconsistently low with some of the most heavily loved games of all time.

Pokémon Go is a great example of a game that became a cultural phenomenon and is one of the highest grossing games of all time, generating money on what really amounts to a whole lot of good will since the game is free and isn’t pinchy at all. Sure, some reviewers scored it high, but the meta score is low.

I think the difference here is that you are equating time spent or money spent with quality. Though these may be correlated, they are different concepts.

In the case of Pokemon, it is an international sensation and has been for many MANY years now. Pokemon Go being a cultural phenomenon is less about the game itself and it's mechanics (specifically early on) and more the fact that it is Pokemon. We can easily see this by looking at the popularity of Niantic's previous game before Pokemon and even the Harry Potter and Pikmin themed versions (which were all MUCH less successful). A reviewer is providing an objective measurement of a subjective experience based on graphical presentation, narrative, gameplay mechanics, etc. But we know that quality doesn't always equate to sales (Persona 5 vs. Pokemon Sword and Shield). This doesn't mean that the reviewers are "wrong" as so much more goes into the popularity of a game than just it's general gameplay. Targeted demographic, cost, amount of platforms it's on, marketing, etc. all of these factors play a huge role but none of these are typically considered when looking into a review (aside from maybe cost occasionally). 



In my personal experience reviews are pretty important. When I started playing videogames I usually asked my parents to get me games based on licenses as a kid. Star Wars games, Two Towers, Batman Begins, the early Harry Potter games. I don't regret any of those purchases and with very rare exceptions these games were all fun, but I was very clearly a casual gamer who didn't really get to see what the best of the industry could do other than having Smash Bros. Melee. When the Wii came out and I was buying games with my own allowance money instead of my parents deciding to get me a game every now and then I started paying more attention to reviews, and that's a big part of why I bought games like Twilight Princess and Metroid Prime 3 and saw what games could be. It was reviews that saved me from getting Far Cry Vengeance after seeing the misleading cover box at a Gamestop. It is reviews that got me to check out games like Portal, Mass Effect, and Bioshock in the 2010s when I finally got a computer powerful enough to run them. I've seen games get fantastic reviews that I think are overrated like pretty much anything by Naughty Dog, and plenty of 7-8/10 games that are as worthwhile as a 9/10 game, but for the most part I've learned to tell if a reviewer has similar tastes to me and therefore how much I'd be likely to agree with them about a game.



I can look at a review and intuitively find what matters to me without caring about the score or extraneous factors.

One thing I’ll say though is that quality of gameplay is very much an opinion. You can give/take credit depending on the execution of a gameplay loop. You cannot decide who will like that gameplay loop. You can influence other people’s opinion with your opinion.



Around the Network
Kyuu said:

Examples of games where I "agree" (give or take 5 points) with both the general opinion and critical reception (aggregated):
Countless!

Examples where I kinda agree with critical reception against the general opinion:
Chrono Cross, MGS4.

Examples where I agree with the general opinion against critic scores.
Parasite Eve, Suikoden 2, Final Fantasy Tactics, NieR (PS360), GTA4, MGS5, The Last of Us Part 2

Examples where I agree with neither the general opinion nor critical reception:
Parasite Eve 2, Oblivion, Skyrim, Persona 5, God of War (PS4), Uncharted 4.

Critical reception can be a joke, especially towards established vs unknown games or developers, or when fanboys and platform warriors are involved. GTA4's Metacritic for instance couldn't be more misleading! NieR wasn't well received, and yet the much more outdated (relative to launch) Remaster was highly praised, because it's now considered cool to be a fan of NieR and Yoko Taro, thanks to the hype and success of NieR Automata. But in the end, checking out what's hot is the safer method for a busy or poor gamer than trying to explore games that are criticized or not recommended by many.

A lot of the games I like are those where I disagreed with both the general opinion and critical reception. A lot of the games I didn't enjoy were also games where I disagreed with both the critics and the general public.

NieR is an interesting case. It got absolutely hammered in critical reviews on PS3/360 and didn't sell especially well. NieR: Automata being the huge critical and sales success it ended up being was a mild shock to me, though I did have higher hopes for it with Platinum than with the original developers, Cavia. 



Too much emphasis on the final score. Not enough on the content of the review.

There are issues that just don't bother me. I tend to know what I like and know what I'm going to buy way in advance. I only use reviews when I'm on the fence.



VAMatt said:

I miss the days of walking into the store, having no idea what game you're going to leave with ,looking at pictures and reading the backs of boxes, and leaving with a new game to experience on your own, without the taint of other people's opinions.

I always picked out poop.



Welp, I didn't expect this much discussion... I'm beginning to think I need to integrate more "forum" time into my week, though I ALSO think that would just scare more of you off lol

Having said that, there ARE things I want to address, though there are too many people to specifically quote (I WILL address a few key ones below, though).

I think there's a constant mixup between enjoyment, quality, and ratings.  The reason reviews should be as analytical as possible is BECAUSE the point of reference for enjoyment varies by person, but what does NOT vary by person is, say, poor responsiveness in controls.  Thus, someone could read a review that says the framerate affects the experience by being inconsistent and affects reaction time/responsiveness of controls.  But a player out there could still enjoy it because, say, laser beams and explosions because Michael Bay shit be happening which is what causes the framerate to tank.  Where the mixup happens is that the person who enjoys it believes that their enjoyment invalidates the reviewers experience or analysis entirely.  I see this a lot: it's as if most "gamers" are looking to prove someone wrong by being a contrarian.  They're mixing up the fact that they didn't disprove anything: the framerate was still tanking and decreasing response time.  But to them, "the reviewer's wrong for rating this game lower than I would because I enjoyed it". This is a false equivalence and a mixup of values.

The other issue that often gets mixed up in here is the last topic in my last paragraph: contrarians.  Some people simply want to disagree because validation for them comes from being "different" and not "sheeple".  "I make my own decisions, I make my own calls, I decide for myself".  That's all fine and dandy, but the ideology becomes toxic in gaming where it's almost some sort of "alpha male" play.  You can see that this has nothing to do with the review anymore and just people looking to spew rhetoric in order to give themselves a voice.   When pressured to analyze their "take", you'll often see them fall apart and end up trying to reference other people because they themselves actually lack the analytical ability to come to defined conclusions.  Ergo, you see this often as "this game is mid" and "this game is trash" or "this game is my GOTY, I don't care what anyone else says", but you can't find much else coming from them to back this up.  This is a large part of the console review wars because many of these people rely on the reviews to back their statements up.  They themselves have are latching onto narratives that fit their own to bolster it rather than genuinely reaching said statements through personal experience.

But now I'd like to address the more on-topic thing of the thread: whether review scores or reviews themselves have any value. Call it the Asian in me, but for even $5, I'd like to know to the best of my ability what I'm about to buy.  Not everyone has enough expendable income to simply "buy games and find out".  Some people want to make the wisest choices possible because they can, say, only afford one game every 3-4 months.  It is not FAIR to tell these people they need to ignore reviews, and the more granular and analytical a review is, the more useful it is for them.  Am I personally against just willy-nilly buying games?  Yes: it was part of my own philosophy to more properly break out and build upon my own reviews in an effort to make them useful to end users and not for publications.  One could look at my independent reviews as a more detailed user review because my goal with reviews was to provide the very thing I see being argued in this very thread: information.  Knowledge is power, as the saying goes, and I'm on the side of people who properly analyze reviews so they can understand what they're buying.  And the kicker here is, they may still buy a game in the 60s or 70s based on what they find out about the experience of the game and I would not be against it at all.  What I am against is people trying to encourage others to just buy things without so much as really understanding what the heck they're into, or trying to reduce reviews to such a narrow range of informational output that it hurts people who simply don't have as much money and want to be pickier with their games knowing they may be stuck with something they don't enjoy.

And here's the thing: I wouldn't be buying nearly the games I am for reviewing if I hadn't relaunched the channel.  I'd be doing THE very thing I'm preaching: I'd be reading reviews, looking for commonalities, etc. off the aggregate.  But since I do my best to review in a bubble, I HAVE to buy games blindly because that's the only way to provide other people the most accurate experience going into it as an end user buying a game.

coolbeans said:

My issue with the 2nd half of your video is how confused it is regarding "opinion." Even after bringing up Google's common definition, you don't seem to internalize the "not necessarily based on fact or knowledge" part of it. Considering that malleability, you should've examined other definition sources to get a better grasp on what that entails. Because this argument of divorcing "opinion" and "factual analysis" as much possible has some big potholes.

You're not wrong, but I was aware of this simply because the definition of "opinion" encompasses other industries where the term is used.  For example, when you talk to a doctor over the phone or for a checkup about your condition, until they actually perform scientific medical analysis, it's just an "opinion".  Thus, saying, "I'd like to get another doctor's opinion" has a somewhat different context than in gaming.  Doctor's are professionals with knowledge, but even their prognosis can be incorrect, and thus it is an opinion until it is backed by facts (testing, etc.).  And even then, the medical testing could prove them wrong, hence why the definition accounts for this possibility of not necessarily being fact.  So this Google definition encompasses the most general use of the word "opinion", though it's not entirely unfitting, either.  The idea is that even with knowledge, one could be wrong without analysis. Ergo, it's an opinion until it's proven otherwise.

In gaming, this applies rather differently.  It's used to define the idea that someone's statements are invalidated by disagreement.  As mentioned above, someone could still enjoy a game with a bad framerate, but they see this as means to define the review as an "opinion" because they "still enjoyed this game the reviewer gave a 7 to".  And yet, the reviewer could have very well stated and proved facts within the review, also as above: the framerate factually dips during heated action which reduces input responsiveness, thus they enjoyed it less or didn't enjoy it.  Using an analogous example: the reality is that Digital Foundry is applying the "medical testing", and thus you rarely see people refer to Digital Foundry as "opinion".  Then again, graphics technology IS easier to do this with, but I know there are others that, like myself, try to do the best we can with less resources to show what I say so that it's not just wild conjecture like "this game is absolute trash" without actually showing or talking about WHAT is so bad or what makes it that way.  And based on how you broke down what I said, I'm betting you possess the necessary intelligence to see or at least have witnessed exactly what I'm saying, here.  There IS such a thing as an unhealthy amount of opinions in a review, but again, not everything in a review is simply "opinion" because "someone disagreed".  Yes, there are highly opinionated reviewers, but that's also why I talk about actually reading/watching reviews and trying to understand it, not necessarily agree with it.

Chrkeller said:

Scores should be one of 3 options; buy, price drop, avoid

I see this idea show up a lot, but it contains THE biggest problem of all: imagine if a score said "avoid" on a game you bought and enjoyed.  The very problem you think you've solved has actually been worsened by binary choices.  Gradation is important because HOW good or HOW bad can make or break someone's experience with a game.  I would never use a "buy" or "don't buy" method, it's dangerously close to just being a paid (or unpaid) sales rep.  If a review's summation is to decide what your conditions are for purchasing, then that's not a review I would put much stock in because it's likely based around the "sales" idea and not the "information" idea.  We should be getting smarter with the advent of more information at our fingertips, not trying to dumb everything down to the point that we might as well be back in the days of blindly buying games because we saw a commercial on TV.



Check out my entertainment gaming channel!
^^/

The advantage of review scores is that you can analyse huge datasets with numbers and filter / sort / exclude games by scores.

For example I wanted to know, how many Steam games with at least 70%, 75%, 80%, 85%, 90% SteamDB-Rating are already marked as Steam Deck compatible.

No problem thanks to a SteamDB rating slider in the Steam Database:

Wouldn't have worked with reviews without a score but only pros and cons.

Wouldn't have been that detailed with only three categories (good, average, bad).