By using this site, you agree to our Privacy Policy and our Terms of Use. Close
Reasonable said:
twesterm said:

Most of those games you listed I'm sure you expect to get between a 9 and 10.  A bad game (or even a game that just doesn't deserve that score but still good) isn't automatically an 8, it's just something else so that variation is meaningless.

Lets just pretend that Awesome Exlclusive VII for the PS3 and Bodacious Exclusive IV for the 360 both have a 9 on metacritic.  Edge reviews them and gives Awesome Exlclusive VII a 7.5 while giving Bodacious Exlclusive IV an 8.5

According to you, that means Edge is automatically biased against the PS3 and both should either get a 9 or an 8.5 or just the same score when in reality they thought one game was a 7.5 and the other was an 8.5.

Do you see what I'm getting at?  They're different games so they can get different scores.

Just to be Devil's Advocate, while of course two seperate games can get different scores, if those scores are inconsistent with a general set of scores then it does raise questions about the scoring, particularly if you take the view that, opinions or no, they are opinions from supposed 'experts' on a common topic and should show a relative degree of consistency - i.e. either the lighting is great or its not, it can't be both.  Either the AI is good or its not, it can't be both - and so on.

As an analogy, if a film get's consistently praised in 90% of reputable reviews for having a good script, and one review site says the script is bad, you can see it as just another opinion, or you can wonder as to why?  Does that reviewer have weaker grasp of scriptwriting theory?  Did they not like the film particularly and let this influence their view on the script?  and so on.  The evidence would definately imply that there is reason to be cautious of the single poor opinoin of the script vs the universal acclaim.

Rather than just moaning, which I agree with everyone is a pain, the post on Edge vs metacritic averages is interesting to me because it appears to be factual, based on real data, and it clearly appears to show that, on average, a 360 title will review closer to or above the metacritic average than a PS3 title, which certainly does imply a bias.

Of course, that bias could be accidental.  I hate to sound elitist, but I don't really hold game reviewers on the whole as particularly 'expert' on game mechanics, etc.  and it may be no more than the PS3 reviewer is simply being harsher than the 360 reviewer, who really likes the 360 titles and his enjoyment reduces there percieved impact of any flaws.

But clearly, there is a skew there - if the shown data is correct.  I hate to actually pick up on this, as I normally really dislike the whole 'they're bought off by MS, etc' tinfoil hat nonsense.  But this analysis does show a skew.  Of course it could just be taste - that can't be discounted.  The reviewer may have genuinely felt KZ2 was simply average (to take the obvious example) while ODST is pretty stellar.

What I'd really be interested to see is a similar analysis for all major review sites / sources to see if skews are fairly common and chance over time - i.e. it is probably just different tastes, etc. - or whether only a few sites have a noticable skew from others.

 

 

Yeah, but we're just looking at the numbers so we have no idea why they give those numbers.  Review scores are meaningless without the actual review.

For all we know, 360 exclusives could just be better (I'm not saying they are so don't start :-p).  I don't know why they gave some games higher and others lower because I don't see those reviews. 

Sure, if they gave a glowing review to Resistance 2 and then gave it something 25% lower than everyone else, then yes, shenanigans, sure, but I'm guessing they gave a reason.

I'm guessing it is just accidental like you said and the fact some games get hyped much more.  Generally, a hugely hyped game will get huge scores despite the quality (assuming it's at least not bad).  Taking that theory in mind, look at your general 360 exclusive and PS3 exclusive.  It's hard to deny that PS3 exclusives get announced very early and get very hyped very quickly.  That's just how Sony works.

Look at the newest example, The Agent.  We know nothing about that game but people are already using it as one of the great new Sony franchises.  I'm sure when it's released it will get many 9's and 10's because it's hyped, it's exclusive, it's Rockstar, ect and then sites like Edge will actually review it.

If it's a 9 in their eyes, they'll give it a 9 like everyone else, but if it's a 7 in their eyes, then we have another one of those -20%+ plus outliers.  Now we could call shenanigans like everyone else, or maybe they're just hard reviewers and the game isn't as good as everyone is hyping.

Note that above is all hypothetical, but look at the exlusives on that list.  All those ones people are screaming shenanigans at are games that people love or hate and if you take away the hype, it's pretty reasonable to give them scores that aren't your hyped 9. 

We could keep calling shenanigans or just call a 7 a 7 and be done with it.