By using this site, you agree to our Privacy Policy and our Terms of Use. Close

Forums - Gaming Discussion - Edge vs. Metacritic (Bias Confirmed!!!)

slowmo said:

No point recylcling old arguments is there usually, I've made my point several times already on this site.  This thread isn't bringing new data to the table just the same old data people post packaged in graphs, we don't need the arguments again.  By showing that the OP is basing his entire argument of proof of bias on flawed data isn't unfounded by the way. 

I assume you consider Metacritc is a perfect measure to judge and only the top 30 games a suitable measure though?  Lets not take into account the hype factor that causes many games to receive inflated reviews from certain reviewers or indeed that some reviewers have in the past been proven to take bribes for scores.  The point is Metacritic is hopelessly flawed as a metric for comparison, its not really up for the debate by anyone interested remotely in a proper statistical analysis.  Thats without mentioning the very poor selection criteria for reviews on Meta, the score conversions they do, how reviewers gauge their average rating. 

As you can see thats one paragraph and there is plenty more reasons that can be used for why this isn't a proof but why should I bother trying to convince someone so intent on spinning numbers (the OP not you).  The OP despite what he wrote after he posted the article (which he admits he didn't check or consider the validity of the data) had already made their mind up and this was never a debate in thier eyes, just an excuse to dismiss anyone with a different opinion.

 

I see what you are saying BUT is it not suspicious that Edge's scores are similar to the average trend for 360 yet constantly under the trend for PS3 games? There is no excuse for this, and any reviewer swayed by hype is a bad one.

To simily dismiss every and any trend doesn't make sense.



Around the Network

I wonder, if Edge started to score PS3 games higher than 360 games, would that mean Edge is fair or biased against 360?



Mazty said:
slowmo said:

No point recylcling old arguments is there usually, I've made my point several times already on this site.  This thread isn't bringing new data to the table just the same old data people post packaged in graphs, we don't need the arguments again.  By showing that the OP is basing his entire argument of proof of bias on flawed data isn't unfounded by the way. 

I assume you consider Metacritc is a perfect measure to judge and only the top 30 games a suitable measure though?  Lets not take into account the hype factor that causes many games to receive inflated reviews from certain reviewers or indeed that some reviewers have in the past been proven to take bribes for scores.  The point is Metacritic is hopelessly flawed as a metric for comparison, its not really up for the debate by anyone interested remotely in a proper statistical analysis.  Thats without mentioning the very poor selection criteria for reviews on Meta, the score conversions they do, how reviewers gauge their average rating. 

As you can see thats one paragraph and there is plenty more reasons that can be used for why this isn't a proof but why should I bother trying to convince someone so intent on spinning numbers (the OP not you).  The OP despite what he wrote after he posted the article (which he admits he didn't check or consider the validity of the data) had already made their mind up and this was never a debate in thier eyes, just an excuse to dismiss anyone with a different opinion.

 

I see what you are saying BUT is it not suspicious that Edge's scores are similar to the average trend for 360 yet constantly under the trend for PS3 games? There is no excuse for this, and any reviewer swayed by hype is a bad one.

To simily dismiss every and any trend doesn't make sense.

I don't disagree on that point but it happens all the time.  Heavy Rain is a perfect example of a recent PS3 exclsuive that received undeserving 10's yet people jumped on Edge for not bowing down to the hype and following the crowd.  This is a game I would add that loses all impact on the second playthrough as the chief gaming mechanic is the story and not the actual gameplay elements.  It was released with numerous bugs too I would add that seemingly didn't reduce the score either. 

My point is it's obvious there is some terrible reviewing going on with PS3 exclusives that is raising the Metacritic average in some cases.  When you accept that this could be a serious issue then PS3 reviews on Metacritic could be trending higher thus offsetting the relative balance Edge places on its reviews.

I would add though that reviewing in general is becoming a joke anyway which is why anything that uses Metacritic to prove a point will always have me in opposition.  Arguably the only games this generation that deserved a 10 that I've played are Uncharted 2, Gears Of War, Rock Band 2, Bioshock, Oblivion, Super Mario Galaxy.  I have played the following notable titles that don't deserve a 10 imo: Halo 3, Killzone 2, MGS4, Mario Kart, GTAIV.



I can't believe some people are taking this seriously.........

If you replace a couple of games on the ps3 list and a couple on the 360 list with other exclusives then the differences between relative edge scores for the games and the metacritic scores practically vanish. That is how silly this study is, the creator just chooses games to prove a point.

For example if he had replaced the Formula one game on the ps3 list which edge had scored much lower than the metacritic score (40 to 75)with Siren Blood Curse which not only had a higher metacritic score than the Formula One game, but an even higher score from Edge (78 to 80)! (They gave it a higher score than Metacritic) the graph would start to look vastly different. Replace Yakuza 3 with the vastly superior Uncharted 1 which Edge was also nearly spot on the metacritic with and the difference becomes smaller still. Hell, let me pick a few more games I will 'confirm' Edge has a pro PS3 bias lol :)

The moral of the story is that we prefer random samples to cherry picked data for a reason.



Barozi said:
MikeB said:
I think it was also great an expert group awarded Killzone 2 for its advanced AI last year. Halo 3's AI is rather so-so, but Killzone 2 got a 7/10 and Halo 3 a 10/10 from Edge! LOL

IMO they are shooting themselves in the foot...

I would give Killzone 2 8/10 and Halo 3 10/10, so I don't see the big difference. (Based on Edge's reviewing system)

Same!

 

 

Though I think they rated Halo Wars too low >.<



Around the Network
dsister44 said:
Barozi said:
MikeB said:
I think it was also great an expert group awarded Killzone 2 for its advanced AI last year. Halo 3's AI is rather so-so, but Killzone 2 got a 7/10 and Halo 3 a 10/10 from Edge! LOL

IMO they are shooting themselves in the foot...

I would give Killzone 2 8/10 and Halo 3 10/10, so I don't see the big difference. (Based on Edge's reviewing system)

Same!

 

 

Though I think they rated Halo Wars too low >.<

I think most places did, I loved that game and would have given it 8/10.  I think it was rated harshly by being played by former PC RTS gamers who didn't review it from a console gamers perspective in most cases.  For those concerned about the Halo branding over the game, the story was so far removed from the main franchise that it wouldn't sway you to like it just because it was Halo imo.  Mind you I loved Heavenly Sword too so perhaps my tastes are just weird.



It's very easy to make data say what you want it to say. I can take the same set of games, chop it up by it's development origin and find a new bias for Edge. I can do the exact same study and probably find IGN biased in the other direction. The thing is, it's typically only a handful of games screwing with the results in either direction. It doesn't help that Metacritic was the measure used either. First thing is, we're using Metacritic to compare with Edge, when Edge's scores are inside of every Meta, screwing with the results. Secondly we don't know the weight of each publication comprised in a Metascore. Third, which score is being used comparing the multiplatform Meta's? Each game has two. The discrepancy could have more to do with a very favorable PS3 site like IGN getting a huge advantage in raising the scores.

Edit: Forgot the most important thing also. Edge rates games 1-10. Metacritic scores games 1-100. Anything under a 5 point difference should simply be ignored and washed clean because of an inability to score them the same way Meta does. 



Tag: Became a freaking mod and a complete douche, coincidentally, at the same time.



slowmo said:

I don't disagree on that point but it happens all the time.  Heavy Rain is a perfect example of a recent PS3 exclsuive that received undeserving 10's yet people jumped on Edge for not bowing down to the hype and following the crowd.  This is a game I would add that loses all impact on the second playthrough as the chief gaming mechanic is the story and not the actual gameplay elements.  It was released with numerous bugs too I would add that seemingly didn't reduce the score either. 

My point is it's obvious there is some terrible reviewing going on with PS3 exclusives that is raising the Metacritic average in some cases.  When you accept that this could be a serious issue then PS3 reviews on Metacritic could be trending higher thus offsetting the relative balance Edge places on its reviews.

I would add though that reviewing in general is becoming a joke anyway which is why anything that uses Metacritic to prove a point will always have me in opposition.  Arguably the only games this generation that deserved a 10 that I've played are Uncharted 2, Gears Of War, Rock Band 2, Bioshock, Oblivion, Super Mario Galaxy.  I have played the following notable titles that don't deserve a 10 imo: Halo 3, Killzone 2, MGS4, Mario Kart, GTAIV.

 

I'd say though that the 360 has just a large, if not larger, amount of undeserved high scores, such as Halo 3 etc.

I think what is bizarre is that Edge will be willing to go hype-out crazy with some games like Bayonetta and Halo 3, but be very reserved when it comes to PS3 titles. Granted that the PS3 scores may actually be closer to what people believe the game may deserve, but it still shows bias towards one console over another, which really brings into question the integrity of the magazine.

Though as you said reviewing is becoming an utter joke with many, many games getting rediculous praise such as Halo 3, Killzone 2 (great multiplayer, crap single player), the wii in general. GTA4, MW2 etc. I'm willing to bet though Starcraft 2 will show all the poor journalists out after playing the woeful beta though, but I digress.

I think people would be much better off ignoring reviews all together and just rent games.



All I can see if potential proof that, with regard to the top titles on each platform, EDGE reviewers tend to score lower (exceptions aside obviously) and that they inherently prefer the 360 titles.

That isn't bias in a conspiracy sense - they deliberately mark down PS3 or inflate 360 - so much as a lapse in maintaining outright review neutrality - i.e. reviewer genre/gameplay mechanic preference has crept into their scores a little and the reviewers simply prefer, as an example, Mass Effect 2 over God of War 3.

This happens all the time with reviews, though, and is particularly rife in videogames reviews, so I think all anyone should take from it is an understanding that, a few exceptions aside, EDGE slightly prefer the titles on 360 to PS3 from that sample and in general does seem to score more harshly than other sites.



Try to be reasonable... its easier than you think...

Onyx, chopping the games up by development origin doesn't make any sense, because that actually relates to the quality of the game. Who developed a game has a direct impact the the quality and genre of a game, what console it's on doesn't. A non factor like what console a game is on shouldn't show up too heavily in the averages, and over time (And with enough games) it should totally disappear.

Otherwise, I agree with you, and I think this study is very flawed.