Quantcast
Edge vs. Metacritic (Bias Confirmed!!!)

Forums - Gaming Discussion - Edge vs. Metacritic (Bias Confirmed!!!)

Every time EDGE magazine reviews a Playstation 3 exclusive, they are accused of bias by PS3 supporters, who are then mocked as fanboys who aren't satisfied with anything other than perfect scores for their beloved exclusives. EDGE are known to be harsh graders, and some respect them for using the full 1-10 scale instead of using 7-10 scale like the mainstream gaming press. The counter argument is that EDGE reviews every game harshly and it's pointed out that for every PS3 exclusive that gets low scores like Heavy Rain, there is an equivalent 360 exclusive.

 To investigate whether EDGE is just harsh overall, or if it truly judges and scores games based on their platform, I made a comparison chart of the top 30 games from each platform, as well as the top 30 multiplatform games.

 To make such comparisons, we need an unbiased standard, so we can measure EDGE scores against it. While we don't live in a perfect world and there is no universal metric that tells how good a game is, the closest to such a standard is Metacritic. It is a weighted and normalized average of all sites that review a video game. Metacritic says that it assigns different weights to different critic sites as well as making sure that the outliers don't affect the average "metascore" too much.

 For this analysis, I found the top 30 games for each platform and compared their metascore to the EDGE review score to find the difference. I've limited my analysis to the top 30 due to time constraints and because the top 30 gives a very broad analysis of exclusives available for each system while filtering out the real duds. The #30 exclusive games for both platforms have metascores in the low 70s.

 The ground rules used for counting and determining exclusives were as follows:

 1) Only disc based games were counted, downloadable games, expansion packs, and DLC were not included. The most important reviews are for new release disc based games, since they influence the buying decisions greatly.

2) Games that EDGE did not review were not counted, such as God of War collection on the PS3, MLB The Show series, other sports games, etc.

3) A game that's on the PC and Xbox 360 is considered an exclusive.

4) Games that came out on 2005 and early 2006, when there was no PS3, were counted multiplatform if they were also released on PS2, since it was the 360 competition for a year until the PS3 release. A such example is Call of Duty 2, which came out on 360 and PS2. Since Call of Duty has never been a 360 franchise, I felt this was the right thing to do. On the other hand, a game such as Dead Or Alive 4, was not on any Playstation platform, so it was counted as a 360 exclusive.

5) When comparing the score of a multiplatform game, the version that EDGE reviewed was taken as the baseline. For example Bayonetta has a metascore of 90 for the 360 version and 86 for the PS3 version. Since EDGE reviewed the 360 version, the metascore was taken as 90.

6) Metascores can change, albeit slightly, over time, so this list of the top 30 games is only accurate as of March 13, 2010.

 So without further ado, let's start with the the EDGE vs. Metacritic comparison of the top 30 multiplatform games:

As we can see from the chart, EDGE's reputation as a harsh critic is much deserved, with the EDGE scores (red) being lower than the metascores (blue) most of the time, although some exceptions like GTA4, The Orange Box and Bayonetta, where the EDGE score was a perfect 10.
Next, we compare the top 30 Playstation 3 exclusives' metascores to their EDGE scores:
We can see that only LittleBigPlanet, Demons Souls and Disgaea 3 have exceeded their metascore on Edge, while Buzz Quiz TV matched it. Everything else scored significantly lower. EDGE is harsh indeed.
Finally, we compare the top 30 Xbox 360 exclusives' metascores to their EDGE scores:
It's immediately apparent that many Xbox 360 exclusives received much higher scores from EDGE compared to multiplatform games or especially Playstation 3 exclusives. The blue area in the Xbox 360 graph is much less than the other two graphs.
For the final and the most crucial graph, I calculated the delta's (score differences) between EDGE and Metacritic for each game, and plotted a running average chart:

We can see from the above chart that EDGE, scores 360 exclusives consistently about 4 points below their Metacritic average, scores multiplatform titles consistently about 8 points below their Metacritic average, and scores PS3 exclusives about 12 points below their Metacritic average. The top 10, 20 and 30 game score difference averages are plotted in the chart.

 1) Top 10 games: These are the cream of the crop for each platform. These are the Halos, Uncharteds, COD4's, GTA4's that sell millions of copies and systems. When comparing the top 10 scores, EDGE gives 360 exclusive games 3.7 points less than their Metacritic average, multiplatform games 6.9 points less than their Metacritic average, and PS3 exclusives a whopping 11 points below their Metacritic average.

 2) Top 20 games: When comparing the top 20 scores, EDGE gives 360 exclusive games 4.6 points less than their Metacritic average, multiplatform games 7.6 points less than their Metacritic average, and PS3 exclusives 11.2 points less than their Metacritic average.

 3) Top 30 games: This list includes games ranging from Mass Effect 2 to Folklore, and a broad range of scores. However, the overall picture remains the same as EDGE gets a little bit harsher to the "above average" games. When comparing the top 30 scores, EDGE gives 360 exclusive games 4.8 points less than their Metacritic average, multiplatform games 8.5 points less than their Metacritic average, and PS3 exclusives a whopping 12,5 points below their Metacritic average.

 The consistency of the score differences is remarkable. Whether comparing the top 10,20 or 30 games, the 360 is always scored higher by EDGE, followed by multiplatforms, and followed by PS3 exclusives in the last place. The lines are relatively straight and flat, with no lines crossing each other or switching places after the top 10. If EDGE really scored games irrespective of their platforms, we'd have expected the three lines to cross each other and be all over the place, but this is not the case. There is a strong pattern of discrimination for 360 exclusives and against PS3 exclusives.

 While people may have different opinions on a few games, and could indeed review them differently and get different scores, 30 games consistently scoring lower on the PS3 and higher on the 360 means only one thing:

 EDGE is biased to give 360 exclusives higher scores and PS3 exclusives lower scores. Their scores should not be taken at face value, as the platform unfortunately influences a game's score on EDGE. EDGE magazine has well written reviews, but their journalistic integrity is questionable.

Source: http://edgevsmeta.blogspot.com/

Yes the source is blog, but someone took enough time and effort into it, so it's worth to read through. I haven't checked all the evidence myself, but from a brief look it seems legit.



MY HYPE LIST: 1) Gran Turismo 5; 2) Civilization V; 3) Starcraft II; 4) The Last Guardian; 5) Metal Gear Solid: Rising

Around the Network

Inherently flawed study, as it's only rating the top 30 games on each (plus multi). You can't claim institutional bias when you're looking at ~5% of the games on these platforms, it's borderline irresponsible. "Journalistic integrity" indeed, lol.



Dude I hate edge too and their poor excuse of we like innovation(MW2 - 9 LMAO!!) but I dont make hate threads about them evry 2 sec.



jarrod, that would be true if the case study was of the industry as a whole. but a vertical slice, and one taken from the very best the industry has to offer, is an accurate representation of a bias in review score. i wonder if i could use this info to create an equation that could guess edges next score lol.

however aragod I believe you should take a vertical slice from a development perspective. for example only use first(possibly 2nd) party studies games from sony and microsoft compared to multiplatform games.

anyway , good read.



jarrod said:
Inherently flawed study, as it's only rating the top 30 games on each (plus multi). You can't claim institutional bias when you're looking at ~5% of the games on these platforms, it's borderline irresponsible. "Journalistic integrity" indeed, lol.

Guess not everyone has the resources to put hundreds of hours into this. Anyway even from these "30" titles, you can see that the graph lines are pretty much static, which can say a lot. These graphs cover pretty much every notable exclusive or release, so we are talking about the "important" titles.

If you care to add more titles, please do so. Until than, you haven't proven any flaw. It's like crying about public opinion research that only covers 50 000 people out of 50 000 000.



MY HYPE LIST: 1) Gran Turismo 5; 2) Civilization V; 3) Starcraft II; 4) The Last Guardian; 5) Metal Gear Solid: Rising

Around the Network

>>> BTW this isn't my work, I found it on some N4G news article yesterday, just reposting. I can't take credit for it, even though I was planning to do something similar in the future.



MY HYPE LIST: 1) Gran Turismo 5; 2) Civilization V; 3) Starcraft II; 4) The Last Guardian; 5) Metal Gear Solid: Rising

Yet more comfirmation of what a steaming pile of crap Edge is.  I checked this out before and ended up coming to the conclusion that they were a bunch of Halo fanboys, who generally despise any FPS on the PS3.  http://www.vgchartz.com/forum/thread.php?id=101000



The dude abides   

The chart speaks value.

Is Edge a 'harsher' review than most?? The chart said yes
is Edge much more "harsher" on PS3 review??? The chart said "HELL YES"

now, it's up to Edge to justify their reviews.



Soriku (Feb 10/08): In 5 years the PS3/360 will be dead.

KH3 bet: "If KH3 comes to Wii exclusive, I will take a 1 month of sig/avatar by otheres open a thread apologize and praise you guys' brilliance." http://vgchartz.com/forum/thread.php?start=50&id=18379
Original cast: Badonkadonkhr, sc94597 allaboutthegames885, kingofwale, Soriku, ctk495, skeezer, RDBRaptor, Mirson,

Episode 1: OOPSY!
selnor
: Too Human I even expect 3-4 mill entire life and 500,000 first day. GoW2 ( expect 7 - 9 million entire life and over 2 mill first day), Fable 2 (expect 5-6 million entire life and 1.5 mill fist day) BK3 (expect 4 - 5 mill sales entire life and 1 mill first day).. Tales/IU/TLR should get to 2 or 3 million! post id: 868878
Episode 2:
Letsdance: FFXIII (PS3+360) first week in NA = 286K
According to pre-order rate in week 13 (post id: 2902544)

exactly aragod he has no basis for rebuttal, his argument contradicts himself. talk about journalistic integrity, when he has formulated an opinion about your research purely based on a misinformation. you took a vertical slice of a section that would have the greatest disparity, a smart move. some people just like to disagree and argue, anthropologically speaking , this is a defense mechanism, therefore we can assume he is an edge fan... or is just ignorant.



jester2358 said:
exactly aragod he has no basis for rebuttal, his argument contradicts himself. talk about journalistic integrity, when he has formulated an opinion about your research purely based on a misinformation. you took a vertical slice of a section that would have the greatest disparity, a smart move. some people just like to disagree and argue, anthropologically speaking , this is a defense mechanism, therefore we can assume he is an edge fan... or is just ignorant.

It's not my research :), I can't take credit for someone else's work.

I've just decided to post it here, since it might interest a lot of this community.



MY HYPE LIST: 1) Gran Turismo 5; 2) Civilization V; 3) Starcraft II; 4) The Last Guardian; 5) Metal Gear Solid: Rising