Region of origin is what I meant, not the actual development house.

Oh, my bad.

I haven't read all of the responses but here's some observations:

A 5% slice can be an adequate sample, but we don't know for sure because no one has done any statistical significance calculations on the results.

There should have been a few control review sites selected. Any random sites as long as they weren't XB / PS affiliates. As the study is only comparing average difference between PS3, multi and 360 scores it doesn't matter if the site has a reputation for harshness or handing out 10/10's like candy, because the important information is whether there is a bigger difference between that site and meta for the various platforms.

Assuming statistical significance of this analysis, there is clearly bias in the Edge reviews, but you have to determine what sort of bias. Is it platform bias or is it game genre bias? is it malicious or incidental?

Edge only gives whole number scores, whereas Meta's % equates to 0.1 increments on a 1-10 scale. This itself introduces inherent bias into the system. Any score on Edge must be regarded as equivalent to a meta score between -0.5 and +0.4 of the Edge score. This accounts for the scoring system bias to allow for a proper analysis of significance. So an Edge score of 8 is equal to any metascore between 75 and 84. Any game that falls into this zone of fuzziness should actually be regarded as equal between Edge and Meta. I suspect that if you take this systemic bias into account then 360 average difference between edge and meta will be very small or zero because it's the average 4.8 point difference in the raw data is pretty much inside that zone of systemic bias. I imagine there will still be a large difference for PS3 and for multi's.

I think to determine whether there's any genre bias going on you'd need to take the top games by genre and see how Edge and Meta play out. Could be that genres that are more abundant on PS3 are genres Edge doesn't like much, whereas the genres on 360 are genres Edge likes a lot.

Didn't Edge come right out and say that everyone should buy a 360 if they wanted to save gaming. I recall this from a few years back.

I couldn't say for sure as I'm not that bothered to check all their scores but yes Halo 3 with hindsight was a 9/10 game due to the poor single player campaign which didn't improve on the criticism Halo 2 was given IMO.  Bayonetta from what I've read could be deserving of a 10 but I've not played it so cannot comment.  At the end of the day I read Games TM for my reviews as they generally are spot on with their opinion in comparison to my tastes.  This is the main issue I take with people shouting bias, everyone has different tastes and instead of trying to make publications toe the line to your opinion just seek out and read ones you enjoy and suit your tastes instead. 

I've long held the view that reviews would be better for removing the numbers and forcing people to actually read the text that a writer spends his precious time creating for their enjoyment.  I bought Assassins Creed 1 on the basis of the text in a review that only gave it 4/10, the criticisms that were cited at the game I knew wouldn't bother me and so I'm glad I took that time to read it.  It didn't make that 4/10 a bad review or biased though as it was just a writer giving an honest opinion of what the game meant to him, obviously the "flaws" bothered him more than me.

the bolded part... abso-fucking-loutely.  The whole quick score, 60 seconds news, soundbite, etc. trend for information is just terrible.

I read the text of film reviews or game reviews if I actually want to understand their view of the game.  The score I treat with suspicion as to be honest, even the best reviewers can't help but be inconsistent in how they apply them so far as I've observed, and of course they are almost always taken out of context.

I think another thing people need to bear in mind... Let's say the average metacritic score for the 360 top 30 and the PS3 top 30 are roughly the same... the 360, with only an install base of a few million more, probably has double or more the sales of that top 30. Somehow, an equally reviewed set of games with a fairly similar install base have a massive disparity in sales. Certain games appeal to more tastes and in NA at least (granted, the console sales parity isn't present here), those tastes lean towards the 360. Fact is, in general, some people like shooters with a focus on multiplayer gameplay the most, and review them the highest in general. Guess what? this will massively skew their average reviews in 360 vs PS3 games. A lot of people call Halo 3's 10 overrated because of the single player campaign. Most 360 players i know didn't love it - but guess what game has spent the most time in their disc drive? If 10/10 meant absolute, total perfection, then no game would earn it, but quite frankly when a fantastic game has hundreds of hours of replay value, and really certain aspects which blow all or most of its competitors out of the water, it can earn a 10 with obvious flaws. (I.e. Mass Effect 2's massive slew of 10's - it's not perfect, but it deserved them)

I agree that a 10 shouldn't ever be considered perfect or no game could be awarded it, on any scale every mark must be achievable!  If you put 100's of hours into the multiplayer than Halo 3 was worth the 10, if you played the single player campaign through once and tried online and that was it then its obviously not worth a 10.

I'm so confused, I thought metacritic was garbage? Is it okay to use Metacritic when proving a pro-PS3 point? I remember a thread that said somthing like "360 has 50% more AAA exclusives than PS3!" and used metacritic as justification. There were like 2 pages of "LOL Meta is garbage" replies and then it was locked.

I don't like moving the goalposts like this.

You get used to it.