Edge vs. Metacritic (Bias Confirmed!!!)

[binary solo]

I haven't read all of the responses but here's some observations:

A 5% slice can be an adequate sample, but we don't know for sure because no one has done any statistical significance calculations on the results.

There should have been a few control review sites selected. Any random sites as long as they weren't XB / PS affiliates. As the study is only comparing average difference between PS3, multi and 360 scores it doesn't matter if the site has a reputation for harshness or handing out 10/10's like candy, because the important information is whether there is a bigger difference between that site and meta for the various platforms.

Assuming statistical significance of this analysis, there is clearly bias in the Edge reviews, but you have to determine what sort of bias. Is it platform bias or is it game genre bias? is it malicious or incidental?

Edge only gives whole number scores, whereas Meta's % equates to 0.1 increments on a 1-10 scale. This itself introduces inherent bias into the system. Any score on Edge must be regarded as equivalent to a meta score between -0.5 and +0.4 of the Edge score. This accounts for the scoring system bias to allow for a proper analysis of significance. So an Edge score of 8 is equal to any metascore between 75 and 84. Any game that falls into this zone of fuzziness should actually be regarded as equal between Edge and Meta. I suspect that if you take this systemic bias into account then 360 average difference between edge and meta will be very small or zero because it's the average 4.8 point difference in the raw data is pretty much inside that zone of systemic bias. I imagine there will still be a large difference for PS3 and for multi's.

I think to determine whether there's any genre bias going on you'd need to take the top games by genre and see how Edge and Meta play out. Could be that genres that are more abundant on PS3 are genres Edge doesn't like much, whereas the genres on 360 are genres Edge likes a lot.