Why Metacritic doesn't matter

Monday 23-Feb-2009 6:04 PM Do review scores affect sales?

www.cvg.co.uk

There is a prevailing theory that review scores affect sales. It is certainly true that there is a high correlation between aggregate review ratings, like those found on Metacritic.com, and sales; games rated over 90 percent tend to record higher game sales than their lower scoring peers. The question that was posed to the developer panel at DICE Summit 2009 was whether they designed games with reviewers in mind. The answer was an overwhelming no.

"The first person you have to design for is yourself," says Danny Bilson, creative director at THQ. He acknowledges that Metacritic is a powerful force. He's found that he can game Metacritic by improving what he calls "linear assets" that don't affect the actual code, which the team spends most of their time writing. By that he means offering up a better story, better voice acting, and better music. Reviewers have to write, says Bilson, so give them something to write about. By paying attention to such things, you can gain a five to ten point bump because in Bilson's eyes game reviewers care more about story than consumers.

The consensus among panelists was review scores in and of themselves don't matter. Fingers pointed to the success of games like Wii Fit that scored in the 60 percent range. Reviews of such games just aren't relevant to their audience, says Rich Hilleman, chief creative director at Electronic Arts. Social gamers are not the people who are going to be reading hardcore game magazines.

No, the issue isn't the impact of the score on sales, it is how your company uses them, says THQ's Bilson. They are either a touchstone or they are a weapon. Companies use Metacritic scores to project game sales. When teams pitch a game, they're asked to estimate the score. Why would a development team ever shoot for anything less than 100 percent, he asks. The problem is that these projected scores determine game budgets and marketing spends.

Metacritic scores are trailing indicators. Chris Taylor, chief executive of Gas Powered Games, doesn't need a Metacritic score. He knows if he put out a "shitty game." Developers get that information from retail, he says. If it sells well it was good, if it didn't it was bad.

The worst part about these reviews, says Mona Hamilton, SVP of marketing at Capcom, is they are suggestive--and their scores often depend on how drunk you got the media at a Street Fighter event.

Another problem with Metacritic scores, says Bilson, is they aren't weighted. Reviews from Time Magazine are treated the same as some kid's blog. A low score from a "couple weird sites" can significantly drag down average review scores. [Metacritic's website actually notes that scores are weighted. Thanks to readers for pointing this out.]

Hilleman suggests that game developers don't need Metacritic at all. With telemetric data, they can see what is wrong with all of their products--the answers are in the data. Community, he says, is the most important thing.

--------------------------------------------

i also say no.

ive posted many times before if game:A gets a score of 90 based off 55 reviews, and game:B scores 87 based off 80 reviews. it isnt hardly fair at all.

plus many reviews are added from blogs and many times from outlets who dont even review games as such.

do you think metacritic matters to you as a gamer.

what a pointless post, you mean better games sell better then crappy ones???!?!? How can that be? And you mean developers dont really like people passing judgement on their masterpiece?

the last line was an interesting quote, "Community, he says, is the most important thing." Does he not realize the community is where all this metacritic reviews come from?All parts of the community, even Time magazine.

my personal point was aggregated scores would only be fair if they have the same amount of reviews to make up the final score.

the rest ive just posted the article, take from it what you will.

imo many many reviewers that pop up on metacritic you never hear of them.

I do not care for metacritic, MGS4 has a lower average than LBP but I prefer MGS4. Tekken 5 DR my all time favourite fighter is lower than Tekken 3, But i prefer Tekken 5 DR.

phantasy star universe has a low metacritic, yet ive spent more time playing that game than any other game this gen.

Skeeuk, I think you are arguing against the use of Metacritic as a fanboy weapon, and if that is your point, I absolutely agree.

I also agree that critiques of any type (film, book or games) are at best a possible predictor of how much you would like a game, not an absolute rule.

That being said, meta-analysis of multiple video game critics does help to eliminate bias present in ANY single critical editorial. EVEN if 30 reviewers were biased TOWARDS a 360 or PS3 game and only 10 were not, those 10 do help normalize the meta score. Furthermore, as the sampling number increases the normalization SHOULD increase (I get his point about the legitimacy of those sites that are polled for metacritic though).

Personally I only use Metacritic when I am really on the fence about purchasing a game and want to see some general consensus, or to refute ding-dong fanboys who spout metacritic scores themselves.

I use metacritic as a source for finding out whether a game is shitty or universally acclaimed. But that is pretty much the only thing I use it for. The site was created with the purpose of giving you a rough estimate of what reviewers are thinking. There are games rated above 90 on that site that I seriously do not enjoy. On the same token, there are times when I enjoy games that are rated low. It is extremely subjective

Read up a little bit on their system of rating games, to me it sounds like someone came up with it when they were on crack. The number of reviews, as well the sources used for the score differ very much from game to game.

The only reviews that matter to me are specific detailed reviews. It would be stupid to buy a game that you know little about. So the more detailed a review is in regards to the game's features, the better informed I'll be as a gamer as what the game is about. And then how reviews judge the quality of those features comes next. That's when I consider the concensus of those reviews from a good pick of detailed reviews. Nothing to do with Metacritic. So in short, the answer is a legit "no."

Misleading title - the article actually says that metacritic does matter and does affect sales. It's just that it also says that metascore doesnt matter to developers.

And I agree with Chris Taylor. I dont buy into the "this game sells on its name alone!!!" bullshit. If the game sells well then it's a good game, and if you think otherwise then that only means that this game is not for you.