Game Score-to-Sale Theory

[sinha]

New York Times: Gamers Agree With Critics

This spring a game-company analyst at UBS named Ben Schachter published a report with a subversively innocuous title: “Does Video Game Quality Impact Sales?”

To find out, Mr. Schachter and his team reviewed data for more than 1,500 games released from 2002 to 2005, looking for correlations between sales and the games’ scores on Gamerankings.com, which compiles reviews from many publications and Web sites into a numeric verdict between 1 and 100. Their conclusion was simple: Top-rated games sell a lot more copies than bad ones.

...

To conduct apples-to-apples comparisons, I used the scores at Metacritic.com, which pulls together multiple reviews to give media products a score ranging from 1 to 100 that reflects the collective wisdom of professional critics.

The data was clear. The Top 10-selling games of last year — including titles like Gears of War and Guitar Hero 2 —had an average Metacritic score of 87.5. Only one of the top-selling games scored less than 80. (More about that later.) Meanwhile, the Top 10 box-office films of last year — including titles like “Pirates of the Caribbean: Dead Man’s Chest”’ and “X-Men: The Last Stand” — collected a poor average score of 62.9.

...

“When something costs $59 and you’re 18 years old, it’s not like going to the movies,” said Jeff Brown, vice president for corporate communications at Electronic Arts, the No. 1 game publisher. “Video-game consumers are the single most sophisticated shoppers in the entertainment industry, despite their age and what you might think. People go to a movie or buy a book on a whim. But buying a video game is a much more methodical and judicious process. Next time you’re in an airport, look at how many magazines are dedicated to video games. And they’re not writing about the sex lives of game designers. They are writing about the content.”

(from Kotaku and Game Set Watch)

--------------------------------------------------------------------------------------------------------------------------------------

 

Survey: Game Score-to-Sale Theory Disproven (2005):

The new issue of the Susquehanna Financial Group's Video Game Journal contains an in-depth analysis of the correlation, or lack thereof, between video game review scores and the resulting performance at retail. According to the authors of the study, Jason Kraft and Chris Kwak, there is little relationship between a game's critical and commercial reception.

Survey: Game Score-to-Sale Theory Again Disproven (2006):

In fact, the study found that by increasing the sample, "only 15.8% of the movement in game unit sales ... can be explained by movements in game ratings," a figure down from its original 17.3% from the sample of 275 games.

To help try and trap for other factors that might explain the findings, the authors looked specifically at franchise correlation, including sequels and new franchises, and selected data from other consoles, in all cases failing to find sufficient direct score to sales matches outside extreme isolated cases.

In conclusion, the authors state, "a theory (that game ratings matter) that fails under scrutiny is accepted as conventional wisdom. Conventional wisdom is wrong. And we have not even addressed the causation argument – that a higher rating causes a game to sell better. There is no reason to argue causation, because while correlation does not equal causation, the absence of correlation means no causation in our case."

--------------------------------------------------------------------------------------------------------------------------------------

FYI, I tried to look at Metacritic scores and game sales in a previous post.