Quantcast
How Trustworthy Are Reviews?

Forums - Gaming Discussion - How Trustworthy Are Reviews?

I would say not at all. For this reason...

Game reviewers salaries are paid mostly on advertising.

As such, if your a smart advertising company your going to want reviewers that review your big games high. Not saying the reviewer has to be corrupt... but certain types of reviewers likely will be picked.


It's the problem with Videogame reviews coming out in specialty magazines.

It's different from things like movie reviews which are paid for by magazine advertising of all kinds.



Around the Network

I only read reviews to get an impression of what the game is about, but I never let a review decide if I buy a game or not.



Onyxmeth said:
reask said:
RolStoppable said:
Most reviews are less trustworthy than I am and that says a lot.

People who actually paid full price for a game give you a much better opinion than most of the guys who review games for a living and get free copies along with some bribe.

I agree with you on the gamers prespective rol.

I just find that the really big hyped games are becoming like hollywood blockbusters.

What I mean is the pressure on the sites is too much for them to give  anything but a glowing review for said game.

So how do you distinguish the good from the  bad?

Even taking it to a different level us bloggers are getting as bad.

I mean if you support a particular console you will go to hell and high water to defend it and its exclusives.

I do not disclude myself of been guilty of that on ocassions.

(Dont know if that word disclude is legit but I like it.)

The hollywood blockbusters line isn't very good, since movie reviewers have tons of integrity and will slam the shit out of any movie they want to, regardless of it's budget or status. If you look at the Oscars and the movies that get represented, it would be the equivalent of games like Demon's Souls and Scribblenauts being top contenders for GOTY instead of Uncharted 2 and MW2.

That's because movie critics aren't paid by the movie industry like game reviewrs are... since if you got to game reviewer sites...  you see ads mostly for games.

Magazines and Newspapers, well they sell adds to everbody.

Though even then there is a lot of worry that Newspapers will become more compromised and biased as they move away from subscriptions and a forced to rely soley on advertising.



madskillz said:
.jayderyu said:
Reviewers get paid by advertisers. Big game companies are advertisers. Reviewers get free gamers from game companies. If advertising game companies don't like the reviews because, they put a strangle hold on the reviewer. I'm sure if you search around enough you can find enough cases of reviews being pulled because the company doesn't like the review.

Reviewers make good reviews by the companies that pay them. This is why some companies don't get as favorable reviews. They don't pay enough or/and the reviewer just doesn't spend as much time. There are of course where games are just bad and theirs little a reviewer can do about it.

Uh, no. I get paid by the Hearst Corporation not an advertiser. Where are you getting your information from? And free games? Not quite. You do realize the game is free, but the labor isn't? And if you can't write worth crap, you can ask all day and they'll never send you a game.

Reviewers have an obligation to report the truth. Sure, some reviewers overlook ethics, but a lot don't and just want to inform readers.

Reviewers make good reviews because they have literary skills. They have a very good command of their language and know how to make a story flow.

You write at a Newspaper right?  So if you slam an activision game you don't have to worry about advertising pulling their adds which account for like... 40% of your companys income.



for me, i buy the games that i want to buy unless the reviews are beyond horrible. that being said, i can be swayed by reviews to buy certain games. thats how i first got into resident evil 4 and gears of war 1. prior to them being released, i had zero interest in them. to be honest, i hoped gears of war would suck ass because at the time, i was strictly a nintendo fan, but when i heard it was awesome, i got it and a 360 as well. now it's my most played game even 3 years later. re4 was the same way. had it not been for the amazing reviews, i wouldn't have even considered it. i never liked the re franchise until that game, but i'm glad i saw the reviews for it. my favorite game from last generation



Around the Network
Torillian said:
Avinash_Tyagi said:
Torillian said:
Avinash_Tyagi said:
Torillian said:
Avinash_Tyagi said:
 


  Then explain why reviewers went to an acitivision showing of MW2 to do the review, all expenses paid.

 

Then notice that the user reviews of MW2 are far lower than the critics reviews of the game

The reason for that is simple.  MW2 took away alot of things that PC players love, and therefore they downrated it whereever possible like crazy.  Done and done.

LOL, or maybe its because the game was overhyped and under delivers, the PS3 version has 723 ratings, and the 360 has 1180, its not just people running around downvoting it because they didn't like the PC version.

So you don't trust reviewers, but you trust random fanboys on the internet?  They don't even have to justify what their doing, just click a button.  Those 1903 ratings could all be done by 6 people.

When most games are reviewed, the user reviews tend to be in the same ballpark, sometimes a little higher or a little lower, these many poor reviews of a game, which the critics supposedly rated so highly sends up warning flags, and its not the first I've heard of people saying the game was overhyped.

Sure it could be just a few fanboys dragging it down, but unlikely I've seen a lot of disappointment from this game, not just on Metacritic, but acorss a bunch of forums, and blog posts, etc.

And I've seen a bunch of people very happy with their purchase.  Guess it's your fake data against my fake data.

And since when do you care about what people think?  I thought you were the ultimate proponent of capitalism like Colbert.  MW2 sold like crazy, therefore it must be great.

Actually I said that sales show how much a game appeals to the market, and that low sales show that games have low appeal and that very high sales indicates that the market finds a game to be more appealing, and I did say that I'm not interested in the reviews of the "official media", but user reviews are actually important, as word of mouth can impact sales, if people hear from others that a game is bad, then the appeal will die out, and the sales will drop off.  The game has sold well, but has yet to break 10 million, or even get close to the likes of a game like Mario Kart Wii, and bad word of mouth is one thing that could easily cause its sales to drop off in the coming weeks.

also user reviews aren't fake data, I can show you the low metacritic user reviews of the game to prove my point.



 

Predictions:Sales of Wii Fit will surpass the combined sales of the Grand Theft Auto franchiseLifetime sales of Wii will surpass the combined sales of the entire Playstation family of consoles by 12/31/2015 Wii hardware sales will surpass the total hardware sales of the PS2 by 12/31/2010 Wii will have 50% marketshare or more by the end of 2008 (I was wrong!!  It was a little over 48% only)Wii will surpass 45 Million in lifetime sales by the end of 2008 (I was wrong!!  Nintendo Financials showed it fell slightly short of 45 million shipped by end of 2008)Wii will surpass 80 Million in lifetime sales by the end of 2009 (I was wrong!! Wii didn't even get to 70 Million)

Avinash_Tyagi said:
madskillz said:
Avinash_Tyagi said:
madskillz said:
.jayderyu said:
Reviewers get paid by advertisers. Big game companies are advertisers. Reviewers get free gamers from game companies. If advertising game companies don't like the reviews because, they put a strangle hold on the reviewer. I'm sure if you search around enough you can find enough cases of reviews being pulled because the company doesn't like the review.

Reviewers make good reviews by the companies that pay them. This is why some companies don't get as favorable reviews. They don't pay enough or/and the reviewer just doesn't spend as much time. There are of course where games are just bad and theirs little a reviewer can do about it.

Uh, no. I get paid by the Hearst Corporation not an advertiser. Where are you getting your information from? And free games? Not quite. You do realize the game is free, but the labor isn't? And if you can't write worth crap, you can ask all day and they'll never send you a game.

Reviewers have an obligation to report the truth. Sure, some reviewers overlook ethics, but a lot don't and just want to inform readers.

Reviewers make good reviews because they have literary skills. They have a very good command of their language and know how to make a story flow.


  Then explain why reviewers went to an acitivision showing of MW2 to do the review, all expenses paid.

 

Then notice that the user reviews of MW2 are far lower than the critics reviews of the game

I got invited to the same event - it wasn't paid - and even if it was, I would refuse. That really, really blurs the ethics line.

If you are getting goods from a person/company and the like - and things with the person/said company go south, you'll have in the back of your mind that 'Hey, they hooked me up with a favor.' Accepting gifts is forbidden. A T-shirt? A pack of energy drinks? That's minute. However, an all-expense paid trip - if you are a journalist, you are on the company's dime to be fair and balanced.

As I stated, anyone with access to a blog can claim to be a journalist. However, unless you've had classes on libel, defamation, ethics and the like, you're just fooling yourself.

I have seen tons of instances where gifts/perks have more than swayed people's opinions on matters.

Except Joystiq Admitted that they and others accepted it and that activision paid for it:

 

Disclaimer: The preceding review is based on an event organized and paid for by Activision, in which media outlets were provided hotel rooms, each equipped with an Xbox 360 and copy of Modern Warfare 2. As this was Joystiq's only opportunity to review the game in advance of its release, we willingly deviated from our standard policy of not accepting accommodations and used the room. We did so because we felt that participating in this event best served the interest of our readers. 

 

Seems to me they crossed the ethics line there

 

http://www.joystiq.com/2009/11/10/review-call-of-duty-modern-warfare-2/

That definitely crosses the line. Now, all Activision has to do is press them "What about that free trip" and they will bend. Bad move.

@ Kasz - ads make up about 30-40 percent of the newspaper income, but Activision - and most video game companies - don't advertise with newspapers anyhow. A mag, yeah. A newspaper? No. However, the print version is what's king. To get into print - with over a million daily readers in print - and even more online - for just a $60? You can't buy that kind of advertising for that cheap.

I am going to be real and honest in my review of games. I can say that no company can say I laid ethics aside just to get a game early or the like. I'll just be content getting the game the day of release and playing it and posting my review then. If it's crap, it's crap. I am not going to hold my tongue in fear of never getting another game from folks. That's why Sony isn't sending me games. I am calling it like I see it. Even Valve sent me a review copy of L4D2, a game I had blasted for having *racist undertones,* which I admitted was wrong in my review of the game.

A professional is going to stick to the ethics standards come hell or high water. We are supposed to offer an honest view of products. The day we don't do that, we are just an extension of a game publisher/devs ad and marketing department.



Whether I'm reading Metacritic reviews or user reviews, I have to weed through the trolls either way, but I find it easier to weed through the trolls with users. They're not paid, and they're usually upfront about their bias. They'll tell you if it's their favorite system, favorite controls, favorite genre, or favorite franchise, so you know where they stand, and see how the game fits into their tastes (if it does). And it's usually pretty obvious if they have an agenda.

With paid reviewers, they're an arm of the marketing branch of the publishers, and it is a well known fact that they get paid and bribed to give good early reviews so that the Metacritic number is really really high on launch day, before all the real honest reviews come in and knock it down a few pegs. So with those guys, the only ones I actually trust are VGChartz, for 2 reasons: the explanation loadedstatement gave above, and because I've met a couple of them in real life, and discussed video games with them in person, and trust them.

Outside of VGC and actual gamers (mostly members here, or friends of mine in meatspace), I would trust a reviewer like I'd trust an orc.



As a reviewer myself, this is how I see it:

You write as a gamer and for gamers. That's why one has to place one's self in the shoes of a purchasing gamer. What would I want if I paid full price for this? What information does the average need from my review to know whether they want this game as well? Other than that, a reviewer basically has nothing to really deal with.

The major issue many people have unfortunately, is that they blindly follow that tiny thing, whether it is a 87 or a 90. Many sites usually give numbers merely because it's what everyone wants, a rating of some sort. I could write the best possible essay on why this or that game deserves all the sales it gets, but if I give it 86 for example, people will focus solely on that number, without actually looking at what I write and focussing on that instead.

So it's basically a exchange. Reviewers need to realize they write to inform, not to hype with no reason. Readers need to realize that a number often merely accompanies the information and that they need to look for the good reviewers out there, because not every reviewer is worth reading.



Im really not sure how other websites go through reviewing a game. But I can tell you how VGChartz reviews games. First of all, the reviewers have no contact with the company that made the game they are reviewing. Dan and I do all the talking to game companies regarding reviews. Second, there IS a desire to get a new game's review out quickly, but there is no penalty for taking too long. Every single review coming out of VGChartz has been written carefully and is that person's honest opinion of the game.


Beautiful :) That's how we do it as well