Short Form:
What Reviews Are (And How to Read Them)
This week Eurogamer‘s Simon Parkin somehow managed to cause quite a stir by awarding Uncharted 3 a mere 8 out of 10 in his review. The criticisms that caused him to score a teensy little bit lower than many other major review outlets were familiar: I’d already written similar things about Uncharted 2. I remember how that was received a couple of years ago, and it seems like many fans haven’t used the intervening time to grow up.
Given that Patrick Klepek over at Giant Bomb has already done a fine and comprehensive job of defending Parkin’s review, I don’t want to say too much about it specifically. But it does bring me back to a point that I haven’t yammered on about for a while: what reviews are for.
A lot of people believe that it’s a reviewer’s responsibility to be as objective as possible. At first, if you haven’t given it much consideration, you could be forgiven for thinking that this is perfectly reasonable. But it falls apart once you dig down into what the job of a reviewer actually is.
If you want to find out objective information about a game, there are dozens of places you can get that, especially in recent years. The game’s website would be a good place to start, but perhaps you want information that isn’t quite so filtered. So you might go to online videos like the ones produced by Giant Bomb (a site I can’t seem to praise enough lately), or more amateur efforts on YouTube, or podcasts, or just general online chatter.
You can also get a lot of this information by reading reviews. But if the only point of a review was an objective analysis, or the sort of thing you can read any-old-where there would be nothing to distinguish them from all these other sources of information. So what is the point; where does a review have an edge?
Reviews are a space where someone can give a prominent, uninterrupted, detailed and personal account of how it felt to play a game, and that is nothing if not subjective. That’s not to say there isn’t a place for facts – if, say, the framerate objectively drops below 20 FPS regularly then that’s probably worth mentioning. If, however, you go away from a review having no idea what the game felt like for the reviewer to play, then what on earth did you just read?
So much for objectivity, then.
There are other responsibilities that reviewers don’t have, too. They don’t have to be completely and comprehensively cover everything anyone might want to know in a review, as if it were the only review in the world. Nonsense. Different reviewers will notice different things about a game, and want to talk about the various aspects, giving more or less weight to some areas and perhaps even ignoring a few points that another reviewer finds highly important. Time spent trying to cover every single topic makes it harder to achieve the goal of talking about what really mattered to them: that subjectivity stuff, again.
But if reviews are largely a sea of opinions and personal preferences, what good are they? Well, that’s where you, the reader comes in. Once you realise that all reviews are written by human beings with opinions, tastes and priorities that may be quite different to your own, you can get somewhere. Knowing this, and getting to know different reviewers’ foibles, you can take it into account when choosing which you want to pay attention to. If they have done their job well, you can meet the reviewer halfway and depart with your own set of conclusions, like:
“I think the things this guy dislikes about the game are things that would drive me nuts, too… no sale!”
Or “These things the reviewer is annoyed by are actually things that I usually find quite fun, so maybe this game is worth a shot.”
Or even “This person clearly has absolutely nothing in common with my tastes, so I’ll probably seek some other opinions.”
Isn’t that nicer?
I always enjoy Peter’s articles. I tend to agree with his opinions.
Would it be better if we eliminated scores? I love the way Topgear, not gaming I know but bear with me, reviews things. They give you the good and bad and leave you with an impression. It’s up to you to take it or leave it. No score or percentage.
@Adushan: I don’t actually mind scores, despite the problems they cause, and here’s why.
Scores make the site more useful in some ways. For example, if I just bought a console and want to find out what the best games for it are, it’s much easier if I can go to my favourite site and check out the highest rated games. It’s also a shorthand. As much as reviewers would like to think we all have time to read all their reviews, we don’t. So sometimes it’s just easier to go: oh, cool, blah thought it was good, blah thought it sucked at a glance. It’s not as good an impression as reading the review, but it might give an idea, or even entice into reading (“Woah, X rated it a 10/10? I usually trust him, what does he like about it?”). So there are just a few ways that having that 1 number/letter summary can actually be helpful that aren’t immediately obvious.
I think getting rid of scores is an option, but I think the only sites who do this really successfully are ones that stop calling them reviews altogether. Even though I’m on the side of reviews being subjective, the word carries weight, and it’s interesting how Rock Paper Shotgun using the phrase ‘Wot I Think’ instead of ‘Review’ takes all the pressure off, and stops people complaining. It’s an interesting psychological trick: ‘Oh, if it’s just what someone thinks then they can say what they like. And why would what someone thinks have a score anyway?’ But when you read them, they’re reviews, they totally are.
The problem with scores is the problem I’ve had with numerous exams. (verbal ones) You go in, answer questions, and they give you a mark they FEEL you deserved. There was no objectivity or standardisation. It’s just a feeling.
Some reviewers FEEL a game deserved 82%. I mean come on! Why didn’t it get 81? Or 83? Why that specific 82%? That’s why I like reading a review and getting impressions. Or the 5 point system. But not percentages.
And yeah I do agree that some reviews are TLDR. And I will admit to looking right at the score before reading the review. It’s a force of habit.
@Adushan: Well, like I said, reviews are all about feeling, so I don’t mind the idea of putting a number on that feeling. But readers have to understand the number is as subjective as everything else.
As for percentages, a 100 point scale is too granular, sure. I think the a 20 point system is about as fine as the mind can really distinguish between. I could probably find 20 games and put them in boxes from ‘near perfect’ to ‘near-worthless’, just about, without it being too arbitrary, but a 10 point or 5 point scale is better.
A problem with the 5 point system is that you’d get many 5/5 games. Because there would be no way to say this game is slightly better than 80% other than saying it is a 100% game.
IMHO no game is a 100% experience. Harsh, I know, and that’s like saying nothing is perfect. But I can’t think of a game without a flaw.
That’s partially why the fanboys went nuts over the Uncharted 3 review. They had far too high an opinion of IGN and the like who regularly give game 9/10 or 10/10, and to dare insult a game by saying it is only 8 is outrageous!! When 10 is the norm, 8 is an insult.
@Adushan: Ah, there’s yer problem. You’re thinking that 5/5, 10/10 and so on means perfect. But there’s no reason to think so.
Imagine a 2 point system: 0/1 means don’t buy, 1/1 means buy. That’s a rubbish scoring system, but you wouldn’t think for a moment that 1/1 meant perfect, right?
The more granular you get, the more perfect a full marks game has to be, but around the point of the 5 and 10 point systems, there’s still some wiggle room.
Here’s a way of thinking about it: instead of thinking it as a sliding scale which is capped at at 0 for worthless and 10 for perfect, think of it as a bunch of boxes, and in the 10 boxes you put the very best games. They might not be perfect, but they are special enough to be named among the very best. A full marks game should either be about as good as you can imagine it being, or special enough that any small flaws it may have simply don’t shake the feeling that it deserves to be among the very best. A scoring system isn’t useful if you can’t use the whole scale, full-marks included.
Spot on, Pete. I love it when you write words.
The giant bomb 5 star system is enough. Their review guide is quite enlightening.
I had this argument with my dad recently, he said that publications should have a team of reviewers to go see movies, because this one critic he reads doesn’t agree with his opinion; he thinks that a team of reviewers that each cover their genre (popcorn action movies, political dramas, etc.) would be more effective. My counterpoint is that not everything needs five stars by selecting the right man for the job, it needs a consistent opinion from a few men that you can extrapolate from; he dislikes this thing I like so I’ll consider it even though he said to avoid it, or we have a similar opinion on most things so I can just up and accept his opinion.
To get the most out of a review, I agree that you need the context of the reviewers tastes as they relate to your own before you can do anything with it.
@Faye – I suppose it means more if someone is a specialist in their genre. However if someone who hates comedies says Weekend at Bernies is unmissable then it’s saying a lot. Hehe, Weekend at Bernies…