Review Symposium: Review Scores (Part 1, Off-topic)
By John Kershaw on Friday, December 26th 2008
Part 1 of Shawn Elliott’s review symposium has been available for a few days but as it’s around 3 miles long, it has taken an understandable number of hours for me to read through. To help you get the general ideas behind what was said, I’ve broken the whole thing into 3 separate posts.
In this first post, I will try to highlight all the off-topic, yet still very valuable things discussed. In part 2 I will go over the actually relevant areas of discussion (i.e. topics directly related to the idea of reviews having scores). In the final part I will try to respond to the on-topic questions and ideas raised, adding my own (completely correct and un-questionable) opinion to the mix.
As I said, this first part will cover all of the interesting parts I found in the symposium which are not directly related to the topic at hand. As with any attempt at re-wording and simplifying what other people have created, if you find yourself interested, you really should give the full symposium a read through.
To Critique Or To Review?
The first and most striking thing I took from the symposium is that many of the “reviewers” taking part seem to dislike everything a reviewer is meant to do. Instead, they seem to very earnestly want to be viewed as a critic. It’s an idea I’ve noticed in almost all work from people such as N’Gai Croal and Leigh Alexander. The articles they produce always have a feeling of verbose grandeur, like using the words “verbose grandeur” to mean “long-winded and needlessly complex”. It seems that the content within each article is fairly simple, but worded in a very academic way. As if to add validation and a sense of intelligence to what’s being said.
A lot of this falls into the field of “new school journalism”. Besides linguistics, another example of this shift towards to critique, Croal spoke in part about how he feels he doesn’t need to complete a game to critique it.
[H]ow much of this game am I going to be able to complete before my deadline? That’s very different from how I approach plays, television, theater or literature’”I wouldn’t dream of critically assessing a piece of work from those media without having completed it. Why doesn’t that stop me from doing the same with videogames?
The explanation’”or is it an excuse?’”that I offer is that I don’t review games. We’ll get into this more in the Reviews vs. Criticism section of our symposium, but the way I see it, a reviewer answers the question, how well does this game work, but a critic answers the question, how does this game work? A reviewer helps consumers decide whether or not they should buy a game; a critic helps players think about a game that they’ve played’”in its entirety /or/ in part’”and that is the end of the spectrum where I believe my writing lies. (That’s also why, on a game by game basis, I don’t think I need to have completed a game to have some insights about it’”but I do think that if I were advising someone on how to spend their money, I’d feel obligated to play most or all of the game.)
From which we can deduce that you shouldn’t read a review by Croal.
Continuing to break the mold of what is thought of as needed with a game review, Shawn Elliot himself points out the drudgery of some of the apparent necessities of a review.
The paragraphs on a game’s graphics, sound, and so on in previews and reviews produce recognizably generic writing devoid of the discovery and perception that might make them worth reading. They are lazy in that they eliminate both the need to transition thoughts and to interpret a game as the complex product of interconnected components (instead of simply summarizing these parts).
Again, this highlights the fact that may of these people just don’t want to write game reviews. As a reader, I want to know what it looks like. I would prefer some concise yet generic writing about a game’s presentation, rather than a six page deconstruction of its meaning. But only to an extent. I agree I don’t want to be bored when reading a review, but I also want to actually get the information I want from it.
John Davison added his thoughts on the content of reviews, mentioning that reviews often need to include some of the boring stuff, if just to fulfill their role.
The innovation and creativity of games design (or lack thereof) is more than just artistic expression, and an assessment of the overall experience is often incomplete without some mention of the mechanics. Criticism of games reviews often focuses on the fact that we spend too much time on this stuff, but it is often a more important consideration than the “art”. I think we’d all love to spend more time on digging into what a game is trying to “say”‘”but we’re often faced with impenetrable control mechanisms, or distracting technological problems that cannot go unmentioned. There are also still an awful lot of games, particularly in the current climate of “casual” style games on DS and Wii, that are practically *all* mechanics, and an intelligent discussion of how well they work is all that’s needed.
Watch out, hype-wagon crossing.
There was a lot of talk about how hype should or should not affect a review. Many of the people taking part feel that almost all of the pre-release PR should be simply ignored. Why though? The people who read the reviews haven’t ignored it. I like to think I’m smart enough to be able to write about a game whilst keeping the PR side of things in check.
The key question raised though is; should it affect the actual score? Of course it should! The score is a summation of the text, and the text is a combination of an opinion and a deconstruction of the game. The opinion part makes it a review, interesting, useful and good. Without that, it’s just a game design document.
The real trick is to simply use your skill as a reviewer to know what the readers are thinking and expecting, and try to cut out the PR-hype that yourself and relatively few others have been exposed too. They almost certainly won’t have had the same experiences as you, as Francesca Reyes explains.
[T]he trap is that we all play a shit ton of games, right? But our readers do not. Yeah, some of them play a lot of games that we haven’t. Some of them may play as many as or more than we have. But that’s the small portion of our audiences. Most of them pick and choose what they buy and we have to understand that spending their cash may rely heavily on what reviewers say. You have to respect that and go in to a game with the same expectations that someone without the months of exposure to a title might have.
Jeff Gerstmann added that it can often be harder for a game with a large PR budget, as people have raised expectations. But on the flip-side, games with limited budget can be given a very negative review, without much fear of reprisal.
I find [Kieron Gillen]‘s comment about big budget games catching a bit of a break in reviews to be possible in isolated cases, but I don’t think it’s the norm at most publications. I’ve also seen the opposite, where the big budget game gets trashed for not living up to the insurmountable mountain of marketing hype while the low-budget indie darling catches a break because it was made by a team of five people or something. And I’ve certainly seen little, unmarketed games get absolutely thrashed in reviews. Sometimes it seems like the reviewer is doing this because it’s “safe” to do so, like it was part of some sort of “see? We totally use our entire 1-10 scale” chest pounding.
Exclusive reviews suck.
Towards the tail end of the discussion, the topic of exclusive reviews come up. It’s something that nobody likes the idea of, but is something that sadly this industry and its continual groping for money puts up with. A few people commented on the topic, but nothing particularly new was added. I thought that Jeff Gerstmann’s comments were quite interesting however. He points out that the exclusive review is likely on the decline, enjoyably.
Exclusives are yet another by-product of this business’ print roots. Between magazines stories getting scanned, website text getting pasted into message boards, and the way that most of a site’s traffic doesn’t come in through the front door (where the big promotion of the exclusive always is), exclusive stories are waaaaay more trouble than they’re worth. Exclusive reviews doubly so. On top of that, in my experience, most big exclusives don’t really move the needle that much, traffic-wise. They’ve always felt like a huge waste of time to me. Attracting users by differentiating your coverage from the pack has always seemed like a better goal.
Whilst all very interesting, each of these topics will undoubtedly be gone over in more depth as the symposium continues. Keep and eye out for part 2 of our coverage, due to be posted in the next few days. As always, discussion is welcome either here in the comments, or over in our forums.


Good write-up.
But I feel that there are too many different tastes for any single useful type or format of review to exist. And I can see these things as something of a mould-maker or perhaps trying to suggest a priority format.
It’s interesting what was said about exclusive review. Thanks to the internet, it’s always a race to get the “world exclusive” and I absolutely hate that! The news is repeated over and over throughout multiple website and its all the fracking same. It annoys me to no end.
I’m glad someone read all of that so I didn’t have to. Yeah, as far as I have read, N’Gai Croal talks about games and gaming, but doesn’t really review them. Is there something wrong with that?
I do not see why critics and reviewers need to play entire games to review them. They are stating their judgment on what they have played and use that as a gauge as to whether they would recommend the game or not. Pro reviewers don’t have the time to play all; these games; so if people wand game critics to review games they’ve totally finished, you’ll have small websites and thin magazines.
Why stop there? Why shouldn’t reviewers 100% every game they play, including the super secret sidequests and find all the easter eggs? Come on! If a game blows, it will most likely not magically become better later on.