| 
Rawr
Advert
Subscribe to the RSS feed Add to Google Become a fan on Facebook Follow us on Twitter Get email updates
Subscribe:
wardrox

Why The Average NG Review Score is Not -5

By John Kershaw on Wednesday, December 2nd 2009

Count the scores; we have an 11 point scale

Our review scale goes from 0 to -10. Why then, with the recent public release of easy to reach statistics based on our reviews, is our average not -5? It’s a question I want to answer now rather than later as it’s an inevitable basis for criticism and I would rather have it debunked before it’s used.

Before I go on I want to take a quick side-track to explain why I’ve made this data so easy to access; most sites don’t. My reason to list this information is basically because I can and I’m interested to know it myself. If I’m interested to know what the standard deviation of our reviews is 1.9 then chances are somebody else out there is too.

When more people than just me started writing reviews Negative Gamer I needed a way to track people’s reviews more accurately: who is reviewing which game, what format is it being reviewed on, etc. Previously all of the review information was saved in one spreadsheet and while practical, wasn’t ideal. As more reviewers, reviews and games were added it was clear it wasn’t going to scale well. Because of this I created a new review system built directly into Negative Gamer that does most of the admin stuff automatically. The review page is simply the public interface for it.

I also don’t happen to agree with the most frequent reason other sites hamper the gathering of equivalent data from their sites; it causes arguments. It’s the same reason many sites don’t disclose their freebies, like a holiday resort based review day for Modern Warfare 2. I call bullshit on that. Reviewers are worrying about a small minority of readers misunderstanding the data and commenting with some silly reason as to why said reviewer probably likes men. They really care about these idiots? I don’t, and I don’t think they do either. I think the real reason is fear that they would have their integrity called out. Just being open and honest does that.

That’s enough about why I’ve made the data so easy to get at, now to the important question: why is the average score not what is described in the score breakdown as the most average rating? (Note how I’ve used the description of the score rather than just the number. The words are the meaning behind the numbers. An interesting point:  our scoring goes from 0 to -10 inclusive so there’s actually eleven different scores a review can take and the average of those is -4.5 [and I am bad at maths - thanks for the correction ParaParaKing].)

We prefer to play games we think we’re going to like and most of the time we’re right with our assumptions. If every person on staff reviewed every game, I think you would find the average closer to -5. As it is only one person reviews each game. That person is probably the person who most wants to play the game, thus highly likely to know what the game is about and like it. Simple.

It’s probably also worth mentioning that the mode (most common score) is, at least for now, 5. We give more games an “average” than any other score.

About the author
When not complaining on the internet as Nukezilla's Editor in Chief, John is usually either in the UK or New Jersey enjoying gadgets, beer and the depressing weather. He has a personal blog on Tumblr if you're interested in that sort of thing.
Categorised as Editorial, Article.
Tags: data, Graphs, reviews, Statistics
Sharing options loading...
Advert


Comments


player66 Says:

Wardrox, please take down that graph! Graphs are scary!!!


ParaParaKing Says:

You have to check your math. -4.5 is not the average between 0 to -10.

MATH:
0-x=-10+x
2x=-10
x=-5

The average is -5.

For more shocking math:
5 isn’t the average between 1 and 10.
(Hint: It’s 5.5)


The reason I always assumed most sites don’t make their review data and scores more easily accessible is because that requires extra work. Work that probably isn’t easy to do with the drag and drop blogging software they used, and for one reason or another they won’t make something themselves.


ParaParaKing Says:

@ScottyGrayskull: You can view all reviews of one site on gamerankings.com . There is no reason for any big enough site to build something like this.


wardrox Says:

@ParaParaKing: Maths corrected sir, not sure what I was thinking there.

@ScottyGrayskull: To be honest the only hard part is putting the data into the system. I assume most sites keep some record somewhere of all their reviews. If not, it doesn’t take too long to manually check them all. If there are hundreds of reviews to add, work as a group, or just work slowly. I certainly agree though that one reason sites don’t do this is the effort.


@paraparaking
Oh yeah… I’ve always liked gamerankings’ data too. :)

Although at the risk of sounding like a dick, why go through the trouble of making the data available on this site when you could’ve just used gamerankings? You put it on there for the potential extra exposure, and get that as a bonus.

Unless they don’t take smaller sites, in which case nevermind then. :P


wardrox Says:

@ScottyGrayskull: They don’t take smaller sites for one, and for two, this publicising of the data is just an added bonus to the new review system.

The new system was made to help me track all our reviews (keeping track of who has what review due when gets complex with 30+ writers, lol). Originally all the data was going to simply sit in the database, away from public eyes. Then I added the stats for my own personal interest, and then thought it would be nice for others to see.

It also compliments something else I’m working with the site; adding more game data to pages. Still in the very early stages, but my plan is for every game to soon have it’s own auto-generated page, with reviews/haikus/info on etc. All data will be gathered via Giant Bomb (who have an API, which is actually used internally to get game data).

Then you can go to, say, Modern Warfare 2′s “page” and get general game info, as well as all the material NG has on that game, all in one place.

I’m also going to experiment with wikipedia style info boxes to go in articles. Just to give a light smattering of facts and such :)

There is much more I’m tinkering with and planning, but as so much can change and the plans are so early, I don’t want to go too much into it.


Leave a Reply

You are not currently logged in. Comments by registered users are highlighted and are much more likely to be read. You can either login here, or register for Nukezilla here. It's also worth noting that if you're not registered and your comment contains a link, it will be marked as spam and may take a while to be manually approved.

For help with formatting and posting images click here.

| More