By The Numbers – What Role do Review Scores Play? (OPINION)

I’ve never used numerical scores in my reviews, be they for games, movies, comics, or anything. Recently, I’ve started using a new (to me) method that other sites employ, a list of pros and cons of a game. They’re largely a bullet point review of things I said earlier in the article, so readers can check out info at a quick glance. “But Afro,” you might be asking. “Don’t review scores give information at a quick glance as well?” To a certain extent, yes. But unfortunately, it’s not the greatest information out there.

First and foremost, numbers don’t really mean a whole lot. They mean something, but they don’t have an agreed upon value. Two reviewers could review the same game, and give it the exact same review. Word for word (think one in a kazillion chance and not plagiarism). However, that doesn’t mean that they will give the game the same score, one might give it a seven and the other an eight. But how does that make sense? How can they write the exact same article, and end up with things not being exactly the same? Well, it’s because one’s seven is the other’s eight. Review scores have no intrinsic value, they bend under the weight of opinions, scoring toughness, what kind of community the reviewer is a part of, and many other factors. We can all agree that a metre is one hundred centimetres, which is ten millimetres. We can for the most part agree on the value of products and money (against other currencies we certainly can), although there is the whole bargaining thing that makes that one a bit more grey. One thing we can’t agree on, however, is review scores. Sure, your seven might me my seven, but we’re not basing that on any agreed upon value like currency or a unit of measurement. Review scores do mean something, but they’re in a pretty big grey zone, and until we can agree on what they mean (which will be never), they aren’t a great form of measurement.

Numbers also can make people make a decision without really thinking. 6/10? Bad game, not worth buying. Beyond the fact that 6/10 is technically average but seen as garbage by many, a six might actually not be that bad for some people. If people read the article for a 6/10 game, they might see that it’s something they’re actually interested in. Review scores are so at a glance, and have so little real information in them, that they have tremendous sway with little meaning behind it. Now, that can be said for any “at a glance” system. A bullet list of pros and cons doesn’t have much information, simply because of the nature of bullet points. But it does have information in it. “Beautiful graphics” is way less information than the reader would get by going through the whole article, but it’s way more than a numerical review has. Oh, you gave the game an extra half point because of the graphics? How the heck am I supposed to know that, at a glance? Even if  I do read the whole article, it’s unlikely to say “I raised the score by 0.5 because of the graphics.” Once again, it comes down to the fact that the numbers don’t really mean anything, they’re just numbers.

Via Metacritic Sept. 3/16
Via Metacritic Sept. 3/16

Finally, let’s talk about aggregate scores from sites like Metacritic or GameRankings. Now, this is going to be a bit of a shorter section, because I’ve covered parts of it already, and aggregation sites for the most part just smush pre-existing numbers together. First of all, we have the whole “numbers don’t mean much” again, which is even more potent in ranking sites because they’re just numbers, possibly with or without a short review, and maybe a link to a full review. There’s also the fact that not all sites use the same scoring system, which can screw up there data in relation to each other. Some places use a scoring system out of five, some out of ten, some out of one hundred. Think about that, the margin of error on reviews from sites that use smaller numbers is incomparable to those that use smaller numbers. A site that uses a system out of five has a delta of 1/11, or ~9.1% between each value (if zero is included), whereas a site that uses a system out of one hundred has a delta of 1/101, or ~0.99%. In their own ecosystem that’s fine, but once you put them up against each other, those differences can really mess things up.

Now, there are others who don’t do review scores, and I wanted to give you some of their points. Kotaku doesn’t, and Eurogamer stopped last year. I grabbed some quotes from my own colleagues here at Ready Up Live, as well as a 2013 article by Kotaku’s Jason Schreier, and the aforementioned Eurogamer article, but Oli Welsh. They had some good things to say, and I went through them after writing this article to hear their thoughts. Here’s a bit of what they had to say:

In terms of review scores, I find it’s not exactly necessary for a set number score, because when you review things, it’s going over the subject and covering it in detail. While the scores are good for a quick glance, I like to get a “you should buy this” or “you shouldn’t buy this” instead of a potentially ambiguous number. – Mat “Guitarguy” Olson, RUL; PM

 

Numeric values are garbage – in part because they’ve been inflated by [reviewer’s] fear of giving a bad review… SO many games are now stuck being rated between 9 and 10 that there is no longer a quantifiable distinction. – Mark “Spook” Barry, RUL; PM

 

Is Grand Theft Auto V, an open-world mayhem generator, better than Gone Home, a quiet, focused story about a girl and her family? There’s no answer to that question. They’re totally different types of games…[y]et because of review scores, and because of score aggregators like Metacritic, gamers feel encouraged to stack them side by side. – Jason Schreier, Kotaku; The Problem With review Scores Part V

 

How should we score an excellent game with severe networking issues? A flawlessly polished game with a hackneyed design? A brilliantly tuned multiplayer experience with dreadful storytelling? If you expect the score to encompass every aspect of a game, the task becomes an exercise in futility. – Oli Welsh, Eurogamer; Eurogamer has dropped review scores

All in all, review scores have their pros and their cons. But in the end, the cos quite outweigh the pros. I’ve said this again and again, but review scores don’t really mean anything. They’re not comparable between dissimilar games, and they’re not comparable between reviewers. On top of that, the little information they convey is done poorly, simply because of personal biases and the fact that, once again, they don’t mean much. Would I like a world without review scores? You bet I would. Is it likely? Probably not I’m afraid. They’re just too easy for readers, and too powerful for games. If the industry does change, it won’t be overnight. It’s a big industry, and it’s hard to move. For review scores to disappear, they’re going to have to go away site by site, publication by publication, Youtube channel by Youtube channel. And in my book, even just one reviewer taking them away is a good thing.

What do you think? Do scores have a place in reviews? What kind of systems do you like in reviews?

Tell us in the comments below, or on our subreddit!