In light of the scores that NHL 15 received around the Internet, I started thinking about the concepts of review scoring and user expectation when it comes to sports games. To be sure, there have been a lot of vocal fans and consumers who are upset about what they feel happened with this year's version of NHL 15. You don’t have to throw a stone far to hit someone who is bothered by what they feel is an incomplete product that doesn't fit with the steady quality that NHL has enjoyed up until this point. I echoed a lot of these sentiments in my own review, where I described a "dissonance" in the way the game provided some enjoyment with its gameplay and presentation while at the same time disillusioning me with its missing features and paltry mode selection.

To be fair, though, a large number of commenters and consumers have rightfully voiced their positive opinions about the game. Many of them concede some missing features, but they ultimately feel that gameplay is king and that a review should value that above any other metric. It's hard to begrudge anyone for having this view, as they may enjoy playing one-on-one online and some basic spin on a "career" mode — and NHL 15, in one way or another, has those things. It also has, on the whole, some dazzling presentation and some fun gameplay (particularly on harder settings/sliders or with human opposition). No one is going to convince anyone in that camp that the game isn’t meeting their needs.

But a review, such as it is, is one critic looking at a product and giving his or her opinion on it while lensing that commentary through the intended audience. Operation Sports, for instance, mainly serves hardcore gamers, but we obviously have a wide array of gamers and sports fans — casual, mid-level and hardcore — who read and consume our content. It's an inexact science, but the goal is to inform as much of that audience with an opinion that is, hopefully, coming from someone who understands what that audience might be looking for. Our audience, for instance, wants more detail on certain minutiae that you wouldn't find in a game review from your local paper. The structure is similar; the depth is different.
 

In this way, it's sometimes hard to step back and really think dispassionately about what it is we're talking about. Even though Operation Sports serves a predominately hardcore audience, that audience gets hardcore about different things. Some users are more into stats. Some are into gameplay. Some are into feature depth. Some want presentation realism. And yet others want a blend of all of those things.

When we think about assigning a score to a game, it's really just a way of giving some kind of shorthand for what the general thrust of the text is. It's a way to orient the reader so that they know the tenor of a review before diving in. The score is never meant to be an emblem for a game’s quality, but that often becomes the way its interpreted by readers. The text is ultimately meant to inform and “defend” the score to a degree, but it also provides much needed context for those who might still want to engage with a product even though the score is low, for instance.

The perils of scoring to some kind of artificial “game curve” — the 7-to-10 scale, if you will — is that everything just ends up feeling the same and lacks any differentiation. It’s an irony that all of the granularity afforded by a 10-point scale, or even larger, can be lost in the concept of scoring a game just to fit in with the herd of sites and mags that don’t really seem to want to rock the boat. When people are aware of something like Metacritic, it can be a terrifying proposition to be a “dissenter” or outlier that has some kind of wildly incongruous position that doesn’t fit with the accepted narrative about a product.
 

In that way, I’ve always tried to use the whole scale in everything I review, and I feel that’s something that all of us try and do here. We have a clearly defined scoring rubric, and it breaks down, with great specificity, what is going on with all of these numbers and descriptors. You can find it here.

The problem is that sometimes even a well-defined rubric can’t stop the flood of emotion that some people feel about a score that doesn’t fit in with the usual “game curve.” There is precedent for these kind of scores, but not a lot of it. It’s even rarer to see a “low” score for a AAA game. The crazy thing is that 5.5, for something like NHL 15, is “average” on our scale. Average is nothing like bad, and nor is it anything resembling “great” or “all-time classic.” It is a game that falls in the middle, and it may appeal to certain audiences. That doesn’t mean the product is reprehensible on every level and should be avoided; it means that there are major issues that situate it firmly in the middle of our review spectrum.

It’s certainly no fun giving a “middling” score to a game that I, personally, was quite excited for. I know that many users were equally excited for it. As I said in my review, the main issue is with the decision-making process higher up at EA (above the developer level) that led to wholesale cuts to major, moderate and minor features. These omissions and gaps left a shell of game that had to rely on its visual punch and some reasonable gameplay improvements.

Still, some users felt that a 5.5 for NHL 15 was unfair — that the game was being unduly punished when peers in the sports game space are (seemingly) not. The issue, as I see it, is that every review, like it or not, has some element of bias, and it also needs to be created with a good deal of context in mind. No matter how hard anyone tries, the shadow of previous versions or comparable games is going to loom over any review, and that is not something that can just be hand-waved away. To want tabula rasa each time out just doesn’t hold water, especially when a franchise is continuing a brand. It’s easy to say that every game should be reviewed “in a vacuum,” but that just isn’t how anyone thinks. Expectations and subtle biases will creep in, however unintended they may be. Every game — sports or otherwise — is usually stacked up against some sort of comparable product, and even previous versions of that franchise are brought into discussion. If I know that last year’s Be-A-GM mode was better than this year’s, why would I give that a pass?
 

Of course, there’s still a thought that standout gameplay should trump some level of feature omissions, especially in light of this being a “reboot” year for NHL. This is important context that is valuable to consider, but I think it also muddies the waters, prohibiting almost any game from serious critique. In that way, what game would ever score low if it was constantly judged as a “standalone” product? For NHL specifically, what level of feature or gameplay loss would’ve been deemed “acceptable” or “unacceptable” by the masses? By comparison, I gave EA Sports UFC a higher score because I felt it was the start of a brand and because the “expectation” for a one-on-one fighting game is wholly different than that of a team sports franchise that relies heavily on shared experience. On top of that, I thought EA Sports UFC was just a better game.

My main beef with NHL 15 is that it is emblematic of a certain type of cynical game design that should not be encouraged in this industry. When we buy a product, we enter into something of a compact with those who make it. By continuing to purchase a product — and peripheral content — we are showing an affinity for the choices a company makes. The choices EA made for this game, in this particular year, were not good ones. These choices impacted the quality, depth and value of the product for what I felt were a great many of our readers.

But even with “beef” towards NHL 15, we still scored it average. This doesn’t mean the game is devoid of value for all. There are clearly users enjoying the game right now, and that’s a good thing. The issue emerges from the score-based nature of the industry and how that drives bonuses at studios, marketing budget and PR spin. That leakage — from the developer and marketing side — has the ruinous effect of clouding the meaning of scores, and it codifies them as something they’re not. Consistency is still our core goal, and that’s only possible with a fully functional review system that is employed to its fullest.