WHEN ANALYTICS AND VIDEO GAMES MEET: REWORKING NHL 20’S RATINGS USING REAL-LIFE DATA
JUNE 11, 2020BY BILLY BERTRAND
When Analytics and Video Games Meet: Reworking NHL 20’s Ratings Using Real-Life Data
SHARE THIS POST?
Facebook Google+ Twitter
EA Sports, It’s In The Game
Simply hearing those few words brings back memories of the countless hours I and many others have sunk into the different EA sports titles. FIFA may be the best-selling game but as a hockey fan, the NHL series has been near and dear to my heart for years. NHL 04 on PC was my first ever, with Thrashers’ Dany Heatley on the cover before he was replaced by Joe Sakic. It’s also around that time my hockey fandom really kicked off. I’ve always been more of a GM/Franchise mode player, so player ratings have always been of particular interest to me.
I remember Peter Fosberg being the highest rated player in the game at a 97, 3 points higher than anybody else. I remember Alex Kovalev, who instantly became my favorite player after being traded to the Canadiens because he was the only 90+ rated skater on the team.
With the hockey world going into lockdown in March, right as the playoffs were about to kick off, NHL 20 was the best way to get my hockey fix. This time, however, instead of hopping into franchise mode, I went to the rosters page. Now, we all know the rating system isn’t perfect. Boiling things down to one number is nearly impossible, especially when there are so many different ways to be an effective hockey player.
EA has done a solid job of placing players in a logical overall range for the most part. However, there are some red flags that come up for me when looking at the individual attributes. Here are a few examples:
In EA’s most recent roster update (March 30th), Duncan Keith is as fast as Patrick Kane, both at 90 speed. Alex Ovechkin has a better stick check rating (89) than Jaccob Slavin (88), despite the Hurricanes defenseman having one of the best and most active sticks in the NHL and totaling the second-most stick checks in 2019-20.
Same goes for goalies, as Frederik Andersen has a 90 five-hole rating despite being beat 26 times between the legs this season, 3 times more than runner-up Braden Holtby and one of just 3 goalies to allow 20 such goals this year. Connor Hellebuyck, meanwhile, has allowed just 17 in about 400 more minutes and has an 89 five-hole rating.
Now, I understand that determining player attributes for the nearly 900 skaters and 60+ goalies who made at least one appearance last season is a massive task and that doesn’t even take into account the AHL, SHL, DEL, CHL, and multiple other leagues EA has to also rate. Still, with all this time with no hockey to cover, I decided to try my hand at updating the NHL 20 rosters myself but with an analytical approach, based on real-life data from Sportlogiq.
For every player who took the ice this season, we track over 200 different events per game that range from pretty straightforward stats like shots and shot locations to more granular things like puck on stick time for skaters and save percentage by area of the net for goaltenders.
With all those metrics to my disposition, I wanted to see if I could come up with a way to rework the ratings system in NHL 20 and help make players more closely resembling their real-life counterparts. It took many hours of data compiling, playing around in excel sheets, and manually entering the new ratings in-game and the results are definitely interesting.
The Process
For each skater, I took nearly 60 data points into consideration to rework the attributes and over 30 for goaltenders. Most of the statistics used were per-20 for skaters and per-60 for goaltenders. There was also a playing time threshold to remove small sample size outliers. I settled on 20 games for skaters and 650 minutes played for goalies.
I needed to normalize the data a bit because comparing percentages with seconds and per-20 averages would be a headache. I settled on using percentile ranks. That way, every statistic is on the same 0-100 scale, making it much easier to work with data in different formats. Also important to note that forwards and defensemen were graded on separate percentile scales.
I then classified each data point under the relevant attribute. Some were extremely obvious, like stick checks going under stick checking, others weren’t as straightforward, like controlled entries and exits going under speed. Each in-game attribute had at least 2 or 3 statistics taken into consideration, with some having over 10.
In most categories, I also added some form of time on ice variable after running into many issues of small sample sizes being overvalued. Anton Khudobin, for example, consistently graded out as one of the best goalies in the league in nearly every category and ended up being around a 90 overall in my first few attempts.
Nobody would say Khudobin should be rated in the 90s, but he did post a .930 save percentage and a 2.22 goals against average, which ranked first and third respectively this season. He hit those numbers on just 26 starts though, 46th in the league. Khudobin and Jordan Binnington both saved roughly 0.3 goals above average per 60 minutes for their respective teams. However, Binnington did so over 50 starts, nearly twice as many as Khudobin, therefore Binnington would receive a higher rating because he maintained his level of play over a larger share of games.
Adding total minutes played for goalies and time on ice per game for skaters was my way of rewarding players who maintain a strong level of play while also handling a heavy workload.
Now, for the process itself, I’ll use Braden Holtby’s five-hole attribute as an example. To make up the number that will later be entered in the game, I used three data points: five-hole save percentage, five-hole actual to expected goals, and time played. Each data point was represented by Holtby’s percentile ranks, which are as follow:
Five-hole save %: 16th percentile
Five-hole actual to expected goals: 23rd percentile
Time played: 84th percentile
In the case of the five save stats, all three data points were weighted equally, each making up 33.3% of the final number. The weighing will vary in different categories and will not always be an even split. This gives Holtby a 41 rating. Now, I can’t take that 41 as-is and input it as Holtby’s five-hole attribute for a couple of reasons.
First, the lowest a player attribute can be in NHL 20 is 36, not 0. Holtby’s 41 would be one of the worst possible in the game when it is actually more middle of the road.
Second, EA has more than just the NHL to worry about. Players from all leagues, going from the AHL to the various CHL junior leagues as well as European pro leagues like the SHL, need to work within the same overall system. Holtby’s five-hole was weak this year by NHL standards, but there’s no way it would be a 41. That would make him worse than 16-year-old backup goaltenders playing junior hockey. That won’t do.
To work around that, I needed to take this new five-hole rating and put it on an NHL-caliber scale. To achieve this, I used a bell curve to bring everybody up to NHL standards. This basically takes all the new ratings and forces them into a new range.
After some trial and error, I settled on 75 being the minimum and 95 being the maximum. I kept the maximum at 95 because EA has gone with lower attributes in recent years and hasn’t doled out any 99 that I could find, at least not outside of Ultimate Team, so this allowed me to keep things within a range that still looked legitimate to what players are used to with the base game. I went with 75 for the minimum, because anything lower than this isn’t considered NHL caliber anymore and all the eligible players had at least 650 minutes of ice time this season.
This means that Malcolm Subban, who posted the worst five-hole rating at a whopping 8, will receive a 75 when transposing to the game. Connor Hellebuyck had the highest five-hole average with 88, so his in-game rating in that area will be 95, the maximum on the new scale. Everybody else will fall in between those two. Holtby’s 41 becomes an 83 with this new method.
The same process was repeated for all eligible players and all the attributes I included in this reworking. I then entered the roughly 5000 new attributes into NHL 20 manually. Some surprising names shot up in overall when it was all said and done, some superstars dropped quite a bit, and a few goalies even lost their number 1 status.
Now, I won’t do an in-depth dive into the results right here, or else this article would be way too long. Starting next Tuesday, I will detail the results of my experiment, with goaltenders first, followed up by the skaters. Then, I’ll run a playoff sim on NHL 20, trying to approximate the new playoff format as best I can, to see who comes out on top and get a better picture of how the new ratings impacted various teams.
NHL 20 OVERALL OVERHAUL: NEW GOALIE RATINGS BASED ON REAL-LIFE DATA
JUNE 16, 2020BY BILLY BERTRAND
NHL 20 Overall Overhaul: New Goalie Ratings Based on Real-Life Data
SHARE THIS POST?
Facebook Google+ Twitter
With the hockey world going into lockdown in March, right as the playoffs were about to kick off, I turned to EA Sports NHL 20 for my hockey fix. Using Sportlogiq data, I decided to rework the overall system and assign new attributes to every NHL player based on their real-life performance. In Part 2 of this 5 part series, I will break down the results for goaltenders. If you missed Part 1, in which I explained the process, click here to go give it a read to make sure you see the method behind the madness.
Reworking the goalie ratings system was the most fun part of the experiment for me. Rating skaters is difficult because they can be effective in vastly different ways. It’s tough to boil things down to one number to compare a defensive ace, like Anthony Cirelli or Phillip Danault, to a pure goal scorer like Alex Ovechkin or a speedy playmaker like Mat Barzal.
Goalies, meanwhile, have one job: stopping pucks. Yes, there are other skills like passing where a goalie can excel, but it all boils down to their ability to stop shots. A great goalie can carry a team on his back and a bad one can sink even the most talented of squads. There are also a lot fewer netminders than skaters, so it was easier to do multiple attempts with different formulas.
After pouring through the individual ratings for each goaltender, I started noticing a strange trend. While there is a good degree of variance between the best and worst goalies in the NHL, there is a total lack of strengths and weaknesses for each individual. It’s especially glaring when you look at what I will refer to as the “five save stats”: Five Hole, Glove High, Glove Low, Stick High, and Stick Low.
Braden Holtby is a perfect example of this. Here’s what his five save stats look like in the most recent EA roster update:
Five-Hole: 89
Glove High: 89
Glove Low: 89
Stick High: 89
Stick Low: 89
Odd, isn’t it? Just perfectly even all around. And this isn’t an isolated case either; most goalies have their five save stats in a very small range, if not identical like Holtby’s. After noticing this trend, I wanted to properly quantify it to make sure it was more than just an impression. I compiled a list of every goalie and classified them in two categories: those whose best and worst five save stats were within 2 points of each other, like Holtby, and those who have a delta of 3 or more between their strongest and weakest save attribute. Here is what the distribution looks like:
Nearly 80% of NHL goalies have little to no variation in their attributes. Only two had a delta of at least 5 points between their best and worst five save stats: Elvis Merzlikins and Carey Price. This means that no matter who’s between the pipes, there is no real purpose of targeting specific areas of the net. You can’t spot a weak glove hand and try to attack it or see a goalie that excels with High shots but struggles with anything that is sent down low and game plan accordingly.
Back to our guinea pig, Braden Holtby. With Sportlogiq data, we can track how goalies actually perform in all five areas of the net. Holtby was having a pretty tough season before the stoppage, with a sub .900 save % and a GAA over 3.00. He was also beat five-hole 23 times, second-most in the NHL behind Freddie Andersen, and ranked in only the 16th percentile in five-hole save % among eligible goalies. He was much better low on the blocker side, where he ranks at the 73rd percentile. Here is a full breakdown of where he ranks in save % per zone:
Five-Hole: 16th percentile
Glove High: 59th percentile
Glove Low: 11th percentile
Stick High: 37th percentile
Stick Low: 73rd percentile
The five save stats are the most glaring, but the other attributes have a similar pattern of being confined to a small range with little variation across the board. Once again looking at Holtby, here is his full rating card:
Screenshot from NHL 20 on PS4
Of the 19 attributes NHL 20 has for goaltenders, only passing, puck playing frequency, and durability are not contained in the 88-91 range for the Caps starting goalie, none of which directly impacts his ability to stop pucks. The game could use some variety between the different netminders to better reflect real life and make each one feel more unique.
Using real-life data for each goaltender, I came up with new attributes that better reflect their real-life performance. I used only data from the 2019-20 season, so don’t be surprised if Jimmy Howard crashes down in overall or a few unexpected names jump up in rating. If you want to know in detail the process I used to rework the stats, take a look at the full breakdown in Part 1 here.
First, of the 19 different in-game attributes, there is a handful I decided to leave untouched.
Aggressiveness: I wanted to have data on how deep in their nets each goaltender played: the furthest out of their crease on average, the higher the aggressiveness rating would be. Unfortunately, the NHL’s plan to have chips on players isn’t yet fully implemented, so this one will have to wait.
Speed and Agility: Once again, the lack of tracking data makes it tough to assess the athletic profile of players through numbers only, so I chose to leave it untouched
Durability and Endurance: I only used data over one year, so I didn’t want to dock a usually durable player for a freak injury or prop up an injury-prone player who went through a healthy year.
Poise: Poise is one of those weird stats that tries to measure intangibles. In this case, it seems to influence mostly how one performs in the playoffs. Since I used data only for this past regular season, I decided to leave it untouched.
Breakaway: The sample size here was just too small over a single season and this attribute has a big impact on the final overall number, so I left EA’s ratings untouched here.
This leaves 13 attributes that were included in the overhaul, with a total of 31 data points taken into consideration for each goaltender. Some, like poke checking, were fairly simple, while others like rebound control and angles used 5 or more data points each. I also included a time played variable to stop Backups like Anton Khudobin, who posted a .930 save percentage, being rated as the best goalie in the NHL.
After running the numbers, all that was left was input the new data into the game and the results were very encouraging. Remember earlier when I said roughly 80% of goalies had their five save stats within 2 points of each other? Here’s what the distribution looks like after the overhaul.
What about our boy Holtby? What does his new player card look like?
Screenshot from NHL 20 on PS4
Instead of 89s across the board in his save stats, he now has a much wider range, topping with his stick low at 92 and down to his five-hole and glove low at an 83. His overall also dropped from an 88 to an 86, which makes sense given his poor 2019-20. He was still used as a number one goaltender this past year, so his high ranking in minutes played kept his overall from plummeting and so did the stats I left untouched.
The effect is seen league-wide too. The distribution of the five save stats was turned on its head. The 80% of goalies with their stats within 2 points on the default roster has shrunk down to less than 2%. Over 80% of goaltenders now have a variation of at least 5 between their five save stats. This makes each goaltender a lot more unique, each having strengths and weaknesses. An 80 overall goalie could potentially have a better glove than a 90 overall, but be extremely weak everywhere else. I believe this could lead to a more dynamic playing experience, as you would need to adapt to each goalie you face.
Now, before I show you some of the more interesting results, I want to specify a couple of things. First, remember this only includes 2019-20 data. Name value or past accomplishments do not bring value outside of the few attributes I left untouched and kept the default EA number.
Second, while I played a lot with the weighting of different statistics for each attribute, I did not have a say over the final overall number. Some attributes, like Rebound Control or Recover, have a much bigger impact on a player’s overall. Others, such as vision, have a significantly smaller impact. Whether I agree or not with those weights is irrelevant, I had to go with what the game’s algorithm dictated.
With that said, here are the results, starting with the league’s top-5 rated netminders
TOP 5 GOALIES
Only 5 goaltenders made it into the 90 overall club and they’re all deserving candidates. Tuukka Rask led the league with a 2.12 GAA. Connor Hellebuyck is a lock as a Vezina finalist and is probably the favorite to take home the hardware. Corey Crawford might be the most surprising name with a 2.77 GAA, but the fact he was able to post a .917 save % while also facing the 3rd most slot shots per game and 5th most inner slot shots per game is extremely impressive. Binnington proved his Cinderella run to the Stanley Cup last season was not just a fluke with another strong season. And Vasilevskiy continued to do Vasilevskiy things. I was surprised to see his overall dip, but it was mostly due to an 86 Glove Low and 88 Recover, nearly everything else was at least a 90.
BIGGEST RISERS
This is where I want to stress once again that 2019-20 was the only season taken into consideration. They might not be the first names that pop up when you think of quality starters, but all four names were able to put up respectable numbers behind some of the worst defensive teams in the NHL. They all had a big share of the workload, ranging between 34 and 42 starts, and faced a ton of quality shots, all falling in the top 21 in slot shots faced per game. In terms of goals saved per 60, all four cracked the top 20. These overalls would be a different story if we included multiple seasons, but that’s neither here nor there.
BIGGEST FALLERS
Devan Dubnyk had a difficult season as he dealt with off-ice family issues, so I don’t want to go too in-depth with him. Jimmy Howard started the season just 6 wins away from the 250 career wins mark. 27 starts later, he is still 4 wins away from the milestone. In fact, the last time a goaltender posted a mark worse than his 4.20 goals against average in at least 20 starts, Mario Lemieux led the NHL in scoring with 161 points. For the curious ones, it was Artus Irbe, who posted a GAA with the Sharks in 1995-96. The remaining four fallers all posted save percentages under .900 in limited appearances. Of the 61 goalies to have played at least 650 minutes, those 6 all rank 55th or lower in goals saved per 60 minutes.
IF YOU’VE MADE IT THIS FAR…
…I assume you’re as big of a stats nerd as I am. So here is a Google Sheet including all the new individual attributes for the 61 goalies included in this exercise if you want to import them yourself to your Franchise Modes or if you’re simply curious as to where your favorite goalie stacks up. I hope you enjoy!
Link to the spreadsheet:
https://docs.google.com/spreadsheets...gid=1931576402
Make sure to stay tuned for Parts 3-5 of this series. Up next is a breakdown of the results for skaters on Thursday, which will be followed up by a playoff simulation in NHL 20 with the new overalls and using the new return to play format next Tuesday. Then, to finish everything up, I will outline a few suggestions on how I believe the overall system could be tweaked and improved in future games.
NHL 20 OVERALL OVERHAUL: NEW SKATER RATINGS BASED ON REAL-LIFE DATA
JUNE 18, 2020BY BILLY BERTRAND
NHL 20 Overall Overhaul: New Skater Ratings Based on Real-Life Data
SHARE THIS POST?
Facebook Google+ Twitter
With the hockey world going into lockdown in March, right as the playoffs were about to kick off, I turned to EA Sports NHL 20 for my hockey fix. Using Sportlogiq data, I decided to rework the overall system and assign new attributes to every NHL player based on their real-life performance. In Part 3 of this 5 part series, I will break down the results for skaters. Make sure to check out Part 1, where I detailed the method behind the madness, and Part 2, in which I break down the goaltender results.
Coming up with a new system for NHL skaters was by far the hardest part of this whole exercise. First, the number of players and the amount of data taken into consideration was massive. Second, it was harder to do multiple iterations since, with each attempt, I needed to input all this new data in NHL 20 by myself, as roster sharing isn’t yet implemented. Last, and by no means least, boiling every player’s talent to a single number is almost impossible.
There are so many different ways to be an effective hockey player. How can you grade a shutdown defenseman like Niklas Hjalmarsson and a pure sniper like Alex Ovechkin on the same scale?
At first, I attempted to include as many attributes as I could in my rework. However, I quickly realized that a few stats made this a near-impossible task: Offensive and Defensive Awareness. These two numbers have a huge impact on the final overall and thus, nearly every elite player has a high rating in both. They are also very vague categories that are tough to quantify and the definition changes from player to player. Anders Lee setting up in front of the net is a smart play, but if Johnny Gaudreau did the same, it would be a waste of his abilities.
Let’s take Ovi as an example. Now, I don’t think anybody would claim Ovechkin is a defensive ace. Yet, his defensive awareness stands at a very good 87. That’s just one point lower than Phillip Danault, who has established himself as one of the best defensive centers in the game over the past couple of seasons. Danault even had some Selke recognition in 2018-19, finishing 7th in voting, including one vote for first place.
In one of my first iterations, Ovechkin’s defensive awareness graded out at 78. Now, I don’t think that’s too bad of an evaluation for the Russian’s defense. The problem comes with how that attribute translates in-game. Ovi starts out as a 92 in EA’s latest roster update. If I simply drop his defensive awareness by 9 points, he falls all the way down to an 89. If I include that same change with the other adjustments I made, he drops down to 86.
This issue continued to pop up for pretty much every high-rated player. It would turn a lot of elite offensive talents with average defense into good but not great overalls, and inversely could make some role players jump way up in rating. Matt Grzezlcyk became an 85 in one of the early iterations, for example, and was nearly as good as Torey Krug, if not higher rated. This in turn led to some wonky lineups when the CPU would go “Best Lines”.
So I decided to scale down the edits and try to find a balance between overalls that pass the eye test and adjustments that reflect real-life performance. This way, there would be no chance of superstars being slotted on the 2nd or 3rd line due to poor defense. Here are the attributes I decided to tweak in the end and what I took into consideration for each.
Speed: While the NHL’s chip tracking system isn’t implemented yet, I judged speed by how fast a player could play rather than pure straight-line speed. So I included things such as rush chances, end-to-end rushes, etc… The results aren’t perfect, but until we get the actual speed of every player, this is as close as I could get.
Body Checking: I took some creative liberties with this one and decided to go with effective hits, or hits that separate a player from the puck, rather than the pure hits stats as the NHL tracks it.
Puck Control: Here I included things such as puck on stick time, the ability to maintain the puck through stick check attempts, and the amount of touches a player gets.
Passing: Passing is an obvious one, but it ended up being the category with the most data points taken into consideration, as Sportlogiq tracks a very large amount of different pass types. They were also weighed pretty differently per position, with defenders getting a bigger boost from stretch and outlet passes while forwards had more weight placed on passes to the slots or for one-timers. I also added assists, to boost the top playmakers in the NHL.
Strength: Since I can’t ask all NHL players to put up as many bench press reps as possible, I graded strength on their ability to win puck battles and maintain the puck through contact.
Faceoff: This was another very straightforward category, although i did add some extra weight for players who excel in defensive faceoffs.
Hand-Eye: I stayed simple on this one and went with a player’s ability to deflect pucks. This is probably one of the stats that hurt superstars the most, as they don’t necessarily park themselves in front of the net.
Shot Blocking and Stick Checking: Nothing really crazy here, both are as simple as it gets.
Discipline: Once again went pretty simple here and took the number of penalties taken per 20 minutes of play. Discipline barely affects a player’s overall but it will be interesting to see how impactful it is in simulation.
If you want to know more about the process I used, check out Part 1 here, where I go in more detail as to how I determined the new attributes. I left out the non-playoff teams from the exercise as a time-saving measure since they won’t have any impact on the playoff sim coming up in Part 4. You can still access the reworked attributes of non-playoff teams in the spreadsheet linked at the end of this article, although the overalls are not included.
THE RESULTS
Here are the new overalls for the 24 playoff-bound teams:
Lines and pairs were determined by the game’s “Best Lines” system to be even for every team. A few players got called up from the AHL when relevant and others, such as Dustin Byfuglien and Brendan Leipsic, were removed for obvious reasons. I also Left Mark Pysyk and Brendan Smith untouched, as both played mostly forward this year, but the game still classifies them as defensemen, so their numbers would be heavily skewed by that.
Like in the goalie adjustment, which you can find here, I want to stress the fact that the final overall number was out of my hands. I controlled the attributes, but EA’s algorithm made the decisions as to which ones impacted the final number the most. 2019-20 was also the only season of data to be taken into consideration.
High overall players took a hit around the league. In the default roster, nearly every superstar has above average to great defensive attributes to help take them to a final number that fits their real-life impact. My edits took their defense down to reflect their contributions this season and cost them a few points.
Cale Makar is a good example. The Avs defenseman was having an outstanding rookie season with 50 points in 57 games but the reworked attributes dropped him one overall point. However, this does not do justice to the changes I made.
Yes, his Stick Checking, Shot Blocking, and Body Checking ratings dropped 4, 7, and 7 points respectively, but his offensive statistics are now through the roof. He received a +4 Speed (now 97) and Passing (93) as well as a +6 to Puck Control (94). In my opinion, this makes him a much more dangerous defenseman and reflects his actual playstyle pretty well.
As I mentioned earlier, Alex Ovechkin is another player that took a nosedive in rating, dropping 4 points from 92 to 88. The issue is that Ovechkin was already highly rated in nearly all attributes, so anything that he still graded out as elite didn’t rise that much, while areas he graded out as more average, like Stick Checking or Passing, took a big fall and hurt the final number. He did jump from 88 to 94 Speed and kept his Body Checking rating in the 90s, so he will likely be just as dangerous and fun to use, if not more.
AND IF YOU’RE A FULL-ON DATA NERD LIKE ME…
…Here are the full spreadsheets of the edits done in this new roster.
https://docs.google.com/spreadsheets...gid=1165548428
Like I mentioned earlier, I did not input non-playoff teams’ data in NHL 20 as a time-saving measure but their new attributes are still there if you wish to use them, the only thing missing is the change in overall they cause.
https://docs.google.com/spreadsheets...DZTA/htmlview#
08-04-2020, 08:21 AM
