Crashing the Crease: Statistical Deviants

2011-12's Best Value Goaltender

2011-12′s Best Value Goaltender – Image Courtesy of Christian Petersen/Getty Images North America

When I started writing this week’s article, I had initially planned to try and give some insight into the pressures of goaltending and how a massive contract can impact a guy. I kind of veered left, though, as I looked for a way to quantify not just a goalie’s performance, but his overall value to the team.

It may seem pretty straightforward, but to fully understand it, I felt that some simple metrics could be combined to give a better total insight.

Using, I pulled the cap hits for the goalies with the top 25 cap hits for the 2012-2013 season, but listed their cap hits from the previous season. Since I view cap hit as a real measure of a player’s perceived worth to a team, this was an obvious place to start, as it gave me a ranking based on perceived market value. By using their upcoming cap hit, I get an idea of their most recent market valuation, while showing the actual cap hit from the previous season shows who may have been providing more value to his respective team.

From here, the plan was to simply create a metric that should correlate to overall value by looking at three separate aspects: (1) Save percentage [Sv%], (2) relative shot count intensity relative to the other top 25ers [RelSCI] and (3) percentage of games played out of 82 [GameCtAdj].  Basically, Sv% tells me how effective they were at a base level, RelSCI tells me how “hard” their workload was and GameCtAdj should tell me how much of a workhorse they were.

Since I made up (as far as I know, at least) two of the three metrics, I’ll take a second to explain in greater detail.

Save percentage is straightforward. I won’t waste time on that. Relative shot count intensity, again shown as RelSCI, is a pretty simple metric. What I’ve done is calculated average shots per game played for all of the top 25 and then normalized these values to the average of all of these. See the table below for more detail:

As an example, Henrik Lundqvist, who we’ll come back to later, had an average shots per game played of 28.3 according to my numbers. RelSCI, shown on the right, shows a 1.00 for him, which means his average shots per game played (28.3) was almost identical to the total average shots per game played (28.2). Workload-wise, this provides a simplistic argument for how much effort a guy has to put forth in an average game. There are some wrinkles to this approach, though.

First of all, not all shots are created equal.

Martin Brodeur, in his prime, may have seen a relatively low shot count, but with how the Devils played, the ones that did get through were of slightly higher quality. Secondly, shot count can actually reward lower-quality rebound control. I’ve seen the argument posited before that a guy like Luongo actually has an artificially inflated shot count, as his primary focus is not necessarily on trapping the puck, but more on blocking it. Following this train of thought, it could be argued that he creates more rebounds and shot opportunities. Finally, it doesn’t explicitly capture the strength of the team in front of a guy. Or at least, I believe, it may appear that way at first glance.

When the RelSCI for a goalie like Brodeur or Bryzgalov is looked at, it’s notably lower than 1.00 relative. Meanwhile, a guy like Cam Ward has a very high RelSCI. This would actually seem to make sense if analyzed from a team strength perspective, as the Flyers and Devils very likely had a stronger defensive corps than the Canes did.

So that’s RelSCI. It’s not perfect, but it does at least simplistically capture a goalie’s workload. The second stat created is a lot more straightforward, and that is GameCtAdj. Assuming an opportunity to play all 82 games (i.e. no lockout or unplayable games), it’s nothing more than the percentage of games a goalie actually played, seen below:

This number alone can actually speak volumes about the value of certain goalies, such as Pekka Rinne, who played or appeared in an astounding 89% of his team’s games. Conversely, Rick DiPietro, who is in this list again because of cap hit and not because I think he’s anywhere near the top 25 in the NHL, played an overwhelming 10 percent of the Islanders’ games.

The next step was to calculate all three values for each goalie (Sv%, RelSCI and GameCtAdj) and then multiply them together to get an overall market “value.” Since they can all be represented as decimal values around one, the resulting value isn’t too hard to understand. A guy who saves a lot of shots, sees a lot of shots and plays a lot of games will ultimately have a higher market value. While not groundbreaking, there is some insight to be had here. Take a look at how this system compares with the cap hit ranking for each guy:

What really jumped out to me in looking at this was that the guys who my system would rank two through six are not overwhelmingly appreciated or lauded goaltenders. While Ward and Kipper get a decent amount of press, they’re largely overlooked due to their teams’ performance or their market. Adding in the 2011-12 cap hit really drives this point home. To make it a bit more interesting, I took the total cap hit from last season (not salaries) and proceeded to redistribute it based on this new ranking (Sv% x RelSCI x GameCtAdj).

The bargain goaltenders from last year, some of whom that are now no longer bargains (Rinne), really start to jump out at this point. The column all the way on the right (click the image to see in better detail) shows the difference in a particular goalie’s cap hit versus the adjusted salary cap hit calculated using the new ranking (green means value, red means closer to DiPietro territory):

What was really interesting to me, getting back to Lundqvist, was how he started out ranked right where his cap hit placed him based on save percentage. Great for Rangers fans, right? Not necessarily. Factor in the RelSCI and Sv%, though, and you see he drops to the 11th spot. What does this say? Am I really arguing that he’s not a top goalie in the league?

No, I’m not. What I’m saying is that Lundqvist is not a value goaltender. You pay for what you get from him. Also, he has a good backup, because the Rags finished first without his having to play in a quarter of the games. Beyond this, his RelSCI reveals that he sees fewer shots than his counterparts.  Again, it could very strongly be argued that this should not count as a negative, but I think it can be taken at face value here. Ultimately, using a purely quantitative approach with a linear redistribution of cap hit, you actually get less than what you pay for relative to other goalies out there. That is, of course, ignoring the premium you have to pay for top talent.

Looking at Pavelec, the chart above highlights why Winnipeg gave him the contract they did at the end of the year. His cap hit from last year was low ($1.15M), but he provided much more value than that to his team. Importantly, it also emphasizes that Rick DiPietro’s contract is an outright joke at this point.

From a Flyers perspective, it reflects that Bryz was not an absolute bust, but hit the cap much harder than the value he provided to the team. Obviously, this does not indicate his future performance will be like this.

All things considered, I think a lot can be taken from the above data. I could spend many more hours analyzing it, but I think the last column really does sum it up. A lot of guys who provided their teams with exceptional value last year have been rewarded with new contracts (Rinne, Price, Schneider, Pavelec, Dubnyk), and the ones who didn’t were apparent to most NHL fans.

My next step with this will be to track on a historic basis (at least back to 2008), as it seems that the more underpaid a goalie is, the higher his value-based overpayment becomes when he finally does cash out.

As always, I’m interested in your thoughts on this week’s Crashing the Crease, so feel free to comment or reach me on Twitter (@HeyItsBrenno).

  • penaltykiller

    Great work! It was an interesting read.

    • Justin Brennan

      Thanks, penaltykiller. Appreciate your reading it!

  • Geoff Detweiler

    This is definitely interesting, but I’m worried about how heavily skewed the final list is to guys on bad teams. Guys ranked 2, 3, 4, 5, 7 and 9 all missed the playoffs. Then you have guys ranked 6th and 8th sneaking in as the 8th seed and 7th seed respectively.

    That gives me severe pause that 80% of the top five goalies represent the bottom 47% of teams.

    • Justin Brennan

      Geoff, I definitely see where you’re coming from. Logically, poorer teams will give up more shots and scoring chances. Using shot count instantly adds a big wrinkle by somewhat ignoring the performance of the guys in front of the goalie. The way the system is designed, though, attempts to mitigate that variability by normalizing based on average.

      The range between the top and bottom of the RelSCI chart is not as drastic as it is for the GameCtAdj, which I feel has a pretty big impact in terms of value. In essence, the RelSCI is designed to capture the impact of in-game workload, but not to make it the primary determining factor. I think an adjustment based on Team Corsi or another measure of team performance may give a little insight into how much “work” the goalie’s really doing, but I was reticent to throw in too many variables without putting the initial system out in the sun, so to speak.

      I struggled with this as I put the numbers together, I promise. The reason I decided to run with it was, in looking at the upper and lower half of the goalies, I’d ultimately take the upper half of the goalies based on this value ranking over the bottom half. The notable exception is J Quick, whose cap hit for next year is still low. Were it the $5.8M he’ll make in 2013, he’d have definitely made the top half.

  • JFB

    Geoff this is not tracking how good a goalie is it’s tracking value to the team (contract based model). Worse teams rely on their G a lot more than teams with Malkins, Sedins, and the like who can compensate for a soft goal.
    They also see more shots compared to deeper teams, which makes them stand out since shots faced factors heavily in the model.
    What I see as a problem is noted already in the article and it is that it does not truly separate the performance of the goalie from that of the team in front of him, and that should have a bigger role in value (more than how many games a goalie can handle that’s pretty obvious and obscures talented prospects behind a proven #1 or a tandem system like St Louis which put up some of the best numbers for example). Also SOG doesn’t count blocked shots for example, and that impacts SCI too.

    • Justin Brennan

      JFB, I have to agree.

      The big thing for me was building this approach iteratively. I have a fear of throwing too many variables into the pot, if that makes sense, so it’s definitely simplistic at this point. My initial thought is to take something like Team Corsi and use that to adjust the RelSCI, but I’m definitely open to suggestions.

      I didn’t want to play with that stat too much, as a lot of the work I’ve seen, including that research work done on shot quality, seems to still mimic save percentage when all is said and done. On top of that, I didn’t want to bury an undervalued goalie on a bad team simply because he’s on a bad team. This model’s attempting to be a little contrarian without just saying he who faces the most shots is best.

      Still, all that aside, it does need some refining. I’ll be tweaking it as I move forward to see if there’s a way to account for team defensive performance without removing the impact an overlooked goalie may have.

      Thanks for the feedback!