Sunday, June 13, 2004
Objective Statistical Analysis - Breaking the 'Old School' Barriers
Out of all the major pro sports in North American, hockey is, by far, the most 'Old School' of sports when it comes to analysis.
When you talk about a player, scout a player, or compare two players, you generally see terms such as anticipation, hockey sense, balance, "Heart", "He comes up big in big games!" and so on. Rarely do you ever see a meaningful objective comparison or evaluation of players that isn't influenced by some sort of personal bias.
The problem? All of those traits are subjective, and cannot be objectively measured the same way by two different people. Where as I see Rob Niedermayer as a rather average skater due to his lack of first-step quickness and his apparent clumsiness, other scouts see a fast skater who can chew up ice like a sno-cone machine.
Hockey has very few resources for statistical analysis, an area which has always intrigued me. I have planned a few studies on my own, as I had seen nothing out there before. The famed book MONEYBALL: The Art of Winning an Unfair Game, showed how baseball's Oakland Athletics exploited market inefficiencies by using statistical analysis and 'non-traditional' analysis to sign players that other teams didn't value due to lacking 'The Good Face', or the 'Tools' (Speed, Power, etc), yet could perform well enough to justify a roster spot.
As Daryl Shilling put it, "Hockey lives in a completely different world than other businesses. Brokerage firms, for example, review the performances of funds and their workers scientifically, and make business decisions based on study. Hockey
bases almost every one of its decisions based purely on perception, which is
a an extremely limited method of evaluation."
Who is Daryl Shilling? Well, I was absolutely stunned and pleased to recently come across a site called The Hockey Project, where I found a very good counter-argument to my rant for Neely's HOF chances. This site is run by Daryl Shilling, and he has already developed some good studies that could really kickstart a 'Stathead' revolution in the hockey realm.
Basically, there are 2 major areas which present problems in performing objective statistical analysis, and then having said analysis accepted by the fans, media, and powers that be.
Lack of Deep, Meaningful, and Objective Data:
When you want to compare two players, say Martin St. Louis and Jerome Iginla , there isn't a lot of measures out there that can tell you a lot about the seasons they had, in context to the league around them.
From the basic stats readily available:
So, from the stats most commonly available to anyone, what can we see?
Well, St-Louis has more points, a better +/-, far less penalties, and most short-handed goals, while Iginia has more goals, more shots, and more game-winning goals.
So, can we truly conclude, from these basic stats, that St-Louis was a superior defensive player (+/- and SHG), while Iginla was more "clutch" because he scored more Game Winning Goals?
Well, most fans probably would and will conclude that from the statistics. There simply aren't details statistics available to measure defensive contribution, or other 'intangibles'.
A question? Do Short Handed Goals = great defensive play? or sacrificing defense for offense?
I ask this because, subjectively, Pavel Bure is the perfect example of racking up Short Handed Goals at the expense of defense.
Bure had 6 SHG in 97-98, and 34 in his career. From his time on the Canucks, I could say that he often 'poached' while killing penalties, looking for opportunities to score goals. For the opportunity to score shorthanded, Bure would give up attention to the defensive responsibilies that he was assigned. So, Short Handed Goals seem like a truly shaky argument for defensive ability, given the fact that the object of killing a penalty should be first and foremost to 'not get scored on'.
Unfortunately, the NHL does not keep much data that is 'deeper' than what I have provided you above. The only other statistic that I have not shown you is 'Time on Ice', where we can see that Iginla played 2:11 of shorthanded time per game compared to St-Louis' 1:36. This tells you that Iginla played more on the penalty-killing units, but it doesn't tell you who was better.
The NHL comes up 'Short Handed':
If you look at a typical hockey boxscore, you can't get very much from it (Who scored, assisted, shots, saves, penalties). Until recently, boxscores didn't even contain the player's individual +/- and Shots on Goal totals. So, if you look at a historical box score, you can't get very much at all from it other than the basic counting stats.
This is where the NHL comes in. As a custodian of the sport, the NHL has recenly gone backwards in the realm of record keeping. Just a few years ago, we were lucky enough to finally have access to statistics like hits, giveaways, takeaways, and blocked shots.
Unfortunately, the NHL stats keepers at most arenas, like NHL referees, were wildly inconsistant in their application of tabulating the numbers. Some arenas were extremely generous in giving out hits (New York), and some were extremely stingy.
When the NHLPA was brilliant enough to use stats to show the contributions of 'role' players, the NHL, in a move driven by simply pure greed, decided to stop providing these detailed stats to the public. Although the NHL was ordered to provide this information to the NHLPA, the NHL simply refuses to release these figures to the public.
So, in the end, we are left with only the basic statistics again, which makes it harder to develop meaningful measures and objective analysis.
Lack of Mainstream Acceptance:
I am sure some of you are reading this and thinking, "What a nerd, this guy has never played the game and he thinks you can know a player just by looking at some spreadsheets!"
This type of attitude is rampant among baseball scouts and executives, despite the inroads of statistical analysis into many front offices in that sport.
Why is it so hard for 'Old School' types, and mainstream media and fans to accept objective studies?
1. It's too 'hard' for them - People tend to fear and dislike things that they don't understand. I don't tend to like nuclear physics because it's just generally too much for my head to wrap around, and 'Old School' hockey types don't tend to be the brightest lights in the chandalier.
Hockey people tend to know and understand the 'basic' stats like goals, assists, and ice-time. If you try to present some sort of basic formula to them, they will likely have trouble understanding it, and will fight it because it could be a threat to them. (Some baseball teams have drastically reduced scouting departments because they could use analysis to draft players for a much cheaper price).
2. "You never played the game" - It's a strange phenomenon in the sporting world where the league's top executives tend to be former players and coaches. Rarely is a hockey business run by a professional businessman, but rather by one of the 'boys'. The Edmonton Oilers are the perfect example of this, as the team is run by Kevin Lowe, who was very recently a coach and NHL player himself.
The Hockey Realm is a very closed group, and they don't tend to look favourably upon 'outsiders' telling them what to do. Unless you 'Played the Game', these guys tend to think that you couldn't possibly understand the game.
To them, I ask them this: If you never practiced politics, or ran for public office, does that mean you cannot have an expert opinion on political issues?
As for the media, they tend to repeat and regurgigitate the same old tried-and-not-so-true cliches. Their mentality matches that of the 'Old School', so the mainsteam media tend to look at objective analysis with a "King of the Mountain" mentality.
3. It's Boring - One barrier to objective analysis is presenting it in a format that people can understand, and that people will want to read it!
I could post pages of very useful information, but if it's poorly formatted and explained, people will not want to read it, nor come out of reading it with an understand of just what I want to explain to them.
When I perform some studies in the future, I hope to do so in a format that is 'easy to read' and 'entertaining' at the same time. You are more likely to accept a new idea if it at least intrigues you in some way, even if it comes down to me posting photos of Anna Kournikova.
Yeah, that got your attention, didn't it???
When you talk about a player, scout a player, or compare two players, you generally see terms such as anticipation, hockey sense, balance, "Heart", "He comes up big in big games!" and so on. Rarely do you ever see a meaningful objective comparison or evaluation of players that isn't influenced by some sort of personal bias.
The problem? All of those traits are subjective, and cannot be objectively measured the same way by two different people. Where as I see Rob Niedermayer as a rather average skater due to his lack of first-step quickness and his apparent clumsiness, other scouts see a fast skater who can chew up ice like a sno-cone machine.
Hockey has very few resources for statistical analysis, an area which has always intrigued me. I have planned a few studies on my own, as I had seen nothing out there before. The famed book MONEYBALL: The Art of Winning an Unfair Game, showed how baseball's Oakland Athletics exploited market inefficiencies by using statistical analysis and 'non-traditional' analysis to sign players that other teams didn't value due to lacking 'The Good Face', or the 'Tools' (Speed, Power, etc), yet could perform well enough to justify a roster spot.
As Daryl Shilling put it, "Hockey lives in a completely different world than other businesses. Brokerage firms, for example, review the performances of funds and their workers scientifically, and make business decisions based on study. Hockey
bases almost every one of its decisions based purely on perception, which is
a an extremely limited method of evaluation."
Who is Daryl Shilling? Well, I was absolutely stunned and pleased to recently come across a site called The Hockey Project, where I found a very good counter-argument to my rant for Neely's HOF chances. This site is run by Daryl Shilling, and he has already developed some good studies that could really kickstart a 'Stathead' revolution in the hockey realm.
The Roadblocks to Objective Analysis:
Basically, there are 2 major areas which present problems in performing objective statistical analysis, and then having said analysis accepted by the fans, media, and powers that be.
Lack of Deep, Meaningful, and Objective Data:
When you want to compare two players, say Martin St. Louis and Jerome Iginla , there isn't a lot of measures out there that can tell you a lot about the seasons they had, in context to the league around them.
From the basic stats readily available:
St. Louis - GP G A Pts +/- PIM Shots Sh% GWG GTG SHG SHA PPG PPA
82 38 56 94 +35 24 212 17.92 7 0 8 3 8 22
Iginla - GP G A Pts +/- PIM Shots Sh% GWG GTG SHG SHA PPG PPA
81 41 32 73 +21 84 265 15.47 10 1 4 1 8 13
So, from the stats most commonly available to anyone, what can we see?
Well, St-Louis has more points, a better +/-, far less penalties, and most short-handed goals, while Iginia has more goals, more shots, and more game-winning goals.
So, can we truly conclude, from these basic stats, that St-Louis was a superior defensive player (+/- and SHG), while Iginla was more "clutch" because he scored more Game Winning Goals?
Well, most fans probably would and will conclude that from the statistics. There simply aren't details statistics available to measure defensive contribution, or other 'intangibles'.
A question? Do Short Handed Goals = great defensive play? or sacrificing defense for offense?
I ask this because, subjectively, Pavel Bure is the perfect example of racking up Short Handed Goals at the expense of defense.
Bure had 6 SHG in 97-98, and 34 in his career. From his time on the Canucks, I could say that he often 'poached' while killing penalties, looking for opportunities to score goals. For the opportunity to score shorthanded, Bure would give up attention to the defensive responsibilies that he was assigned. So, Short Handed Goals seem like a truly shaky argument for defensive ability, given the fact that the object of killing a penalty should be first and foremost to 'not get scored on'.
Unfortunately, the NHL does not keep much data that is 'deeper' than what I have provided you above. The only other statistic that I have not shown you is 'Time on Ice', where we can see that Iginla played 2:11 of shorthanded time per game compared to St-Louis' 1:36. This tells you that Iginla played more on the penalty-killing units, but it doesn't tell you who was better.
The NHL comes up 'Short Handed':
If you look at a typical hockey boxscore, you can't get very much from it (Who scored, assisted, shots, saves, penalties). Until recently, boxscores didn't even contain the player's individual +/- and Shots on Goal totals. So, if you look at a historical box score, you can't get very much at all from it other than the basic counting stats.
This is where the NHL comes in. As a custodian of the sport, the NHL has recenly gone backwards in the realm of record keeping. Just a few years ago, we were lucky enough to finally have access to statistics like hits, giveaways, takeaways, and blocked shots.
Unfortunately, the NHL stats keepers at most arenas, like NHL referees, were wildly inconsistant in their application of tabulating the numbers. Some arenas were extremely generous in giving out hits (New York), and some were extremely stingy.
When the NHLPA was brilliant enough to use stats to show the contributions of 'role' players, the NHL, in a move driven by simply pure greed, decided to stop providing these detailed stats to the public. Although the NHL was ordered to provide this information to the NHLPA, the NHL simply refuses to release these figures to the public.
So, in the end, we are left with only the basic statistics again, which makes it harder to develop meaningful measures and objective analysis.
Lack of Mainstream Acceptance:
I am sure some of you are reading this and thinking, "What a nerd, this guy has never played the game and he thinks you can know a player just by looking at some spreadsheets!"
This type of attitude is rampant among baseball scouts and executives, despite the inroads of statistical analysis into many front offices in that sport.
Why is it so hard for 'Old School' types, and mainstream media and fans to accept objective studies?
1. It's too 'hard' for them - People tend to fear and dislike things that they don't understand. I don't tend to like nuclear physics because it's just generally too much for my head to wrap around, and 'Old School' hockey types don't tend to be the brightest lights in the chandalier.
Hockey people tend to know and understand the 'basic' stats like goals, assists, and ice-time. If you try to present some sort of basic formula to them, they will likely have trouble understanding it, and will fight it because it could be a threat to them. (Some baseball teams have drastically reduced scouting departments because they could use analysis to draft players for a much cheaper price).
2. "You never played the game" - It's a strange phenomenon in the sporting world where the league's top executives tend to be former players and coaches. Rarely is a hockey business run by a professional businessman, but rather by one of the 'boys'. The Edmonton Oilers are the perfect example of this, as the team is run by Kevin Lowe, who was very recently a coach and NHL player himself.
The Hockey Realm is a very closed group, and they don't tend to look favourably upon 'outsiders' telling them what to do. Unless you 'Played the Game', these guys tend to think that you couldn't possibly understand the game.
To them, I ask them this: If you never practiced politics, or ran for public office, does that mean you cannot have an expert opinion on political issues?
As for the media, they tend to repeat and regurgigitate the same old tried-and-not-so-true cliches. Their mentality matches that of the 'Old School', so the mainsteam media tend to look at objective analysis with a "King of the Mountain" mentality.
3. It's Boring - One barrier to objective analysis is presenting it in a format that people can understand, and that people will want to read it!
I could post pages of very useful information, but if it's poorly formatted and explained, people will not want to read it, nor come out of reading it with an understand of just what I want to explain to them.
When I perform some studies in the future, I hope to do so in a format that is 'easy to read' and 'entertaining' at the same time. You are more likely to accept a new idea if it at least intrigues you in some way, even if it comes down to me posting photos of Anna Kournikova.
Yeah, that got your attention, didn't it???
Labels: Analysis, Iginla, opinion, statistics
Comments:
<< Home
HR
The +/- stat, which I contend is way too undervalued by the league, would be an even better analytical tool for measuring the effectiveness of a player, if combined with a takeaway/giveaway ratio and the old blocked shots stat. You could then build a solid statistical case that might negate a state like SHGs or GWGs by goal scorers who are a defensive liabilty. I also dislike the GWG as a measuring tool, because it (or any goal, for that matter) may have been purely accidental. I strongly believe there should be a team stat for "own goals", which would remove it from the individual goals scored, assist(s) and goals allowed stats. No CEO of a going concern would quantify luck in analyzing the effectiveness of an employee.
The +/- stat, which I contend is way too undervalued by the league, would be an even better analytical tool for measuring the effectiveness of a player, if combined with a takeaway/giveaway ratio and the old blocked shots stat. You could then build a solid statistical case that might negate a state like SHGs or GWGs by goal scorers who are a defensive liabilty. I also dislike the GWG as a measuring tool, because it (or any goal, for that matter) may have been purely accidental. I strongly believe there should be a team stat for "own goals", which would remove it from the individual goals scored, assist(s) and goals allowed stats. No CEO of a going concern would quantify luck in analyzing the effectiveness of an employee.
# posted by 
: 12:38 pm
: 12:38 pm
The stats for me are more of an entertainment value than anything else. As for evaluating individual players, it is rather difficult to get stats that are independent of other factors like other players on the ice. Hockey is a team sport and stats like plus/minus in particular are dependent on the other players that are on the ice. I once won a hockey pool because I picked a third line player who got traded to the top line of another team. He had a career year mainly because he touches the puck on the top line.
I would definitely like to see some stars that are meaningful to describe an individual's play in different areas of the game. In a team game with many specialised positions, it would be quite an undertaking to determine such stats. I think that EA would be better than the NHL at coming up with such a system.
Post a Comment
I would definitely like to see some stars that are meaningful to describe an individual's play in different areas of the game. In a team game with many specialised positions, it would be quite an undertaking to determine such stats. I think that EA would be better than the NHL at coming up with such a system.
<< Home
