Fair warning: Long read ahead. If you hate my long, rambling posts just abort. If you are on a mobile device not great for reading or just short on time, maybe return and read it later.
Eric Tulsky and his role with the Carolina Hurricanes
Just to make sure everyone starts on the same page, Eric Tulsky works for the Carolina Hurricanes with the unassuming title of “Hockey Analyst.” Die-hard fans know who he is, but because of his somewhat secretive role that sits behind the scenes and out of the spotlight, I figure many fans may not even know who he is. (And that is perfectly okay. Not everyone should take their Canes hockey hobby to crazy levels. :-))
At a basic level, Eric Tulsky is responsible for statistical and analytical work for the team. Exactly what he does is a secret, but clues here and there at least provide a rough idea. Prior to joining the Hurricanes initially part-time just before the start of the 2014-15 season, Tulsky was one of the people at the forefront of the NHL analytics movement that started in the public domain. On August 20, 2015, after one season in a part-time role, Tulsky was hired into his current position on a full-time basis. His work before joining Hurricanes included analyzing in-game play (focus on offensive zone entries and their contribution to offense) and player development analytics (looking at success rates for drafting goalies and also analyzing peak production ages for players) among other things. Those projects offer a glimpse into what Tulsky thought was important and also demonstrated the diversity of what he analyzed. But now three years after he was pulled into the Hurricanes organization and went quiet, what exactly he is working on is wild, speculative guesswork. One can be pretty certain that it has changed significantly. Very little is said of specific projects with the Hurricanes, but snip its of comments from Francis, Peters and Tulsky himself suggest that his work still covers a wide range seemingly with an ‘If statistical analysis can help, I am on it’ kind of mentality. Ben Pope from the News & Observer wrote a great article about Tulsky and his work on October 6, 2016 that offers additional insight into Tulsky’s role.
Analytics in today’s NHL and Tulsky’s potential to be a difference-maker
As much as he mostly hides in the background and works on secretive projects, Tulsky’s role in the Hurricanes’ success should not be underestimated.
Statistical and analytical work in the NHL is still very early in its maturation. There have already been multiple shake ups in the analytics ranks at the NHL level with Montreal and Edmonton hiring and then firing pretty quickly. Some teams are clearly struggling to figure out if/where/how analytics fits into their organization and how to leverage it to gain an advantage or at least not be at a disadvantage. On the other side of the coin, some teams are certainly figuring it out faster than others and gaining a measurable advantage. Just like with the NHL analytics movement in general, the teams that are winning will not often make headlines. Rather than bragging about their wins, they will work even harder to keep what they have learned quiet and away from other teams who are still grasping for straws. In the process, the leading teams will do everything that they can to extend the time that they have an advantage as far into the future as possible.
The genesis of hockey analytics and its early development certainly featured significant cross-pollination when the vast majority of the work occurred in the public domain. But when you fast forward to today, enough of the early leaders and many of the brightest minds have been out of that domain and have been publicly quiet for multiple years. It is personal speculation, but I feel confident in believing that what individual teams are doing right now varies widely. Only after analysts begin to move across teams and share ideas will the process to normalize across the league and as long as the winners keep their leaders, the sharing of best practices will be incredibly slow. That affords the leaders a window during which the disparity from team to team offers significant potential for some teams to gain sizable advantages while others are at a disadvantage.
Put more succinctly, I think the next 3-7 years are high time for ‘hockey analysts’ to make a much greater impact than most will ever know (because of that secrecy about what exactly they are doing again). Eric Tulsky is the primary principal in that game for the Carolina Hurricanes organization.
Reasons to believe that the Hurricanes will be one of the winners in the analytics-based ‘moneypuck’ game
Even with the challenges and murkiness of assessing the impact of a hockey analyst like Tulsky and mostly a lack of information about what he is even doing, I feel strongly that the Hurricanes have much better than a fair chance to be one of the teams that not only keeps pace but actually gains an advantage.
Here is why. First, I think that by acting fairly early, the Hurricanes added one of the leaders in the field, most significantly in terms of being able to creatively identify potential areas that could make a difference.
Tracking data, managing and farming data bases, running numbers and ultimately gaining insight is the end point of a process. There is significant professional skill involved in accumulating, managing and analyzing data. And once you get to the point where you know what data to collect and how to analyze it, that skill is necessary and valuable. But in the NHL realm which I think is still fairly early in the process of figuring out what even to analyze and collect, the skill set that matters most is the ability to creatively figure out what from among dozens of projects has the greatest potential to make a difference. The game right now is not about taking a massive amount of data and and then crunching the numbers by formula or known process to arrive at an answer. Rather, the game right now is about identifying areas of research with the most favorable mix of time investment required, probability of yielding useful information and the magnitude of the potential impact of that information.
More directly, I think right now it is much more about figuring out what even to collect and analyze and much less about doing the calculations and analysis. Importantly, the skill set for these two related tasks is quite different. The latter skill can be learned and is very much something that could be hired for even without the need for creativity or hockey insight. But the former skill set, “figuring out what even to collect and analyze” in a role that is still incredibly new requires creativity and thought leadership to prioritize what even to work on out of a multitude of possibilities that could theoretically help improve an NHL team. Broad areas could include any/all of drafting players, rating players for potential acquisition via trade or free agency, evaluating the current roster and player combinations, assessing opponents and many more.
As an early leader in the public hockey analytics domain and a demonstrated leader in terms of setting directions for what even to analyze that was followed (and still is) by many others, I like the Carolina Hurricanes odds that Eric Tulsky is at the forefront of the NHL in terms of identifying what even to analyze.
Second and maybe even more important than the raw smarts and abilities of Eric Tulsky and anyone else that the Hurricanes hire is the people part of the equation. When Eric Tulsky and the first wave of ‘hockey analysts’ were being hired to join NHL teams, it was uncharted territory. What exactly they would even do was loosely defined at best. How exactly this information would be incorporated into coaching, scouting and other hockey operations was something to be figured out. And how and to what degree different types of analytical insight could benefit a team was yet to be determined. Prior to NHL teams making the jump, there was often (not always) an antagonistic battle between some hockey traditionalists who saw no value in analytics and (some) members of the analytics community who could be antagonistic toward ‘old school’ ideals.
In a professional environment, one would hope that there was more of an effort to work together, but many of us know from our personal work experience that open-mindedness, the ability to work with other people and other soft skills are not distributed evenly across the human population. Especially in an environment of uncertainty with goal of developing a working model that includes a mix of skills from two different worlds, I actually think that the people skills are as important as the raw smarts being brought into the organization. And that is actually one of the key areas where I think the Hurricanes will win and vault up toward the top of the NHL in terms of leveraging analytics.
We are not privy to the details of what Eric Tulsky is working on. Nor do we have detailed information about the working relationship between Ron Francis, Bill Peters and Eric Tulsky. But we have anecdotal evidence to suggest that the relationships are good ones. First, Francis made the move to hire Tulsky after one year in a part-time, seemingly more consulting-type role. Francis could probably have forged farther down the road part-time if things were bumpy in the early going. Instead, he move quickly to make Tulsky a more permanent part of the team. Second, small comments in interviews seem to suggest the trio interact regularly. Most notable was Peters’ comment on an interview last summer in which he recited Eric Tulsky’s phone extension number which he had memorized. Importantly, we also have a pretty good read on Ron Francis’ ego and diplomacy. It is partly opinion on my part, but I think most would agree that Francis is highly unlikely to be a dogmatic old school type who was unwilling to consider new ideas. Finally, in setting a rebuilding plan three years and holding steadfastly to it with patience and perseverance, Francis shown a propensity to set a plan in place and patiently see its course. In inserting Tulsky’s analytics into the traditional processes, there was very likely some trial and error and even failures figuring out how to adjust the process and get positive results. Francis’ patient personality should offer time to absorb small failures on a path to success even if it takes some time.
Netting it out
When I boil it down, it is two things:
I think the potential for analytics to make a significant difference in building and managing winning NHL hockey teams over the next 3-7 years is incredibly high and underestimated because of its secrecy and behind the scenes nature.
I think the Carolina Hurricanes have a unique combination of professional skill and equally importantly people skill/personalities that give the team a strong chance to  to be among the NHL leaders in this area (maybe without anyone really ever knowing) .
What say you Canes fans?
Is Eric Tulsky’s role and the black box that it lives in just too unknown for anything more than unverifiable Hurricanes blog material to help fill the offseason? Or do you think my thoughts could have merit even if the details are sparse?
Recognizing that analytics is just part of the equation, how significant of a difference do you think it can make for teams that are better at it?
It is just wild speculation, but do you agree with my opinion that what individual teams are doing varies wildly and that there is likely a significant disconnect at this point between what is the latest in the public domain and what some teams are doing?
Go Canes!
Matt. Your thoughts definitely have merit. There are a few areas where a team that is willing to completely buy in to analytics could have a quick advantage. One such area might be pulling the goalie for power plays to create 6-on-4 (haven’t actually seen the numbers but did somewhere a few years ago it has value). Let’s say that doing so increases the likelihood of scoring by 20%. So a team that scores 30 PPGs a season would score an extra six. Now it might also increase giving up short-handed goals by 66.6%. But if a team was only giving up 6 SHGs a year, then that is 4 extra SHGs. So the net is 2 extra goals. The problem is getting coaches to buy the numbers and fans not to revolt when a SHG costs a couple of games in an obvious manner.
I think you are slightly overemphasizing the “black box’ nature of analytics. Yes hockey is behind baseball, and even football but the areas for significant improvements are pretty obvious–so once a team starts doing them, other teams can adopt the strategy within the same season. I don’t know if there is a big advantage to be gained in talent acquisition–McDavid, Matthews, Laine, et. al. were known to be difference makers without much analytics. The biggest advantage is probably in talent deployment. I remember reading on 538 that Ben Zobrist was the single player with the most WAR per $ in baseball. And when the Cubs added him to their great young talent you can see what happened. So Tulsky’s highest value should be in helping the Canes’ management determine who is the one bargain addition to add as the team starts paying market value for all the players who are RFAs or UFAs in the next three years (just an aside, if Skinner puts up two more 30 goal seasons his value is probably around $8 million in 2019).
I do remember reading that hockey is the sport with the most randomness. This is true because skating increases the speed of the participants; the puck’s shape; and the fact that on scoring plays multiple players touch the puck–including, quite often, the defense. So being able to “set up” advantageous situations is less controlled in hockey than in other sports where a team can position players prior to any action (or in basketball at least prior to the scoring opportunities). For instance, analytics might indicate that a certain goalie gives up 50% more power play goals on shots from his left. Carolina can definitely try to focus on shooting from that area, but too much focus would diminish the overall effectiveness of the man advantage if the team passed up good shots from the other side.
The one thing I hope you are correct about is the Cans’ willingness to buy into Tulsky’s analysis. I stated in an earlier post that while I am a big Peters’ fan his comment about “AHL-level” players might show a bias toward veterans when the players who most negatively impacted the team last year were likely Hainsey and McClement not DiGussepe or Wallmark.
As you mention, we see analytics in the business world and many don’t accept its conclusions. I work in insurance. For the last 25 years we have known that being in an auto accident that is another driver’s fault increases the likelihood of being at-fault in a future accident. Most people don’t believe that because they don’t like it. But when you think it through it makes sense based on people who are “victims” of accidents are usually such because they are 1) on the road more or 2) driving in areas that have more congestion and/or are more difficult to drive in. But no DOI will let a company charge a driver more for being hit–it just seems unfair.
Let’s hope the GMRF and company are willing to listen to Tulsky even when his analysis seems to go against “good sense.” If that is the case, Carolina might well have an advantage on other teams who are Serious Hockey People.
My gut feeling about analytics, and your assumptions is SKEPTICISM… I’ll try and explain- since we have nearly zero evidence of how it HAS ALREADY HELPED… we can’t make informed decisions about its efficacy. Maybe you have unspoken knowledge or information, or at least anecdotal evidence of that positive results, but the way I read your article, it seems to be circular logic! Because the Canes are early to use it and we aren’t told how effective it is…IT MUST BE EFFECTIVE…huh? We’ve all heard the joke about “lies, damn lies, and statistics… and ANALYTICS seems to be just a fancy word for statistics, so right now I’d take the subject with a grain of salt. I see why you feel positive about using it, and the potential is great, no doubt, but “seeing is believing”! Until the team is winning, and can demonstrate cause and effect, I’ll maintain my skepticism…
Puckgod…I think you are officially our resident message board skeptic, and I thank you for having the courage and candor to poke at (literally) everything I write to keep me honest and remind everyone that much of what I write is an opinion that has different views. 🙂
Matt, thanks I Do try… it’s not personal, and you do an awesome job, so keep at it. I’m not totally disagreeing or ignoring the potential for analytics, but it would be nice to see some success from it’s use that could be verified, eh? Sorry…
Puckgod, you’re funny. Here’s hoping your 60 years of hockey love is followed by decades more making me laugh on these comment boards.
CTCaniac, I respectfully disagree with your ‘talent acquisition’ argument. Sure you don’t need analytics for Laine, Matthews or McDavid, because they were top picks. Where analytics come in is in player comparisons during free agency, figuring out draft boards and trade targets. I would imagine it’s more useful in game situations as well, but analytics can be used in acquisition too. Also, I’m fascinated with your 6-on-4 scenario. I would love to see the numbers on that
Matt, I think you’re right to call more attention to this, because analytics are the ONLY way to take human bias out of the talent development and acquisition game. I mean the analytical effort led to the advent of the Corsi stat (around which Bill Peters seems to model his puck possession strategy) and that’s just the entry to a very deep rabbit hole. The fact of the matter is, a lot of people are quick to discount analytics, because its really freaking hard. You have to think in a certain, data-driven way that many people (especially myself) just get overwhelmed. Still, while you may have to be a true numbers junkie to really understand the statistical formulas, the fact is Eric Tulsky is an essential part of the Canes staff, even if we don’t know what he does.
Why? Because NUMBERS DON’T LIE. There isn’t guesswork when it comes to numbers. They aren’t wrong. They can sometimes mislead, but even that is usually a result of human error. You can’t just use Goals, Assists, Points and +/- to evaluate players, because there are so many scenarios where players can help or hurt their team outside of that, plus it completely undervalues the defensive part of the game.
You can’t just use the human eye to evaluate, because the brain will always see what it wants to see. It’s called scotomization in psychology. Analytics eliminate those human biases and, however the Canes are using Tulsky, the data-driven insights that he provides are guaranteed to give us a significant edge in preparation over any team that doesn’t value them.
Fogger. Thanks for the respect—but we actually agree. I knew as soon as I posted the comment above that I wasn’t at all clear in how I see acquisition and deployment. My point about having a top pick and getting a McDavid, is that you don’t need any analytics to make the correct call—you simply acquire the best player. This is true for the top 4-5 picks in most drafts. But after that point, draft picks are just another asset along with prospects and players on the current NHL roster. So the question is how best to deploy those assets. For a team that is one step away from the top level in the NHL, it makes sense to deploy picks and prospects to add one missing piece. For a team in a rebuild, it might make more sense to trade a veteran for more picks. In both cases I would listen to the analytic folks who make the case that a #12 pick is statistically less valuable than a player who has three NHL seasons averaging 10 goals. Or the quants might suggest that a goalie should never be drafted in the first two rounds—a poor deployment of assets based on analysis over many years.
Of course, players need to be acquired through the draft, trades, and free agency. Personally, I think about anything other than a top 5 pick as deploying assets—and that is where analytics should be a real advantage.
That being said, I think Matt below is being too optimistic about the black box. Again, I work in insurance, our very best analytics can only tell us that if we insure 100 businesses that are “elite” we should have a good loss ratio. We cannot tell which one or two will have six-figure losses. Using that logic, I don’t know if even the most advanced analytics can differentiate among three players as possible first-round picks. My guess is that the analytics will, hypothetically, tell Canes’ brass that the long-term success of D-men drafted in the 3rd round is almost as high as that of those drafted in the 2nd, while the opposite is true for left-wingers. So if both rounds have a choice between D and LW, take the LW in 2nd and D in 3rd. The goal of analytics should be to provide better overall outcomes not individual successes.
The NHL draft is even more complicated in that players are mostly 18 or 19. Football at least has the advantage of drafting players after 4 years of play at a somewhat higher level. And it is the competitive success and not necessarily quantifiable numbers that indicate the best prospects. Often the “beast” of the combine turns out to be a below-average professional.
Fogger is correct though is saying the big advantage is reducing bias in decision-making. I would hope that everyone involved in Carolina talent acquisition/deployment would read “Thinking Fast and Slow.” What we think we know most often does more harm than good—as Fogger says trust the numbers.
Yes, I believe ANALYTICS is a fancy word for statistics. I think it could help but there is a caveat. I relate it back to the financial world. There is statistical analysis and fundamental analysis. The thing is there is no right answer. Within statistical analysis there are many measures of looking at things. At the end of the day they are all just clues and you use all of them in conjunction. Not any one indicator is the answer.
With ANALYTICS, players will change over time and even in a given season. It is a moving target but you can try to find trends. It is like trying to write an equation for emotion, you cannot model that, it never works. The point is ANALYTICS by itself will probably fail but you take that additional input and melt it together with your human opinions and other measures you have and it will probably help a little in making decisions. My point is ANALYTICS by itself is not the answer. It is additional data points.
I could see it helping when deciding what to have on the ice for a given situation. I could see it in assessing whom to trade and whom to target for draftees or UFAs. Bottom line, it is only one of the measurements which will help make a decision.
I do think ANALYTICS can make a difference from a better informed decision standpoint. It is not the holy grail, it is additional data.
Do I think it is all over the map the way different teams are doing it (or not), yes. Everybody will have their own secret sauce. It will be a constant tweak as things about players and the league change.
I like the financial world analogy. I am NOT privy to what exactly Tulsky is working on, details of what he is finding/not finding or any specifics. But my hunch is that just like in the financial world, he spends a lot of time exploring things looking for an edge and that very many of those turn up empty. But if he finds even a handful of nuggets of knowledge especially related to ranking or evaluating pre/post-draft prospects it could make a big difference over time.
I cannot prove it, but I think there is an opportunity for smart, and at least equally important, creative people to make a difference from an analytics angle. I won’t be able to prove that until 15 years from now when a few books come out, but especially given the modest cost relative to player salaries/the business in total, I would invest in analytics and also have some patience with it. I think the potential upside is huge.
All of the above are reasonable opinions on the viability of analytics in hockey. I’ll just add this observation. The accuracy of the source data used in the analysis of a hockey player’s ability to me is questionable. Hockey is such a team sport that a player’s on ice results are affected by himself and 11 other players all of different abilities reacting to situations on a split second basis. I agree that if data is available it can be used to HELP analyze performance, my concern is with the ability to collect accurate data. Everyone writes articles on the use of hockey data, but you never see any articles explaining how the source data being used is collected and verified for accuracy to start with. Maybe an article on how this is done would help in deciding how much reliance one can place on the analytics.
Billy Beane’s original breakthrough in baseball could be reduced to something this simple: build a roster of hitters that is good at not making outs and pitchers that are good at not allowing runners to reach base. How either happens doesn’t matter much.
Eventually, someone is going to start winning hockey games consistently by having a breakthrough that can be reduced to something that simple. Hopefully, it’ll be Eric Tulsky that produces that insight and it’s the Canes that benefit.
I wouldn’t bet against it happening; the real bet is when it happens.
Matt, I enjoyed the original article as well as the comments.
As one who has worked in analytics I wonder if:
1. The Canes limited number of scouts (if I remember they have one of the lowest number in the NHL) is related to the use of analytics.
2. If we won’t find out one day that the Canes have used electronic trackers sewn into the jerseys etc to track player movement. It’s pure speculation but imagine the additional metrics you could glean.