Why Tracking and Analytics Can’t Tell You What to Do

This blog was originally published in 2013.

Tracking systems and performance analytics are able to tell in astounding detail what teams and players did in past games. It has made many overestimate the systems’ ability to tell what teams and players should do in future games

There seems to be an underlying assumption that this tempting formula is now justified:
-> Take the data provided by a tracking system.
-> Analyse the data to find out the most efficient tactics for different situations.
-> Turn the analysis into a game plan by emphasizing the most efficient tactics.
-> Execute the game plan.
-> Win.

In this context, “the most efficient tactic” (or “optimal way”) refers to the tactic that produces the value of a possession most favorable to the team. Hence, on offense, efficiency means maximizing the number of points per possession or offensive rating (ORtg), and on defense minimizing it.

Unfortunately the formula presented above is unrealistic. I can think of 12 partially overlapping reasons why.

1) If opponents have the same data available that you have, the probability of winning may not be improved by the data and the analysis alone.

2) What has been the most efficient tactic in the past may not be the most efficient tactic in the future. That is e.g. because the law of diminishing returns.

3) The returns of different tactics will diminish at different rates. Hence it is hard to tell e.g. how much more frequently an exceptionally successful tactic should be used – or even if it should be used more rarely in order to protect its value.

4) A statistical fact may be used to justify multiple decisions. Say the analysis suggests that you should be taking more shots from the paint. But what actual tactics should be applied to make this happen?

5) It is hard to tell how the efficiency of different tactics may be influenced through practice. Basically, it is easier to improve tactical issues than technical and physiological ones.

6) You’re up against a thinking enemy. You will not be able to tell what the other team will do to change the odds.

7) In some cases, maximizing the average expected ORtg is not the best strategy to optimize the chances of winning. E.g. if you’re the underdog, you should choose a risky tactic. It will not necessarily optimize the ORtg but rather it will increase the variance.

8) Even if you think in the terms of the average, the seemingly most efficient tactic may not be the most advantageous one. In other words, a tactic may produce the best ORtg, but it may come with side effects (fouls, energy used, etc.) harmful enough to wipe out the benefits.

9) The sample size may be so small that the results of the statistical analysis are random. E.g. even if you run a play ten times in a game, the difference between a good ORtg and a bad one may still be just one make or miss.

10) The potentially most efficient tactic may never have been used. E.g. if a team uses exclusively man-to-man defense, there is no way to calculate the efficiency of a zone defense.

11) The efficiency of a tactic may depend on the usage of other tactics. E.g. a trapping defense in a pick-and-roll situation may be the more efficient the more it differs from the team’s standard PNR defense.

12) The nuances of any given situation may be more important than the overall averages, and those possibly contradictory nuances are too numerous to be considered simultaneously in a statistical analysis.

One thought on “Why Tracking and Analytics Can’t Tell You What to Do

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s