Anyway, some of the sportsball fans and political junkies in our audience may be familiar with a statistician named Nate Silver, who, among other things, founded statistics site FiveThirtyEight. He recently wrote this substack article about how the Denver Nuggets broke his model.
His model, called RAPTOR, compares contemporary players in their current season to historical analogs. About Nuggets star Nicola Jokic, Silver writes:
With Jokic, on the other hand, RAPTOR has no idea what to do. It can’t find any other players like him…
…So in the entire history of the NBA, all but four players are more dissimilar to Jokic than similar to him, and the other four are just barely on the other side of the line.
There is a perception that statistical modelling is meant to deliver insights that human reasoning cannot - finding hidden patterns in high-order interactions of big data beyond our comprehension. This is mostly inaccurate. Most quant research is about systematizing and automating persistently useful human-style insights, so that they can be applied to arbitrarily large data sets without having to hire lots of expensive human analysts.
Models are meant to deliver predictably precise forecasts within a tolerable error band. The genius of RAPTOR is not necessarily that it has a more accurate opinion on the Steph versus Lebron debate than Stephen A. Smith, but that it can stack rank Onyeka Okongwu (A young center on the Atlanta Hawks, for those asking) at zero marginal cost, which is not worth Stephen A’s valuable time (especially given the extra hours he’ll have to pull after those ESPN layoffs).
So there are two possible reactions when RAPTOR chokes on one of the NBA’s most high-profile and dynamic players:
1) Pffffft….this model sucks. How am I supposed to trust its confidence in Jokic when it has such low and prima facie absurd similarity scores?
2) The model has identified an edge case. I should spend some time trying to understand what the model does not get and figuring out if that is advantageous or disadvantageous.
Number 2 is the correct answer. Silver’s a smart guy - he bet on the Nuggets to win at rather favorable odds.