A few days back I finished reading Nate Silver’s “The Signal and the noise: Why so many predictions fail”. The book was pretty good, albeit a bit repetitive. I most enjoyed the chapter on hurricanes.
In it, Silver discusses the relationship between the magnitudes of earthquake and the frequency of their occurrences.
Looks pretty unimpressive, but when we switch to a logarithmic scale of the annual frequency the relationship (and remember that the Richter scale is also a logarithmic scale) the relationship looks impressively linear:
Silver argues that such a relationship holds regardless of the region considered – if we narrow down the sample, we’ll have fewer data points but something that looks vaguely linear as well.
I find this pretty remarkable. While we are still far from being to tell which city or region is “due” for an earthquake, we can estimate how likely – given historical events – is an earthquake of a given magnitude.
Silverman then argues that this relationship also holds when it comes to terrorist attacks and presents a similar-looking chart for terrorist incidents in NATO countries, between 1979 and 2009.
That I found even harder to believe. Admittedly – the relationship looks slightly less linear (if one can even talk about degrees of linearity). Intriguingly, he decided to drop terrorist attacks with fewer than 5 fatalities, which I found quite surprising, but I figured it just made the chart prettier.
I therefore decided to test this theory and attempt this chart by myself. I thought I would focus exclusively on European countries. I downloaded data on all terrorist incidents from 1970 until 2015 from the Global Terrorism Database. I only included those in which at least 2 people were killed.
Some lines of code later, I was amazed at the result:
An even better fit than the original chart!
When I repeated the exercise only for Western Europe, the picture looked similar, though there were more deviations.
This made me wonder what other variables this relationship holds for.
In any case, how useful is this for forecasting (the main topic of the book)? Useful, if we think the past will be more or less similar to the future. In my view, as scary as the chart is, it is also rather consoling. It points to relatively low probability of terrific death tolls, or at least very very few incidents with very very large death tolls. I really do hope the chart is right, but I also wish there were not enough data points to make it in the first place.