UPDATED: FIGURE ADDED BELOW!

Karen Clark and Company has released an interesting report (PDF and covered here) that compares the performance of catastrophe models from AIR (the company that Clark formed and ileft a few years ago), RMS, and EQE over the past three years with their five year predictions of losses. Here is what they’ve found:

Three years into the five year prediction period, all of the near term models significantly over-predicted the number of hurricanes that would form in the Atlantic, the number of landfalling hurricanes, and the insured hurricane losses. While the number of hurricanes is running a bit above average for the cumulative period, 2006 through 2008, landfalling hurricanes are running about 22 percent below average, and insured losses are more than 50 percent below average.

Implications for Model Users

While it is too early to make definitive conclusions about the accuracy of the near term hurricane models, in order for the insured losses to reach 40 percent above average for the five year period, in line with the RMS predictions for example, the next two years will both have to be like 2004, or there will have to be another Katrina. For the cumulative period, 2006 through 2008, insured losses are significantly below average suggesting that there is too much uncertainty around year-to-year hurricane activity to make short term predictions. Hurricane activity is influenced by many climatological factors many of which are known but some unknown by scientists. There are complicated feedback mechanisms in the atmosphere that cannot be quantified precisely even by the most sophisticated and powerful climate models. . .

Three years into the application of near term hurricane models, the model predictions have not performed well. While all three major catastrophe modeling companies predicted significantly elevated hurricane activity and losses for the period 2006 through 2010, two of the past three years have been below average. Catastrophe models are designed to simulate thousands of potential scenarios of what could happen to an insurance company – not what will happen in any given year or short time period. While catastrophe models, used appropriately, can provide credible estimates of a company’s potential loss experience, the models are not able to predict where, when or how big actual events will be. While a definitive conclusion on the near term hurricane models cannot yet be made, early indications are that a five year period is too short for hurricane loss estimation.

Of course, these conclusions won’t be a surprise to Prometheus readers

Note: Our software is acting up and I’ll post up a figure when we sort it out.