« Prediction and Forecasting »

A Second Reponse from RMS

A few weeks ago I provided a midterm evaluation of the RMS 2006-2010 US hurricane damage prediction. RMS (and specifically Steve Jewson) responded and has subsequently (and graciously) sent in a further response to a question that I posed:

Does RMS stand by its spring 2006 forecast that the period 2006-2010 would see total insured losses 40% above the historical average?

The RMS response appears below, and I'll respond in the comments:

Yes, we do stand by that forecast, although I should point out that we update the forecast every year, so the 2005 forecast (for 2006-2010) is now 2 years out of date. Apart from questions of forecast accuracy, there's no particular reason for any of our users to use the 2005 forecast at this point (that would be like using a weather forecast from last week). It is, of course, important to understand the correct mathematical interpretation of the forecast. In your original post you interpreted the forecast incorrectly in a couple of ways. Over the last 2-3 years we've issued this forecast to hundreds of insurance companies, and discussed it with dozens of scientists around the world, and none of them have misinterpreted it, so I don't think our communication of the intended meaning of the forecast is unclear. However, some explanation is required and I realise that you probably haven't had the benefit of hearing one of the many presentations we've given on this subject. The two things that need clarifying are: 1) This forecast is a best estimate of the mean of a very wide distribution of possible losses. Because of this no-one should expect to be able to verify or falsify the forecast in a short period of time.

This is a typical property of forecasts in situations with high levels of uncertainty. I think it's pretty well understood by the users of the forecast.

One curious property of the loss distribution is that it is very skewed. As a result the real losses would be expected to fall below the mean in most years. This is compensated for in the average by occasional years with very high losses.

In fact the forecast that we give to the insurance industry is a completely probabilistic forecast, that estimates the entire distribution of possible losses, but it's a bit difficult to put that
kind of information into a press release, or on a blog.

2) Your conditional interpretation of the forecast is not mathematically correct. Neither RMS, nor our clients, expect the losses to increase in 2008-2010 in the way you suggest just because they were low in 2006-2007. I can't think of any reason why that would be the case. To get the (roughly) correct interpretation for 2008-2010 you have to multiply the original 5 year mean values by 0.6. That's what the users of our forecast do when they want that number.

I hope that clarifies the issues a bit.