Forecast Verification for Climate Science, Part 2
January 8th, 2008Posted by: Roger Pielke, Jr.
Yesterday I posted a figure showing how surface temperatures compare with IPCC model predictions. I chose to use the RSS satellite record under the assumption that the recent IPCC and CCSP reports were both correct in their conclusions that the surface and satellite records have been reconciled. It turns out that my reliance of the IPCC and CCSP may have been mistaken.
I received a few comments from people suggesting that I had selectively used the RSS data because it showed different results than other global temperature datasets. My first reaction to this was to wonder how the different datasets could show different results if the IPCC was correct when it stated (PDF):
New analyses of balloon-borne and satellite measurements of lower- and mid-tropospheric temperature show warming rates that are similar to those of the surface temperature record and are consistent within their respective uncertainties, largely reconciling a discrepancy noted in the TAR.
But I decided to check for myself. I went to the NASA GISS and downloaded its temperature data and scaled to a 1980-1999 mean. I then plotted it on the same scale as the RSS data that I shared yesterday. Here is what the curves look like on the same scale.
Well, I’m no climate scientist, but they sure don’t look reconciled to me, especially 2007. (Any suggestions on the marked divergence in 2007?)
What does this mean for the comparison with IPCC predictions? I have overlaid the GISS data on the graph I prepared yesterday.
So using the NASA GISS global temperature data for 2000-2007 results in observations that are consistent with the IPCC predictions, but contradict the IPCC’s conclusion that the surface and satellite temperature records are reconciled. Using the RSS data results in observations that are (apparently) inconsistent with the IPCC predictions.
I am sure that in conducting such a verification some will indeed favor the dataset that best confirms their desired conclusions. But, it would be ironic indeed to see scientists now abandon RSS after championing it in the CCSP and IPCC reports. So, I’m not sure what to think.
Is it really the case that the surface and satellite records are again at odds? What dataset should be used to verify climate forecasts of the IPCC?
Answers welcomed.
January 8th, 2008 at 3:18 pm
Shouldn’t we use the best information available? Using ground based observations skews the spatial coverage toward (subjectively “removed”) UHI effects. Satellite is the best information we have, correct? Also, as Luboš Motl points out, 2007 is the coldest year this *century*. Why would you start at 2000? Try just this century, which started in 2001. What kind of trend do we get then? Interestingly, we get exactly what the solar influence people have been saying – a cooling Earth… Specifically, about 2.5 degrees per century…
http://bp3.blogger.com/_4ruQ7t4zrFA/R33-zmGiC4I/AAAAAAAAAPc/qHr4TPICyfM/s1600-h/rss-msu-anomaly.JPG
January 9th, 2008 at 3:21 am
A couple of thoughts come to mind:
1. When comparing 100 year forecasts in the first 5 years, it is really hard to show much divergence from forecast, even if the forecast is really bad. It might be more interesting to go back to the first or second assessment and compare their forecasts to reality
2. The convergence of surface temperature measurements with satellite should be a source of skepticism, not confidence. We know that the surface temperature measurement system is immensely flawed: there are still many station quality issues in the US like urban biases that go uncorrected, and the rest of the world is even worse. There are also huge coverage gaps (read: oceans). The fact this system correlates with satellite measurement feels like the situation where climate models, many of which take different approaches, some of them demonstrably wrong or contradictory, all correlate well with history. It makes us suspicious the correlation is a managed artifact, not a real outcome.
3. Satellite temperature measurement makes immensely more sense – it has full coverage (except for the poles) and is not subject to local biases. Can anyone name one single reason why the scientific community does not use the satellite temps as the standard EXCEPT that the “answer” (ie lower temperature increases) is not the one they want? Consider the parallel example of measurement of arctic ice area. My sense is that before satellites, we got some measurements of arctic ice extent from fixed observation stations and ship reports, but these were spotty and unreliable. Now satellites make this measurement consistent and complete. Would anyone argue to ignore the satellite data for spotty surface observations? No, but this is exactly what the entire climate community seems to do for temperature.
January 9th, 2008 at 3:23 am
A couple of thoughts come to mind:
1. When comparing 100 year forecasts in the first 5 years, it is really hard to show much divergence from forecast, even if the forecast is really bad. It might be more interesting to go back to the first or second assessment and compare their forecasts to reality
2. The convergence of surface temperature measurements with satellite should be a source of skepticism, not confidence. We know that the surface temperature measurement system is immensely flawed: there are still many station quality issues in the US like urban biases that go uncorrected, and the rest of the world is even worse. There are also huge coverage gaps (read: oceans). The fact this system correlates with satellite measurement feels like the situation where climate models, many of which take different approaches, some of them demonstrably wrong or contradictory, all correlate well with history. It makes us suspicious the correlation is a managed artifact, not a real outcome.
3. Satellite temperature measurement makes immensely more sense – it has full coverage (except for the poles) and is not subject to local biases. Can anyone name one single reason why the scientific community does not use the satellite temps as the standard EXCEPT that the “answer” (ie lower temperature increases) is not the one they want? Consider the parallel example of measurement of arctic ice area. My sense is that before satellites, we got some measurements of arctic ice extent from fixed observation stations and ship reports, but these were spotty and unreliable. Now satellites make this measurement consistent and complete. Would anyone argue to ignore the satellite data for spotty surface observations? No, but this is exactly what the entire climate community seems to do for temperature.