Comments on: Forecast Verification for Climate Science, Part 3 http://cstpr.colorado.edu/prometheus/?p=4307 Wed, 29 Jul 2009 22:36:51 -0600 http://wordpress.org/?v=2.9.1 hourly 1 By: hank http://cstpr.colorado.edu/prometheus/?p=4307&cpage=1#comment-9323 hank Mon, 14 Jan 2008 13:54:56 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=4307#comment-9323 Zinfan, did you ever get a response? Zinfan, did you ever get a response?

]]>
By: zinfan http://cstpr.colorado.edu/prometheus/?p=4307&cpage=1#comment-9322 zinfan Sat, 12 Jan 2008 12:45:40 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=4307#comment-9322 The confidence interval on the forecast looks wrong. I am not a climate specialist, but anyone with a knowledge of statistics would suspect something is amiss here. There is too much data scatter compared to the confidence interval. I did a bit of research, and found the variance in the annual average global temperature anomaly is around 0.1 degree C (1975 to the present). If that is the case, the minimum 95% confidence interval for evaluating a forecast must be at least +/- 0.2 degree. And that is just to account for the expected annual anomaly variation. It should probably be even larger, since each measurement system has an additional measurement error associated with it. The chart should have drawn error bands at least 0.2 degrees above and below the forecast. The confidence interval on the forecast looks wrong.

I am not a climate specialist, but anyone with a knowledge of statistics would suspect something is amiss here. There is too much data scatter compared to the confidence interval.

I did a bit of research, and found the variance in the annual average global temperature anomaly is around 0.1 degree C (1975 to the present). If that is the case, the minimum 95% confidence interval for evaluating a forecast must be at least +/- 0.2 degree. And that is just to account for the expected annual anomaly variation. It should probably be even larger, since each measurement system has an additional measurement error associated with it.

The chart should have drawn error bands at least 0.2 degrees above and below the forecast.

]]>
By: Harry Haymuss http://cstpr.colorado.edu/prometheus/?p=4307&cpage=1#comment-9321 Harry Haymuss Sat, 12 Jan 2008 03:20:48 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=4307#comment-9321 Looking at the UAH interpretation of the satellite data shows that the southern hemisphere - that would be the one not affected (maybe we should say afflicted) by the vast majority of black carbon and land use changes - has not warmed this century. Looking at the UAH interpretation of the satellite data shows that the southern hemisphere – that would be the one not affected (maybe we should say afflicted) by the vast majority of black carbon and land use changes – has not warmed this century.

]]>
By: lucia http://cstpr.colorado.edu/prometheus/?p=4307&cpage=1#comment-9320 lucia Thu, 10 Jan 2008 16:38:08 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=4307#comment-9320 There seem to be two separate issues here: 1) How one would, hypothetically test a theory, like AGW or the Enhanced Green House Effect (EGHE) and ' 2) Whether or not the current data are sufficient, and / or accurate. In my opinion: Of course you compare the theoretical predictions to data. A theory that never obtains sufficient empirical support remains unproven. A theory that is actually contradicted by lots of data is falsified. So, of course, if we collect lots of good data, and the theory of AGW doesn't match, we through it out. As for the recent satelitte data-- it appear there has been a problem? If so, it will be sorted out. But all this means is we may need to be a bit suspicious of very recent data. But it strikes me huge numbers of people is on both sides of this debate watch the data like a horse race. Doesn't one AGW believing blogger show a constantly updating plot showing the extent of ice coverage? Don't hoards of people watch the hurricane count breathlessly all summer? The main problem with making conclusions based on 7 years is beta error. Given the variability of weather, it's not enough data to falsify. Getting enough data to prove anything to a 5% confidence level in 7 years requires luck. Strangely, *proving* things false actually takes longer.(Normally this is not a problem because the scientific convention is supposed to be to treat a theory, product or as unproven until it has a lot of empirical support.) There seem to be two separate issues here:

1) How one would, hypothetically test a theory, like AGW or the Enhanced Green House Effect (EGHE) and ‘
2) Whether or not the current data are sufficient, and / or accurate.

In my opinion: Of course you compare the theoretical predictions to data. A theory that never obtains sufficient empirical support remains unproven. A theory that is actually contradicted by lots of data is falsified.

So, of course, if we collect lots of good data, and the theory of AGW doesn’t match, we through it out.

As for the recent satelitte data– it appear there has been a problem? If so, it will be sorted out.

But all this means is we may need to be a bit suspicious of very recent data. But it strikes me huge numbers of people is on both sides of this debate watch the data like a horse race. Doesn’t one AGW believing blogger show a constantly updating plot showing the extent of ice coverage? Don’t hoards of people watch the hurricane count breathlessly all summer?

The main problem with making conclusions based on 7 years is beta error. Given the variability of weather, it’s not enough data to falsify. Getting enough data to prove anything to a 5% confidence level in 7 years requires luck.

Strangely, *proving* things false actually takes longer.(Normally this is not a problem because the scientific convention is supposed to be to treat a theory, product or as unproven until it has a lot of empirical support.)

]]>
By: Roger Pielke, Jr. http://cstpr.colorado.edu/prometheus/?p=4307&cpage=1#comment-9319 Roger Pielke, Jr. Thu, 10 Jan 2008 07:59:53 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=4307#comment-9319 John Tierney kindly follows up this discussion on his NYTimes blog. http://tierneylab.blogs.nytimes.com/2008/01/10/a-spot-check-of-global-warming/ In particular, he invites the RealClimate folks to suggest predictions and data that might be used for those interested in verifying IPCC forecasts. I hope that they respond. John Tierney kindly follows up this discussion on his NYTimes blog.

http://tierneylab.blogs.nytimes.com/2008/01/10/a-spot-check-of-global-warming/

In particular, he invites the RealClimate folks to suggest predictions and data that might be used for those interested in verifying IPCC forecasts. I hope that they respond.

]]>
By: Roger Pielke, Jr. http://cstpr.colorado.edu/prometheus/?p=4307&cpage=1#comment-9318 Roger Pielke, Jr. Thu, 10 Jan 2008 07:55:25 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=4307#comment-9318 Climate Scientist James Annan, emails me to point to a comment in his own blog saying that this exercise is somehow "wrong" but he doesn't explain why it is wrong. You can also see in his comments some of the haughtiness that is all-too-characteristic of many climate scientists. http://julesandjames.blogspot.com/ Climate Scientist James Annan, emails me to point to a comment in his own blog saying that this exercise is somehow “wrong” but he doesn’t explain why it is wrong. You can also see in his comments some of the haughtiness that is all-too-characteristic of many climate scientists.

http://julesandjames.blogspot.com/

]]>
By: Roger Pielke, Jr. http://cstpr.colorado.edu/prometheus/?p=4307&cpage=1#comment-9317 Roger Pielke, Jr. Thu, 10 Jan 2008 07:04:21 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=4307#comment-9317 This from Jos de Laat, a climate scientist in the Netherlands, "With regard to your post on satellite temperature observations, I would like to draw your attention to this post on worldclimatereport.com. They noticed the same issue with the cooling in the RSS temperatures, but it contains, a.o., an update (9 January) on this issue with some quotes from John Christy. According to him, the RSS data contains some spurious cooling during the last several of months, so UAH has the better series during that period. Well, read for yourself: http://www.worldclimatereport.com/index.php/2008/01/08/musings-on-satellite-temperatures/" This from Jos de Laat, a climate scientist in the Netherlands,

“With regard to your post on satellite temperature observations, I would like to draw your attention to this post on worldclimatereport.com. They noticed the same issue with the cooling in the RSS temperatures, but it contains, a.o., an update (9 January) on this issue with some quotes from John Christy. According to him, the RSS data contains some spurious cooling during the last several of months, so UAH has the better series during that period. Well, read for yourself:

http://www.worldclimatereport.com/index.php/2008/01/08/musings-on-satellite-temperatures/

]]>
By: Mark Bahner http://cstpr.colorado.edu/prometheus/?p=4307&cpage=1#comment-9316 Mark Bahner Wed, 09 Jan 2008 19:35:41 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=4307#comment-9316 Hi Roger, You write, "So here is a plea to the climate community: when you say that you are predicting something like global temperature or sea ice extent or hurricanes -- tell us is specific detail what those variables are, who is measuring them, and where to look in the future to verify the predictions. If weather forecasters, stock brokers, and gamblers can do it, then you can too." Again, you seem to be working under the assumption that the climate change community is interested in doing science. The evidence shows very clearly that they're not. Here's how a real scientist would approach the problem of predicting future global average temperature: 1) Develop probability distributions for all climate forcing factors, e.g. CO2, CH4, N2O, black carbon, sulfur dioxide, organic carbon, etc. In other words, don't develop a bunch of ridiculously bogus "scenarios" that have absolutely no assessment of the likelihood of occurrence. Instead, take one variable at a time, and construct a probability distribution for that variable in the coming century. For example, the current atmospheric methane concentration is about 2780 ppb: Atmospheric methane What are the 50 percent probability estimate values for 2010, 2020, 2030, 2040, etc. Also, what are the 10 percent and 90 percent probability estimates (that is, less than a 10 percent chance of being below that value, and a 90 percent chance of being below that value) for each of those years. 2) Develop probability distributions for the forcing associated with those estimated values. For example, methane is estimated to have a global warming potential of 62 over 20 years, and 23 over 100 years. But those are (presumably) "50 percent probability" values. Probability distributions would need to be assessed. 3) Combine the probability distributions for all the variables into a probability distribution for total climate forcing. 4) Combine that probability distribution for climate forcing with a probability distribution for climate sensitivity, to produce... 5) ...temperature probability distribution for the coming century. I think the problem with using this real-science approach is that most scientists would end up agreeing that the "50 percent probability" value for warming in the 21st century would be less than 2.0 deg C. That simply would not scare enough people. Hi Roger,

You write, “So here is a plea to the climate community: when you say that you are predicting something like global temperature or sea ice extent or hurricanes — tell us is specific detail what those variables are, who is measuring them, and where to look in the future to verify the predictions. If weather forecasters, stock brokers, and gamblers can do it, then you can too.”

Again, you seem to be working under the assumption that the climate change community is interested in doing science. The evidence shows very clearly that they’re not.

Here’s how a real scientist would approach the problem of predicting future global average temperature:

1) Develop probability distributions for all climate forcing factors, e.g. CO2, CH4, N2O, black carbon, sulfur dioxide, organic carbon, etc.

In other words, don’t develop a bunch of ridiculously bogus “scenarios” that have absolutely no assessment of the likelihood of occurrence.

Instead, take one variable at a time, and construct a probability distribution for that variable in the coming century. For example, the current atmospheric methane concentration is about 2780 ppb:

Atmospheric methane

What are the 50 percent probability estimate values for 2010, 2020, 2030, 2040, etc. Also, what are the 10 percent and 90 percent probability estimates (that is, less than a 10 percent chance of being below that value, and a 90 percent chance of being below that value) for each of those years.

2) Develop probability distributions for the forcing associated with those estimated values. For example, methane is estimated to have a global warming potential of 62 over 20 years, and 23 over 100 years. But those are (presumably) “50 percent probability” values. Probability distributions would need to be assessed.

3) Combine the probability distributions for all the variables into a probability distribution for total climate forcing.

4) Combine that probability distribution for climate forcing with a probability distribution for climate sensitivity, to produce…

5) …temperature probability distribution for the coming century.

I think the problem with using this real-science approach is that most scientists would end up agreeing that the “50 percent probability” value for warming in the 21st century would be less than 2.0 deg C. That simply would not scare enough people.

]]>
By: Roger Pielke, Jr. http://cstpr.colorado.edu/prometheus/?p=4307&cpage=1#comment-9315 Roger Pielke, Jr. Wed, 09 Jan 2008 13:07:35 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=4307#comment-9315 Ross McKitrick writes in: "- In my view the key metric has to be the mean temperature of the tropical troposphere. Fig 10.7 of the AR4: http://ipcc-wg1.ucar.edu/wg1/Report/suppl/Ch10/Ch10_indiv-maps.html is remarkably consistent across models that the warming will be focused in that region, and that it is expected to be differentially strong compared to other regions and the surface, right from the get-go. Also, the hindcast (IPCC Fig 9,1) shows that no other forcing is expected to produce the uniquely focused warming there. The other strong warming region is the Arctic surface, but it is geographically small and subject to many other influences. - In response to my T3 Tax essays (http://ross.mckitrick.googlepages.com/#t3tax), some commenters have said that stratospheric cooling is really the appropriate metric. This measure might be confounded with reduced solar output (though apparently the effects are not identical, see the paper on this by Bengtsson http://www.nerc-essc.ac.uk/~olb/PAPERS/len24.pdf). In any case I find it very curious that RSS shows no cooling, and possibly a slight (insignificant) warming trend in the stratosphere over the past 15 years, while CO2 has gone up by at least 25 ppm. And prior to that there was no "trend" per se, just 2 flat intervals interrupted by step-like changes associated with big volcanoes. http://www.remss.com/data/msu/graphics/plots/sc_Rss_compare_TS_channel_tls.png" Ross McKitrick writes in:

“- In my view the key metric has to be the mean temperature of the tropical troposphere. Fig 10.7 of the AR4:

http://ipcc-wg1.ucar.edu/wg1/Report/suppl/Ch10/Ch10_indiv-maps.html is remarkably consistent across models that the warming will be focused in that region, and that it is expected to be differentially strong compared to
other regions and the surface, right from the get-go. Also, the hindcast (IPCC Fig 9,1) shows that no other forcing is expected to produce the uniquely focused warming there. The other strong warming region is the Arctic surface, but it is geographically small and subject to many other influences.

- In response to my T3 Tax essays (http://ross.mckitrick.googlepages.com/#t3tax), some commenters have said that stratospheric cooling is really the appropriate metric. This measure might be confounded with reduced solar output (though apparently the effects are not identical, see the paper on this by Bengtsson http://www.nerc-essc.ac.uk/~olb/PAPERS/len24.pdf). In any case I find it very curious that RSS shows no cooling, and possibly a slight (insignificant) warming trend in the stratosphere over the past 15 years, while CO2 has gone up by at least 25 ppm. And prior to that there was no “trend” per se, just 2 flat intervals interrupted by step-like changes associated with big volcanoes.
http://www.remss.com/data/msu/graphics/plots/sc_Rss_compare_TS_channel_tls.png

]]>