IPCC Predictions and Politics
May 22nd, 2008Posted by: Roger Pielke, Jr.
The May 1, 2008 issue of New Scientist magazine has an interesting article that parallels some of the discussions that we’ve had on this site lately. Here is an interesting excerpt:
“Politicians seems to think that the science is a done deal,” says Tim Palmer, “I don’t want to undermine the IPCC, but the forecasts, especially for regional climate change, are immensely uncertain”.
Palmer is a leading climate modeller at the European Centre for Medium-Range Weather Forecasts in Reading, UK, and he does not doubt that the Intergovernmental Panel on Climate Change (IPCC) has done a good job alerting the world to the problem of global climate change. But he and his fellow climate scientists are acutely aware that the IPCC’s predictions of how the global change will affect local climates are little more than guesswork. They fear that if the IPCC’s predictions turn out to be wrong, it will provoke a crisis in confidence that undermines the whole climate change debate.
The IPCC’s forecasts could be wrong in many different ways, over different time periods and spatial scales, including underestimating future changes. And it is not even clear that scientists involved with the IPCC have a collective view on what it would even mean for the IPCC to be “wrong”. As we’ve argued here often, action on climate change makes sense even if the predictions of the IPCC are not yet perfect. But this is a hard case to make when defenders of those predictions allow no room for imperfections to be seen, or questions to be asked.
May 22nd, 2008 at 3:35 am
Roger,
As someone who has done a LOT of modelling (and model based forecasting and scenario analysis etc.) I can guarrantee this statement by you will prove wrong:
“The IPCC’s forecasts could be wrong in many different ways, over different time periods and spatial scales, including underestimating future changes”
THe first rule of any forecasting is that your forecasts will be wrong. For forecasts of this nature we can expand from the general to conclude that the forecasts WILL be wrong in many different ways, including spatial and temporal scales etc. etc.
But as you rightly point out. It is the failure of the climate modelling fraternity to tell people this that leaves their credibility open to question
May 22nd, 2008 at 7:40 am
“But he and his fellow climate scientists are acutely aware that the IPCC’s predictions of how the global change will affect *local climates* are little more than guesswork. They fear that if the IPCC’s predictions turn out to be wrong, it will provoke a crisis in confidence that undermines the whole climate change debate.”
You know what would help immensely to reduce the possibility of this happening?
If people stopped conflating the extremely uncertain small scale (temporal and spatial) predictions with say the relatively more confident projections of the forced component of the climate over decades on GMST. That would be a nice start.
Unfortunately, it seems as though some people just can’t help themselves and will continue misrepresent the IPCC’s actual positions on things like short term temperature trends, Atlantic tropical cyclones, and uncertainty levels.
May 22nd, 2008 at 9:11 am
Jon-
Thanks, we seem to reaching a convergence here. You write:
“Unfortunately, it seems as though some people just can’t help themselves and will continue misrepresent the IPCC’s actual positions on things like short term temperature trends, Atlantic tropical cyclones, and uncertainty levels.”
And I will assert that one main reason for this is the repeated claim by climate scientists that short term phenomena are “consistent with” the model predictions. So long as scientists assert that short term observations are “consistent with” models, that particular phrasing will be interpreted to mean that the models have short-term predictive skill. It will not be interpreted to mean that (a) the models are not particularly useful on short time scales, or (b) the observations fall within an enormously wide distribution of possible outcomes.
So if it is wrong for people to look at the short term, that judgment must cut both ways, would you agree?
May 22nd, 2008 at 9:46 am
Of course it’s not news that the regional forecasts are worthless, just that someone has put it in print. I hope this is the start of a more open, honest discourse because, being positive, I believe we could make good regional models if we concentrated only on each regional climate, based on actual observations, rather than trying to model the entire world at once based on unproven theories. And such local models would be useful. Possibly one day the results from individual model patches might be sub-structured to give us a better global model. The cart has been put before the horse up to this point.
Nevertheless it is amazing how many impact studies are based on these models, despite their clear inadequacy. Most studies even pointedly ignore that the local climate stats contradict the models. I’ve been wondering for a long time if the authors of such papers are actually aware of the unsuitability of the models they use. Was is communicated to them and they ignored it, or were they completely misled? Don’t they care as long as they are paid?
Perhaps Jon can inform us all on what basis can he say that we can be confident of long term projections when all other projections are poor. Are there tests we don’t know about or is it just another idea that becomes true purely by repetition? Or is it perhaps based on a show of hands?
May 22nd, 2008 at 12:08 pm
“And I will assert that one main reason for this is the repeated claim by climate scientists that short term phenomena are “consistent with” the model predictions.”
This is a part of the “consistent with” issue that I haven’t addressed before, but I’d like to now. There are different instances of consistent with in the literature and media that are not all referring to a single subject- consistent with the IPCC ensemble modeling in the AR4, consistent with a specific model used for that particular study, or even consistent with anthropogenic warming itself rather than a model projection and/or consistent with an observed and presumably projected trend not based necessarily modeled.
Knutson 2008 describes future Atlantic cyclone trends not being not consistent with the the proposed attribution of increased cyclone frequency in the Atlantic basin to correlated Atlantic SST and extrapolations of this correlation. Are the authors actually saying that their study is inconsistent with a given model or the ensemble from the AR4? Granted, I’ve only skimmed the study, but that wasn’t what I remember- I thought they referenced two analyses of data (Mann and Holland, IIRC) rather than modeling studies. I acknowledge upfront that they may well have stated what specific model(s) results their findings were inconsistent with, but I didn’t see it at first glance.
You may feel that a lot of my issues with your posts are semantic in nature, but to me they are about correctly parsing and conveying nuance that ends up making a world of difference in the final result. Keenlyside 2008 being an example- their forecast run never predicts cooling, for example. So to claim that their model predicts two decades of cooling is untrue. Their model shows steady warming. It is their *opinion*, not the result of the model itself, that there could be a pause in temperature increase, because their model undercalls current temps. There needs to be proper discrimination between what a model says vs. what a scientist or group of scientists interpret the results to be, how that fits in to the existing body of knowledge, and whether or not it is met with agreement elsewhere, etc. at the *minimum*.
Above all, I think that all this emphasis on “consistent with” is overblown.
“So long as scientists assert that short term observations are “consistent with” models, that particular phrasing will be interpreted to mean that the models have short-term predictive skill.”
This is opinion on your part. I would assume (opinion as well, of course) that outside of science itself, the nearest experience the general (at least western Anglophone) population has with the phrase “consistent with” would be televised medical/procedural dramas, which do a fair job of illustrating that “consistent with” is not a particularly exact or overwhelmingly certain phrase, and indeed allows a similar broad range of interpretation corresponding with the amount of evidence available. “Consistent with blunt force trauma” allows for much more uncertainty than “a wound consistent with a blow from an object the size and shape of the fire iron in the defendant’s possession”. I doubt that the average person would assume that this means that there is literally no other object that could cause the wound, nor would I expect that the average person would assume that “consistent with” meant that there was no other possible explanation in regards to obs vs. modeling. Indeed, anyone unfortunate enough to have significant real world experience with an illness not immediately diagnosable will quickly appreciate both the value and limitations of the phrase.
“It will not be interpreted to mean that (a) the models are not particularly useful on short time scales, or (b) the observations fall within an enormously wide distribution of possible outcomes.”
To me it seems as though you are placing a much larger significance on what “consistent with” means than is really warranted and making a lot of assumptions about other people’s interpretations in the process. I by no means am saying that “consistent with” is meaningless in terms of policy formation, but rather that it should be considered for what it is worth and in addition to all of the other evidence available- it should be considered neither sufficient on its own nor irrelevant.
“So if it is wrong for people to look at the short term, that judgment must cut both ways, would you agree?”
It isn’t “wrong” for people to “look at the short term”. It’s just *wrong* to compare apples to oranges.
May 22nd, 2008 at 12:47 pm
Jon-
Thanks for your comments, a few reactions follow:
1. You got to some length to explain all of the things that the notion of “consistent with” are consistent with. I agree that there are many such things, so many in fact that I don’t thing the phrasing has much meaning, but it implies a meaning. but perhaps you can humor me by also explaining how the phrase “inconsistent with” should be used in practice?
2. I continue to be amazed about how there are different interpretations of the Keenlyside paper. You are looking only at the model and I am comparing the observations (final red point) with the modeled predictions. Cooling refers to the difference between the observations of the late 1990s (red curve) and what the model predicts to 2015 (green). This seems obvious. Of course, the fact that people cannot agree on what this single study shows, is indicative of the lack of precision in this entire area. When people disagree about such things the name-calling starts, sadly . . .
3. “This is opinion on your part.”
Of course! This is a weblog for commentary and discussion
Thanks again.
May 22nd, 2008 at 3:43 pm
“If people stopped conflating the extremely uncertain small scale (temporal and spatial) predictions with say the relatively more confident projections of the forced component of the climate over decades on GMST. That would be a nice start.”
From the IPCC AR4, what is the IPCC’s confidence that, in the absence of government intervention, the GMST will warm by more than 5 degrees Celsius from 1990 to 2100?
What is their confidence the GMST will warm by more than 4 degrees Celsius from 1990 to 2100?
More than 3 degrees Celsius?
More than 2 degree Celsius?
More than 1 degree Celsius?
May 22nd, 2008 at 7:56 pm
“You got to some length to explain all of the things that the notion of “consistent with” are consistent with.”
All I did was essentially state that “consistent with” can apply to many different direct objects when used in a sentence in the English language and that paying attention to what that phrase is actually referring to has an impact on its meaning.
“I agree that there are many such things, so many in fact that I don’t thing the phrasing has much meaning, but it implies a meaning.”
It has a meaning, but the meaning is necessarily related to its context. Again, use medical scenarios as an analog- you can have symptoms consistent with an ailment- they can be quite vague, or quite specific. The range of ailments they can be consistent with could be large or small. That doesn’t mean that “consistent with” has no meaning, it means that context and other evidence determine the amount of meaning it has.
“but perhaps you can humor me by also explaining how the phrase “inconsistent with” should be used in practice?”
I think that Knutson et al. used it just fine from what I read. The only “problem” I guess would be that you would prefer that they had used it in reference to a model or group of models? It of course entirely depends on whether one is speaking colloquially or attempting to invoke statistical significance by using the phrase. The word “consistent” in this sense is logically ambiguous, in addition to being vague. Hence the need for context.
“I continue to be amazed about how there are different interpretations of the Keenlyside paper.”
Why? I expected pretty much exactly the result when I read it- conflation of the modeled result with their forecast which was based only in part on the result of the model. If someone says their model run shows cooling, they are simply wrong. If they say that the authors interpretation of that run predicts cooling, fine. It was pretty obvious to me at least what was going to happen.
“This seems obvious.”
It is. No argument about that.
“Of course, the fact that people cannot agree on what this single study shows”
I think we both agree that their model run does not itself show cooling, but rather this is the authors’ opinion given observations and their confidence in their initialization’s hindcasting. Which some, myself included, are dubious of.
We don’t disagree with what the study is saying, we disagree on the simplicity with which we are comfortable characterizing what it says. I perhaps am even more picky than others, but I have no trouble explaining to other people why this study doesn’t contradict anything in the AR4 and why I wouldn’t consider it to be a necessarily likely forecast.
When we forgo nuance and context, we risk misunderstanding and confusion.
May 22nd, 2008 at 8:00 pm
Jon-
You’ve been a good sport, perhaps I might trouble your further by asking for an answer to this question:
http://sciencepolicy.colorado.edu/prometheus/archives/climate_change/001440homework_assignment.html
Mark, James, and other climate professionals are also encouraged to reply.
May 23rd, 2008 at 3:52 am
Ah Jon moves smoothly from “not inconsistent with” to “doesn’t contradict”. He’ll be saying “not implausible” next.
I note that with Keenlyside 2008 Jon is happy to regard the short term model results as more important than the author’s opinion yet with the IPCC ensemble he places author opinion above the short-term model results. This is the mental trick known as “cognitive dissonance”. Why not go all the way down the logic trail and admit that the model results and the author opinion are both obviously unreliable.
May 23rd, 2008 at 3:00 pm
For what it’s worth, the the 20 year trend min/max average for the last set of 4 are
0
-.025
.065
.39
And the last 10 is
.50
Although as a value, the anomaly in 1998 is the same as in 2007 with .57
What is the margin of error to be expected, and the range of the base period of 14 centigrade as an estimate? Even the average of the last 10 years is only 3.5 percent of the base, what does that tell us?
May 23rd, 2008 at 3:12 pm
I am not a climate scientist, modeller or statistics person. When I see ‘consistent with’ it is an optimistic phrase for me. I take it to mean ‘matches in a meaningful way’ or ‘is correct’ or ‘the most likely match’. I wouldn’t think it means it is some vague probability or low chance possibility that it’s true.
The proper way to say it is ‘not inconsistent’ to imply it is a possibility. Would it not be even better to give the odds on that possibility?
From a policy standpoint consistent means something different.