Collateral Damage from the Death of Stationarity
June 10th, 2009Posted by: Roger Pielke, Jr.
In recent years climate scientists have come to understand that the climate system may not be stationary – meaning that the fundamental statistics of climate vary and change over timescales of relevance to people. For those who consider that the phrase “climate change” is redundant, this will be no surprise. However, decision makers in a wide range of settings, including flood mitigation, reinsurance and insurance, and even aspects of carbon policy, operate from a framework where climate is perceived to be a stationary process.
In a new essay in the GEWEX Newsletter I argue that if indeed stationarity is dead then collateral damage of the new philosophy of climate necessarily must be the notion that we can ever evaluate the skill of climate predictions using empirical methods. That leaves us relying on a few remaining methods of forecast evaluation, among them political expediency and simple faith.
Here is an excerpt from my essay:
Here I suggest a far more consequential implication of the death of stationarity for the role of science in water management decision making than a need for better models and observations. Rather than basing decision-making on a predict (probabilistically of course) then act model, we may have to face up to the fact that skillful prediction of variables of interest to decision makers may simply not be possible. And even if it were possible, we would not be able to identify skill on the same time scales as decisions need to be made. The consequence of this line of argument is that if stationarity is indeed dead, then it has likely taken along with it fanciful notions of foreseeing the future as the basis for optimal actions. Instead, it may be time to rethink how we make decisions in the face of not simply uncertainty, but fundamental and irreducible ignorance. Rather than focus on optimal decisions guided by prediction, we may need instead to focus on robust decisions guided by recognition of the limits of what can be known.
You can read the entire essay, which includes an excursion into how the “guaranteed win scam” conspires with the “hot hand fallacy” to defeat efforts to judge predictive skill in the context of nonstationarity, at the link below.
Pielke, Jr., R.A., 2009. Collateral Damage from the Death of Stationarity, GEWEX Newsletter, May, pp. 5-7. (PDF)
June 10th, 2009 at 8:47 am
“fundamental and irreducible ignorance” strikes me as an incredibly strong statement. I agree that the death of stationarity is with us, and I think it was coming whether or not change was seen as anthropogenic or not.
But to state that ignorance is irreducible strikes me more as a political than scientific claim. Further, the issue is not black and white. We have some knowledge, but as your recent post on the Danish PM mentions, understanding the level of that knowledge and its reliability is critical. To suggest that our ignorance is both fundamental and irreducible is just a deadend. Maybe we should start looking for an oracle.
On a very related note, I got an email from the National Academies Press about a book called “Informing Decisions in a Changing Climate “. It’s not in the cards for me to purchase this book, but I’m wondering if you’re familiar with it (maybe involved with it) and what your take is.
June 10th, 2009 at 9:22 am
-1-dean
Thanks. I mean “ignorance” in a Keynesian sort of manner (ie., as I use the term in The Honest Broker). Keynes argued that we can’t know what the price of copper is 20 years from now and that lies in the area of “ignorance” because we can’t even quantify the uncertainties. Similarly, knowing how many hurricanes will hit the US 20 years from now, or precipitation in Colorado similarly lies in the area of such “ignorance”. For reasons I describe in the article that ignorance is fundamental and irreducible, though we can easily convince ourselves otherwise (and often do).
The NRC report you refer to actually draws upon some of our work in our NSF SPARC project on climate science policy. It outlines a paradigm for climate research far different that that currently in play, and one quite consistent with that which you’ll find in my research going back to the early 1990s. It can be read free online at the NAP website.
June 10th, 2009 at 9:35 am
Of course, the need for what you call stationarity was what drove the hockey stick study. The Medieval Warming period had to be eliminated in order to create the illusion of stationarity. As Mann’s work has been thoroughly debunked, the MWP is left standing as an example of the kind of black swan that stationary assumptions cannot account for. It also recommends to us that the current warming period may be the same kind of exception that proves (tests) the rule of stationarity.
Regarding your prescription in the second half of the article, I fear it would lead to precautionary principle nuttiness. If the same people use the same tools, I can only assume that the range of possibilities they would present would be the same as their current skill-less preditions. I can’t see Jim Hansen changing his tune, regardless of the instrument you give him to play.
June 10th, 2009 at 9:55 am
Roger – Unless I’m misunderstanding, my previous comment stands. For example, projections of a drier southwestern US are not based on particularly local conditions. They are based on what we currently know about moisture transport, for example the desert zones around 25 degrees Lat, and how we expect that to adjust with a changing climate. And these projections also include knowledge of long-term droughts in the Southwest in the past that go with warm periods. It strikes as extremely fatalistic to believe that none of this will lead to any ability to make projections with some level of confidence.
And it undermines adaptation much more than mitigation, since we can only attempt to plan for adaptation if we have some idea of what we need to adapt to. Mitigation doesn’t really require much knowledge on specific impacts, just that there will be enough to justify mitigation.
June 10th, 2009 at 10:09 am
-4-dean
Your confidence is not based on demonstrated skill, it is based on something more like faith — faith in science, in models, etc.. We can have plenty of confidence in faith-based predictions. My article is about the prospects for demonstrations of skill in forecasts of nonstationary processes, which is a term of art meaning proven experience. However, when there are competing predictions of the future, we then have no empirical basis on which to choose among them, hence the need for robust decision making.
As the idea that we have to know the future in order to adapt is not an assumption universally shared. See:
Dessai, S., M. Hulme, R. Lempert, and R. Pielke, Jr. 2009. Do We Need Better Predictions to Adapt to a Changing Climate? Eos, Vol 90, No. 13, pp. 111-112.
http://sciencepolicy.colorado.edu/admin/publication_files/resource-2720-2009.08.pdf
Thanks
June 10th, 2009 at 11:06 am
Yes, my confidence that we will be able to make projections with some degree of confidence _in anything specific_ is not based on evidence that we will be able to do so, since we won’t know that until we have done so.
However for you to say that our ignorance is irreducible is certainly no more scientific. It _predicts_ that we won’t gain enough accumulated knowledge in the future to make viable prejections, when I would assert that it is quite feasible that we will gain adequate knowledge to do so.
I also don’t see how you can defend this statement: “However, when there are competing predictions of the future, we then have no empirical basis on which to choose among them.” It may be that at a given time, we don’t have the knowledge to choose among them, but you also seem to be saying that we will never have such ability when you use words like irreducible.
Taken as a whole, you seem to be saying that we can never know what the future will hold, so why bother trying. Is that correct?
June 10th, 2009 at 11:26 am
PS – I would add that knowledge and study of past climates can provide empirical evidence with which to evaluate models. Such evidence is a significant part of existing climate models.
June 10th, 2009 at 3:17 pm
-6, 7-dean
I never used the phrase “viable predictions” — my article is about “skillful predictions”.
You write: “It may be that at a given time, we don’t have the knowledge to choose among them, but you also seem to be saying that we will never have such ability when you use words like irreducible.”
Often, on time scales of decisions, uncertainties and ignorance are indeed irreducible.
You can see my views on prediction more generally here:
Pielke Jr., R. A., D. Sarewitz and R. Byerly Jr., 2000: Decision Making and the Future of Nature: Understanding and Using Predictions. Chapter 18 in Sarewitz, D., R. A. Pielke Jr., and R. Byerly Jr., (eds.), Prediction: Science Decision Making and the Future of Nature. Island press: Washington, DC.
http://sciencepolicy.colorado.edu/admin/publication_files/resource-73-2000.06.pdf
June 11th, 2009 at 9:09 pm
[...] Prometheus posted a excellent, much needed weblog titled Collateral Damage from the Death of Stationarity which referred to my son’s [...]
June 12th, 2009 at 8:00 pm
I just spent a couple of days at a conference where the researchers were maintaining that they need to provide downscaled climate models, while the users wanted a larger variety of tools to deal with adapting to climate change
1) a focus on observations through monitoring and flexibility and quickness to deal with problems as they arise in nature,
2) assessments of vulnerability given less specific projections
The problem, in this case, was that too many people know we don’t know some fundamentals – how plants and their offspring adapt to differences, their interactions with pests, microclimates that plants perceive compared to macroclimates that are modelled. So the key things are to stay agile in our responses, and to manage as much as possible so that no matter what scenario is predicted we will still do OK. This is either hedging your bets or designing resilient systems or whatever.
In other words you don’t need to know to manage.. we don’t depend on downscaled economic models and make financial decisions every day. And a corollary is that you can make worse decision from pretending you know (from models) than to admit you don’t.
I sit in meetings with biologists or climate scientists who think that due to climate change, decades of decision science need to be thrown out .. but no one has explained to me why they feel that these sciences are no longer relevant.
June 16th, 2009 at 12:11 pm
Interestingly, the tail-end of the essay that preceded Roger’s cited a 1951 article by Hurst. Now I understand that water management was simply an example of irreducible uncertainty, but it’s my understanding that the methods pioneered by Hurst in sizing dams & resevoirs
June 16th, 2009 at 12:25 pm
Apologies – must have somehow hit the submit button –
Interestingly, the tail-end of the essay that preceded Roger’s in the GEWEX Newsletter cited a 1951 article by Hurst. Now I understand that water management was simply one example of “irreducible uncertainty,” but it’s my understanding that the methods pioneered by Hurst for sizing dams & resevoirs are implicitly based on uncertainty (or non-stationarity), and that hydrologists have been using them successfully ever since.
Demetris Koutsoyannis has certainly stirred the pot lately by suggesting that the statistical methods routinely used by hydrologists could also be applied to the climate meta-system, perhaps more successfully than those ubiquitous GCMs.
I also believe that Mandelbrot honored Hurst as a sort of proto-fractalist/chaoticist with his term “The Hurst Exponent.”
I’ve been wanting to dig deeper into this topic but haven’t had time yet. If Roger or anyone else can point me to some good primers, I’d be much appreciative.
June 18th, 2009 at 6:17 am
I suggest that you make much more of the substantial work that Demetris Koutsoyiannis has completed on this general problem – non-linearity, non-stationarity and prediction. You can find all of his papers on his university website here, http://www.itia.ntua.gr/dk . Any discussion of this complex category cannot get far unless the discussion takes Demetris’ contribution into account. He’s written extensively about Hurst, etc. But see his pioneering work under the heading climate stochastics on his website.
Richard