Executive Summary Prediction In Policy: A Process, Not A Product By Daniel Sarewitz and Radford Byerly, Jr. The Earth sciences, backed by formidable arrays of data gathering and processing technologies, now offers the apparently credible promise of predicting the future of nature; policy, under pressure as always to deliver public benefit at low cost and even lower risk, has strong incentives to accept this promise as one response to environmental issues. We are investigating the role of prediction in the making of environmental policies. Such policies relate to problems that include: planning for and responding to natural hazards (weather, floods, earthquakes, asteroids); planning for and responding to anthropogenic hazards (global climate change, acid rain, nuclear waste); managing natural resources (oil reserves, beaches); and regulating environmental impacts (mining). On September 10-12, 1998, we (along with our co-investigators, Roger Pielke, Jr., and Dale Jamieson) convened a workshop in Estes Park, Colorado, that brought together a diverse group of people who are involved in various ways with the process of prediction. Among the 35 participants were: a scientist who works on climate models; the former emergency manager of a major California city; a banker from a coastal city that is subject to hurricanes; a seismologist; a rancher; a former official at the federal Office of Management and Budget; an engineer who works on nuclear waste isolation; a coastal geologist who studies beach erosion. The goal of the workshop was to apply the collective wisdom of a range of stakeholders (including natural scientists who make predictions, and social scientists concerned with their use) to the problem of how scientific predictions should be used (or not used) in the development of effective policies relating to natural hazards, natural resources, and the environment. Consider two situations: I.  People listen to or read the morning weather report, and then make a decision about clothing, accessories (umbrella? gloves? hat?), mode of transport. This decision is backed by a personal experience of weather and its local fluctuations, and a scientific and technical support infrastructure that in the U.S. issues on the order of 10 million weather predictions per year. These predictions, based in part on real-time observations of weather patterns, are aimed at supporting the specific information that people need. The consequences of a poor prediction, or a poor decision based on a good prediction, are often modest-a wet shirt, perhaps a car skidding off the road-although on rare occasions severe-an airplane crash, the failure to evacuate a town. In either case, users have accumulated enough experience in comparing the prediction to the actual event, to develop a comfort with-to personally "calibrate"-the weather prediction process. II.  In other situations, such personal experience is not possible. Members of Congress listen to testimony from scientists about nuclear waste disposal. Because radioactive waste remains dangerous for hundreds of thousands of years, disposal systems must operate effectively for at least that long. The relevant science uses analogy, mathematical models, and extrapolation to predict events far in the future. Thus, there is no basis in personal experience for evaluating or calibrating the actual performance of the disposal systems or the science. Decisions must be based on abstractions. Action must be taken, but the consequences of a poor prediction, or a poor decision based on a good prediction, are potentially disastrous, both politically (a lost election) and societally (radionuclides leaking into groundwater or even reaching the atmosphere.) Decision making is forward looking, so the allure of prediction is strong. We look to predictions to help us make decisions that can mitigate or evade the impact of nature on society, and of society on nature. In doing so, we need to recognize that prediction has become part of a complex decision-making process, a network of interrelationships that must function well across all of its connections if predictions are to successfully serve society. This integrated process involves policy makers (who solicit and pay for predictions), scientists (who make predictions) and decision makers (who use them--for everything from deciding whether to carry an umbrella to evacuating a city in the path of a hurricane; for establishing levels of insurance risk to negotiating an international environmental agreement). The less frequent, less observable to the human eye, less spatially discrete, more gradual, more distant in the future, and more severe a predicted phenomenon, the more difficult it is to accumulate direct experience. Where direct experience is sparse or lacking, other sources of societal understanding must be developed, or the prediction process will not function effectively. Science alone does not create this understanding. What is necessary above all is an institutional structure that allows policy makers, decision makers, and scientists to interact closely throughout the entire prediction process, so that each knows the needs and capabilities of the others. It is crucial that this process be open, participatory, conducive to mutual respect. Efforts to shield expert research and decision making from public scrutiny and accountability invariably backfire and fuel distrust and counterproductive policies and decisions. How can the prediction process foster sound decision-making? Predictions must be generated primarily with the needs of the user in mind. Television weather predictions focus primarily on temperature, precipitation, and wind, rather than thermal gradients, behavior of aerosols, and barometric pressure. For scientists to participate usefully in the prediction process, they must address the goals of the process, not the goals of science; they must listen to stakeholders. For stakeholders to participate usefully in this process, they must work closely and persistently with the scientists to communicate their needs and problems. The prediction process must be open. To create openness, stakeholders must question predictions. For this questioning to be effective, predictions should be as transparent as possible to the user. In particular, assumptions, model limitations, and weaknesses in input data should be forthrightly discussed. Especially in cases where personal experience may be limited (acid rain, asteroid impacts, global warming), public confidence in the validity of the prediction will derive in part from an understanding of how the prediction is generated. Black boxes generate distrust, especially when a prediction can stimulate decisions that create winners and losers. Even so, many types of predictions will never be understood by decision makers in the way that weather predictions are understood. Experience is important and cannot be replaced, but the prediction process can be facilitated in other ways, for example, by being totally open about predictions, warts and all; and by fully considering alternative approaches to prediction, such as "no regrets" policies, adaptation, and better planning and engineering. Uncertainties must be clearly articulated (and understood) by the scientists, so that users understand their implications. Failure to understand uncertainties has contributed to poor decisions that then undermine relations among scientists and decision makers; we saw this during the Red River flood in Grand Forks, ND. But understanding the uncertainties does not mean that the predictions will be useful. If policy makers truly understood the uncertainties associated with predictions of global climate change or nuclear waste behavior, they might decide that strategies for action should not depend on predictions. Alternatives to prediction must be evaluated as a part of the prediction process. Rather than trying to predict the impacts of hard-rock pit mines on water quality as a basis for environmental regulation, it might be more feasible to spread risk through bonding or other types of insurance. Predicting the consequences of global climate change has caused policy gridlock; other approaches to mitigation and adaptation should be more vigorously sought. Predictions themselves must be viewed as events. The prediction process must include mechanisms for the various stakeholders to fully consider and plan what to do after a prediction is made. When the prediction process is fostered by effective, participatory institutions, and when a healthy decision environment emerges from these institutions, then the products of predictive science may even become less important. Earthquake prediction was once a policy priority; now it is considered technically unfeasible, at least in the near future. But, in California, the close-institutionalized-communication among scientists, engineers, state and local officials, and the private sector, has led to considerable advances in earthquake preparedness and a much decreased dependence on prediction. On the other hand, in the absence of an integrated and open decision environment, the scientific merit of predictions can be rendered politically irrelevant, as has been seen with nuclear waste disposal and acid rain. That is, if there is no adequate decision environment for dealing with an event or situation, a scientifically successful prediction may be no more useful than an unsuccessful one. These observations fly in the face of much current practice, where, typically, policy makers recognize a problem, scientists then do research to predict natural behavior associated with the problem, and predictions are finally delivered to decision makers in the expectation that they will be both useful and well-used. This sequence, which puts predictive research at the core of the decision environment, rarely functions well in practice. In contrast, our work suggests that, for virtually every environmental problem, the key to effective decision making lies in improving the decision environment itself. Such improvement may come from cost-effective, politically realistic alternatives to prediction. The goal of the decision environment must be good decisions, not good predictions.