Uncertainty and Decision Making
December 16th, 2004Posted by: Roger Pielke, Jr.
In many areas of decision making a claim is often made that “reducing uncertainties” are a prerequisite for decision making to occur. Hence, on an issue like climate change we see tens of billions of dollars invested into research justified primarily by a goal of reducing uncertainties to foster decision making. We also see a very public debate, enjoined by scientists, to try to scrub the world clean of any of the last vestiges of lingering uncertainty among the populace (for example, see here). And even the IPCC chairman sees its mission as being “reducing uncertainties.”
All of the focus and allocation of resources to reducing uncertainties raises what I would think ought to be several basic questions:
Is there evidence that reduction of uncertainty actually compels policy action?
Does funding of research actually lead to reduced uncertainty?
Is action impossible is the face of uncertainty?
I’d suggest that the answer to each of these questions is clearly “No”, and there is ample research, theory and experience to back this up (see the end of this post for some links to this literature). But let me illustrate this with an example focused on the 2004 election.
Who won the 2004 election? Well George Bush did, we can be sure of that just by looking to see who is inaugurated next month. But who received more votes? Well, that question, it turns out is a bit more complicated. A paper prepared by Campaignaudit.org, showing research done by students in a fall 2004 class of Communication Technology and Politics at the University of Washington in Seattle, compares error rates in various polling methods with actual margins of victory. They find that in several instances in the 2004 election, the margin of error in polling methods exceeds the margin of victory. This suggests some lingering uncertainty about who actually received the most votes in several states, including Iowa, in the 2004 presidential election. For example, the paper argues that Bush’s margin of victory in Iowa was 0.7% and two different methods suggest that the margin of error in the final tally is 1.0% and 1.3%, meaning that we can estimate the probability that President Bush won, but we cannot not have complete certainty. Here is an excerpt from their paper:
“In this data memo, we explore some of the different ways of estimating the error rate in the 2004 election. There are no pure measures of error in elections, but here we explore three ways of analyzing data about the outcomes of error — technology error, residual votes, and reports of incidents on Election Day. Inherently, this research has political implications, but we first begin by saying something about what this report is not describing. We are not arguing for recalls, recounts, or different political outcomes. We are not recommending one balloting procedure over another. We are not working with statistical models about social inequality, electoral administration and political outcomes. Instead, this is intended to be a focused exploration of margins of error and margins of victory in 2004.”
And if you don’t like this particular analysis of elections and error rates, consider Florida in 2000 or Washington state in 2004 where it is abundantly clear that the margin of victory fell far inside polling margin of error. Who thinks that Washington state will reducing its margin of polling error before installing a governor? Just like in Florida in 2000, a decision will be reached without resolving the fundamental uncertainty. Or to put things another way, it is politics, not information, which is most important for reducing political uncertainty. This is the lesson of policy adoption in just about every major issue of science and policy in recent decades, including the ozone issue (see, e.g., this paper).
If decisions about such apparently simple decision processes as elections are clouded by but not held up by inherent uncertainty, wouldn’t the same be true about more complicated decision processes involving the global environment? Of course. Yet many persist in asserting that reducing uncertainty is a prerequisite for action in a wide range of issues involving science. Why these assertions persist is a topic for another day.
To read more on this, see these two papers by Dan Sarewitz:
(as HTML) Sarewitz, D. 2000. Science and Environmental Policy: An Excess of Objectivity, Chapter
8, pp. 79-98 in R. Frodeman (ed.), Earth Matters: The Earth Sciences, Philosophy,
and the Claims of Community (Upper Saddle River, NJ, Prentice Hall).
(as PDF) Sarewitz, D. 2004. How Science makes Environmental Controversies Worse,
Environmental Science and Policy, 7:385-403.
December 16th, 2004 at 2:05 pm
It’s a bit embarassing how long it took me to catch on to this, which in retrospect seems so obvious. I took a crack at explaining this to newspaper readers here using a particularly apt local case study.