Ogmius Exchange: Editorial Response
Chuck Herrick Responds:
Jim Tozzi argues that the Data Quality Act (DQA) will result in increased public confidence in the scientific information used in the regulatory decision making process. As Mr. Tozzi sees it, the "goal of the Data Quality Act [is] to bring...consistency to the quality of government information by codifying requirements that data used and disseminated by the Federal government...be objective." He asserts that this will "instill confidence among the regulated community and other interested stakeholders that agency rules..have a rational basis in science, thereby lessening the frequency of administrative and legal challenges." I cannot share Tozzi's optimism.
It strikes me that the rationale for DQA rests upon a fundamental misunderstanding concerning the nature of scientific assessment in a policy context. Scientific assessments in a policy context frequently address complex phenomena such as global warming, HIV/AIDS intervention programs for specific ethnic populations, or mixed-use management of resources in National Forests. These issue areas cannot be adequately characterized by means of one or two variables or metrics of concern. Rather they involve the integration of dozens of different data sets, numerous and varied models, and findings from perhaps hundreds of separate studies. The credibility of a policy decision simply does not hinge upon the "objectivity" of discrete units of information. Rather, policy and regulatory regimes are typically based upon a wide variety of informational inputs, some of which are more robust than others. To judge quality, it is essential to step back and consider the entire mosaic, and perhaps misleading to zero-in on a few individual tiles.
The integration of scientific information to support a particular policy or regulatory regime is a challenging proposition, involving both science and judgment. In such a context, it is probably unwise to pre-stipulate absolute measures of acceptable data quality. Rather, the value of a particular data set is determined by its "fitness for use" in a particular situation. It therefore makes far more sense to speak of the suitability of data than its objectivity.
Mr. Tozzi's organization, the Center for Regulatory Effectiveness (CRE), recently petitioned the United States Global Climate Change Research Program (GCRP) and the Office of Science and Technology Policy to withdraw the first National Assessment on Global Climate Change "because it violates the objectivity...requirements of the Data Quality Act..." Click here to view the CRE petition letter. Among other things, CRE asserts that the report was "published without...development of the underlying science." I find this claim astonishing. Published by Cambridge University Press, the U.S. National Assessment is a model-based review of alternative future scenarios designed to identify potential vulnerabilities and adaptation strategies. It is meant to enlighten readers regarding how changes in different climate variables may impact future conditions at the regional scale. Its value is heuristic, not prescriptive. In my view, the whole point of exercises such as the National Assessment is to help us consider the co-development of science and policy.
Science is inherently evolutionary, it advances because researchers publish and critique one another's work. Publication is our most important mode of communication and interaction, and it is only through publication and dissemination of findings and alternative scenarios that meaningful research dialog can occur. Withdrawal of the National Assessment would retard both research and policy deliberation, assuring only an ignorant status quo.
Like scientific research, policy formulation is not a rote, menu-driven activity. Policy formulation is an evolutionary and experimental enterprise. Strictly speaking, it is impossible to predict with certitude how a given community will react to a particular policy intervention. As Emery Roe points out, the policy formulation process is characterized by uncertainty, complexity, and incompleteness. Issues are uncertain when causal processes are unclear or not easily understood. Issues are complex when they are more numerous, varied, or interrelated than previously understood. Issues are incomplete when interrupted, postponed or left otherwise unfulfilled in some important aspect. This lack of epistemic grounding could result in paralysis, but it tends not to. Policy makers muddle through, usually by appealing to theories and operating within administrative frameworks that can accommodate complexity, uncertainty, and incompleteness. This is especially true in the areas such as public health, economic development, and environmental policy where decision makers frequently utilize adaptive policy instruments (such as the petition process) to enable us to learn from experience. The inchoate nature of the policy arena makes the very notion of a priori standards for data objectivity highly problematic. For both science and policy formulation, the best way to root out errors and assure vigorous improvement is through wide-spread publication and dissemination of data and information. The more eyeballs that review the material, the richer the debate, and the more likely we are to end up with rational, effective, and equitable public policies.
If administered in an aggressive, inflexible manner, the Data Quality Act has potential to chill and stultify public debate and enlightened policy development. On the other hand, if OMB is flexible, recognizes the applicability of existing data quality systems, respects the idiosyncratic nature of information use in a policy context, and treats the DQA as guidelines - and not as a codification of immutable rules - then the Act could have positive consequences. In particular, I am optimistic that the DQA emphasis on error correction will help make agencies less bureaucratic, more open and transparent, and more responsive to citizen input.
