Research Highlight

Regulatory vs. Academic Science

In this issue of Ogmius our Research Highlight features an essay by Anne Ruggles, a wildlife biologist who recently received her law degree from the University of Colorado.  While a law student Anne was an extern with the Center studying the role of science and policy in the Klamath Basin controversy.  The following essay summarizes her analysis of the differences between science applied by regulatory agencies in disputes that arise under legislation such as the Endangered Species Act, and science conducted in academia.

Regulatory vs. Academic Science

Regulatory science--the science used and applied by those charged with enforcing statutes passed by legislatures and designed to protect public health and safety-- increasingly has come under fire. Politicians and interested parties charge either that the results are not protective enough or are too protective. In either case the claims are increasingly accompanied by charges that agency scientists have engaged in “junk science” and calls for oversight by an “objective” panel, usually of academic scientists.

Regulatory science arises within administrative agencies. Their authority is delegated to them by Congress and they reside within the executive branch. They manage an overwhelming proportion of the governmental details of our lives. Agencies are subject to direction by the President, oversight by Congress, and review by the courts. Typically, Congress directs an agency to resolve some general problem and suggests some broad guidelines. The agencies promulgate regulations (subject to the requirements of the Administrative Procedures Act and funding by Congress) to implement the mandate of the statutes passed by Congress, and often enforce those rules.

The result is that, by design, the science promulgated by agencies is imbued with and surrounded by policy and politics – the norms, principles, and values that underlie the creation of the agency itself and the statutes that it was created to implement. Further, statutes and their accompanying regulations establish timelines that agencies must meet. In many instances, there may be too little information, personnel, and funds available to conduct needed research. In short, society, culture, and law create and shape the environment within which information is pursued and used in regulatory science. They influence the types of information sought, its representation, and, perhaps most importantly, its application.

The goals of politics – and by extension regulatory science –are different from and often contradictory to the goals of academic science. The goal of the political process is to resolve conflicts and therefore to facilitate action whereas academic science seeks to increase knowledge about the natural world through an ongoing process of questioning, hypothesis-development, validation, and refutation. Politics and law expect predictive certainty from science, and usually in a relatively short period of time.  In reality nature is complex and evolving; thus results are always accompanied with some degree of uncertainty. Academic science thrives on this uncertainty and progresses by questioning and testing previous results, refining what is known, and asking more questions. Certitude and consensus of interpretation are not the driving forces.

Statistical testing allows estimation of the probability that a hypothesis is true in a given set of circumstances and can lead to two types of error. In a Type I error one concludes that the null hypothesis is false when it is, in fact, true. Under this type of error one would claim an effect when there is none.  Conversely, in a Type II error one concludes that the null hypothesis is true when it is false.  Here one would claim no effect when there is an effect.

Academic science strives to diminish the probability of Type I error. This is crucial to the ongoing process of refining ideas and knowledge of the natural world.

However, when the regulatory emphasis is on protection of a societal value (health, safety, biodiversity) some agencies tend to minimize Type II error rather than minimizing Type I error. In effect agencies will err on the side of concluding that there is a problem. In the context of the Endangered Species Act, for example, this means that if the conclusion is in error and there is no problem, the only “harm” may be a curtailment of private property rights that can be compensated. If, on the other hand, the agencies were to strive to minimize Type I error, they might not take action when appropriate and the result could be species extinction – an event from which there is no recovery.

Because of the differences in mandate, goals, accountability, and emphasis in error analysis (see Table 1) the standards of pure academic science without some modification are inappropriate to the needs and constraints of regulatory science. Both try to understand and explain complex, multivariate, non-linear nature, but they have vastly different constraints. Thus asking a panel of academic scientists to apply the standards applied in writing for journals and applying for research grants to making regulatory decisions is inherently unfair and counterproductive.

Overview of regulatory science by academic scientists is not per se unwelcome; however, when asked to review the work of regulatory scientists, academic scientists should understand the context in which regulatory scientists work and apply criteria that are applicable to the public-sector, regulatory context rather than criteria suitable to evaluating a colleague in an academic setting.

View Table 1: Differences between Academic and Regularatory Responses.

Anne Ruggles
Visiting Scholar 
aruggles@igc.org