Research Highlight

Policy Dimensions of NSF’s Criterion 2

Bob Frodeman was a researcher at the Center for Science and Technology Policy Research from 2002-03 and is now a Center Affiliate.  This research highlight discusses some of the work that he initiated at the Center and now continues at the University of North Texas where he chairs the Department of Philosophy and Religion Studies.  The coauthor, Britt Holbrook, is an assistant professor in the Department of Philosophy and Religion Studies at UNT.

In their 1998 Physics Today article “Beyond Basic and Applied” Roger A. Pielke, Jr. and Radford Byerly, Jr. offer a critique of the dominant vision of the science-society relationship.  Pielke and Byerly describe a “paradoxical social contract” based on Vannevar Bush’s distinction between “basic” and “applied” research and the linear relationship between them, where “basic” research, conducted without concern for societal benefits, is a necessary precondition of “applied” research, which directs the pure findings of “basic” research toward societal goals.  Although society is ultimately interested in the beneficial applications of science and not science for science’s sake, without funding pure “basic” research, no such applications will be forthcoming.  Bush’s genius lies in managing the rhetorical feat of preserving the autonomy of scientists as a means of making their work relevant to society.   But this places society in a paradoxical position: only by funding scientific research not directed toward societal benefits will any of the societal benefits of scientific research be realized.

In calling for the abandonment of the Bush model, Pielke and Byerly are not alone.  Democratic Senator Barbara Mikulski and former Republican Speaker of the House Newt Gingrich have both argued that we should move away from the Bush model toward a model that emphasizes “strategic” research. Daniel Sarewitz (1996) has also urged a move away from the Bush model of the science-society relationship. Daniel Stokes (1997) suggested a use-inspired model for understanding the connections between science and society.

The Bush model remains popular with scientists, however, and calls to abandon it can easily be seen by scientists as backing them into the opposite of the corner into which they have heretofore forced society.  From the perspective of scientists who hold on to the Bush distinction between “basic” and “applied” research, the following statement will sound like a paradox: only by conducting scientific research directed toward societal benefits will any societal funding of basic scientific research be realized.  From the scientist’s point of view, the suggestion that we abandon the Bush model, much like calls for the “democratization” of science, threatens the absolutely necessary autonomy, not to mention the highly desirable purity, of science.

Take the example provided by the U.S. National Science Foundation’s (NSF) 1997 adoption of two new generic merit review criteria: (1) What is the intellectual merit of the proposed activity? and (2) What are the broader impacts of the proposed activity?  On the face of things, NSF seems to have two equally essential criteria on which to base funding decisions: one for purely scientific merit, and a second for societal benefit.

In fact, the National Science Board (NSB), NSF’s policy branch, restructured the merit review criteria largely to respond to increased demand for an account of the societal benefits achieved by NSF funded projects.  Congress had passed the Government Performance Results Act (GPRA) in 1993.  GPRA sends the message that Federal funding will go to those agencies that produced “results.”  This message has also been reinforced since President George W. Bush took office by the President’s Management Agenda (PMA), as well as the establishment of the Program Assessment Rating Tool (PART), designed specifically to tie GPRA to budget formation.

Yet, rather than embracing the new criteria as a way of “doing their part” in justifying continued Federal funding, many scientists have reacted negatively.  Summarizing a survey of the science and engineering community regarding the new criteria, the NSB Task Force on Merit Review noted that Criterion 1 was perceived by respondents as more important than Criterion 2, and that Criterion 2 was often perceived as irrelevant, ambiguous, or poorly worded.  Moreover, many scientists surveyed expressed the belief that it is impossible to make meaningful statements about the potential usefulness of basic research. Indeed, as a 2001 report on the new merit review criteria by the National Academy of Public Administration (NAPA) states, “the concept of broader social impact raises philosophical issues for many reviewers – in particular, reviewers who see their task as exclusively one of assessing the intellectual merit of proposals” (authors’ emphasis).

 Since 1997, the number of scientists who address both criteria in proposals and reviews has steadily increased.  Yet a glance at the Reports of the Committees of Visitors (COVs), outside experts who provide guidance to NSF, reveals that the quality of responses to Criterion 2 is lacking.  In fact, the FY2005 Report of the Advisory Committee for GPRA Performance Assessment (AC/GPA) issued on July 20, 2005 reiterates the persistence of the problem.  The report notes that in 2004, 92% of reviewers addressed Criterion 2 (up from 90% in 2003, 84% in 2002, and only 69% in 2001), which represents “clear improvement” in terms of the quantity of reviewers who address Criterion 2.  However, the report continues, “While most COVs mention this improvement, they also all continue to cite the uneven attention of reviewers to Criterion 2 because reviewers, proposers, and POs still don’t fully understand and apply these criteria consistently” (our emphasis).  Although the quantity of scientists who address Criterion 2 has improved, the quality of their attention remains an issue.  Just as society resists funding basic research without an account of its benefits, scientists resist giving an account of the societal benefits of basic research.

The consensus among policy analysts today is that the social contract between science and society needs to be refurbished.  As their reaction to Criterion 2 tells us, scientists are having trouble relating their basic research to societally beneficial applications.  We suggest that when a new model of the science-society relationship comes to be embraced, it will take at least one lesson from Vannevar Bush. The relationship between science and society is a paradoxical one, in that science must be both tethered to and autonomous from society. Any account worth defending will have to account somehow for both of these points.

References

National Academy of Public Administration (NAPA). A Study of the National Science Foundation’s Criteria for Project Selection, a report by the National Academy of Public Administration for the National Science Foundation, February 2001.

NSB Task Force on Merit Review’s Final Recommendations (NSB/MR 97-05).

NSF. FY2005 Report of the AC/GPA (NSF 05-210).

Pielke, Jr., Roger A. and Radford Byerly, Jr. “Beyond Basic and Applied,” in Physics Today, February 1998, p.p. 42-6.

Sarewitz, Daniel.  Frontiers of Illusion: Science, Technology, and the Politics of Progress, (Philadelphia: Temple University Press, 1996).

Stokes, Donald E. Pasteur’s Quadrant: Basic Science and Technological Research (Washington, D.C.: Brookings Institution Press, 1997).