Issues of Integrity in Climate Science
June 9th, 2005Posted by: admin
Rick Weiss reports in the Washington Port today about a study appearing in the latest Nature on scientific integrity (link and link).
Two points I’d like to make:
1- The study was done on NIH-funded scientists and so doesn’t immediately lend much insight into integrity in climate science (not that Prometheus is focused away from health sciences, but the expertise of the authors trend toward Earth and space sciences, so most posts are in that realm). Health sciences researchers have different challenges in collecting and interpreting data than do Earth scientists. It’s not too much of a stretch to point out that climate science is based on interpreting the past and the future while hoping that both shed light on the present (the present is weather, after all). Medical studies are almost wholly focused on the present. While this perhaps is a subtle distinction, my feeling is that it is likely crucial in how it affects the integrity of research processes.
2- Despite what I write above, the authors of the Nature article (Martinson et al.) make a point relevant for climate studies:
“Our findings suggest that US scientists engage in a range of behaviours extending far beyond FFP [fabrication, falsification, or plagiarism] that can damage the integrity of science.”
Ethical and value issues have been raised repeatedly in the climate sciences (if you’re not a regular reader of Prometheus, just spend ten minutes in the archives), but in the published literature the problem has been often portrayed as a problem of the selective construction and interpretation of models (see the Paul Edwards paper and his citations in Science as Culture, 1999, 8(4), 437-472.).
Edwards points out in his paper that:
“To a large degree these debates are in fact about the model/data relationship: whether model results agree with observations, how much each of these can be trusted, and what sort of role model projections should play in policymaking. I will argue that some parties to these debates have relied upon a conceptual separation between models and data that is, at best, misleading. The interdependent, even symbiotic, relationship between theory and observation in global climate science requires a different conception of the nature of scientific work. Uncertainties exist not only because of quantifiable, reducible empirical and computational limits, but also because of unquantifiable, irreducible epistemological limits related to inductive reasoning and to the nature of model-based global science. These uncertainties can be, and have been, employed as political resources.”
I would argue that this is where the integrity of climate scientists must be examined, and it will be a far more subtle effect than in simply asking about dropping a few data points or any of the ten questions asked by Martinson et al. in the Nature study. For example, the dividing line between judging initial model output as likely or unlikely and then following certain research lines based on initial output is one place likely to be influenced by a “consideration of the wider research environment” as pointed out by Martinson et al.
Martinson et al. go on to say:
“In ongoing analyses, not yet published, we find significant associations between scientific misbehaviour and perceptions of inequities in the resource distribution processes in science. We believe that acknowledging the existence of such perceptions and recognizing that they may negatively affect scientists’ behaviours will help in the search for new ways to promote integrity in science.”
The question becomes: do climate scientists also “misbehave” because of their perceptions on how the political system is treating the climate change question? And if they are misbehaving at anywhere near the rate found by Martinson et al. for medical researchers, what does it say, if anything, for the future intersections of politics and climate science? The question should also be asked in the context of Edwards’ points and so also becomes: If climate scientists are “misbehaving,” where in the process are they most likely to do so? And is this a result of poor scientific training, poor or a lack of ethical training, or some deliberate, evil malfeasance?
The question most fascinating to me is: How are climate scientists “misbehaving” unwittingly, if at all, as a result the various influences of values they are swimming in?
June 10th, 2005 at 9:39 am
Recall that two leading climate scientists, Stanford’s Steve Schneider and NASA’s James Hansen, have both suggested that exaggerations (a type of “misbehaviour”) have been used in an attempt to sway the public’s perception of the seriousness of the global warming issue.
Schneider told Discover magazine, back in 1989:
“On the one hand we are ethically bound to the scientific method, in effect promising to tell the truth, the whole truth, and nothing but… which means that we must include all the doubts, caveats, ifs and buts. On the other hand, we are not just scientists, but human beings as well. And like most people we’d like to see the world a better place, which in this context translates into our working to reduce the risk of potentially disastrous climate change. To do that we have to get some broad-based support, to capture the public’s imagination. That, of course, entails getting loads of media coverage. So we have to offer up scary scenarios, make simplified, dramatic statements, and make little mention of any doubts we might have. This “double ethical bind” which we frequently find ourselves in cannot be solved by any formula. Each of us has to decide what the right balance is between being effective and being honest. I hope that means being both.”
Fourteen years later, in summer 2003, Hansen called for this practice to stop, writing in the on-line journal Natural Science:
“Emphasis on extreme scenarios may have been appropriate at one time, when the public and decision-makers were relatively unaware of the global warming issue, and energy sources such as ‘synfuels,’ shale oil and tar sands were receiving strong consideration. Now, however, the need is for demonstrably objective climate forcing scenarios consistent with what is realistic under current conditions. Scenarios that accurately fit recent and near-future observations have the best chance of bringing all of the important players into the discussion, and they also are what is needed for the purpose of providing policy-makers the most effective and efficient options to stop global warming.”
Obviously, this type of “misbehaviour” — giving more empahsis to extreme scenarios than the scientifically warrant in order to garner attention — is something that is not absent from the minds and actions of climate scientists.
For more examples of “misbehaviour” from climate scientists, be it witting or unwitting, and the reasons behind it, see http://www.worldclimatereport.com
June 14th, 2005 at 12:23 pm
The article by Martinson et al. reports on a survey of NIH grant recipients facing conventional academic pressures. Government scientists, particularly those who serve agencies with policy or regulatory missions, face a somewhat different set of pressures–i.e., to conform their research to, or at least not undermine, the policy or regulatory agendas of their employers. Similar pressures have often been alleged (and sometimes documented) with respect to industry scientists.
A particularly distressing form of scientific misconduct not on the Martinson et al. list (but especially relevant to policy and regulatory science) is the mischaracterization of the published literature. I frequently read papers that cite references which do not support the position or statement alleged. When this occurs in a draft or proposed document the error can be presumed to be unwitting or accidental and not evidence of scientific misconduct. However, when such errors persist after the error is identified, whether through public comment, peer review, or other means, accidental error is no longer a credible explanation.
Another interesting type of potential scientific misconduct consists of drawing inferences that are not supported by evidence. This is a common phenomenon in peer reviewed journals that have specific public policy agendas or ideological convictions. In principle it can be policed by letters to the editor, but journal editors may be conflicted–both because they erred in not preventing the problem from arising in the first place, and because they agree with the unsupported conclusions.