22 Responses to “A Response to RealClimate Concerning A New Survey of Climate Scientists”

1
1. Raven Says:
   October 13th, 2008 at 12:30 pm

   Dr Bray,

   When I made my comments on the RC thread I was responding to what I saw as an attempt by RC to define the terms of a survey in a way that would produce the results they wanted to see. An attempt which I viewed as an exercise in propaganda.

   I did not intend to imply that your survey deserved to be painted with the same brush and I do think it is extremely difficult to come up with a survey that all partisans would agree is impartial because (as you say) most partisan users of this kind of information are only interested in it if it advances their POV.

2
2. Roger Pielke, Jr. Says:
   October 13th, 2008 at 5:29 pm

   Gavin Schmidt of Real Climate responds in his usual condescending fashion over at Real Climate. He apparently does not understand that the trackback link is how you let someone know that you are talking about them on blogs (which is I am sure how he found about about this posting, unless he is a Promethues regular, which I somehow doubt;-).

   His faux indignation is rich given that neither Bray nor Von Storch were notified of the original post or that Real Climate had posted their survey on their site without first asking. Shame.;-) Here are Gavin’s remarks:

   “# gavin Says:
   13 October 2008 at 5:28 PM

   Oddly enough, Bray and von Storch have responded to this post here but curiously did not see fit to let us know.

   One minor typo in their post is worth pointing out. In quoting the post above they replace the word ‘corrupted’ (in reference to previous sampling problems, paragraph 4) with the word ‘corrupt’. This might seem minor, but it is the difference between an accidental problem that makes data untrustworthy and implying the researchers themselves are untrustworthy. This is certainly neither expressed nor was intended.

   As I said in the post, I think that there could well be a great deal of community input that would be useful in improving these kinds of surveys, but avoiding talking to us – even to notify us of a response – does not bode well for that interaction. Shame.”

   http://www.realclimate.org/index.php/archives/2008/09/a-new-survey-of-scientists/langswitch_lang/bg#comment-100666

3
3. Roger Pielke, Jr. Says:
   October 13th, 2008 at 6:50 pm

   Over at RC Gavin now claims that his words have been “distorted” and “misquoted” here. I have invited him to join the discussion, as well as Bray and von Storch, so maybe, if we are lucky, we’ll actually have a conversation about the survey. We’ll see . . .

4
4. chriscolose Says:
   October 14th, 2008 at 1:01 am

   Dennis Bray and Hans von Storch,

   Thank you for the time to post your critique of the RealClimate post. This seems to be turning into another episode of the “ha…I gotcha” game between blogs, with cheerleaders for either party, so I hope to provide a bit of focus. If you are new to the blogosphere, I’d recommend you ignore the cheerleading and simply respond to the primary post, since you’ll drive yourself mad going around various sources and reading each and every comment (and expecting them all to be rational). For instance the claim by Stef that this survey was “debunked” clearly makes no sense, as the RC post was not an attempt to “debunk” anything, nor did you make any positive scientific claim that could possibly be “right” or “wrong”…it was simply a survey, and the RC piece an opinion piece on the quality of questions asked.

   For the most part I agree with your responses to the remarks made by RC commenters, but you said very little about what Dr. Schmidt actually wrote. Some of the objections raised by Gavin are probably too trivial, such as what you meant (Q16) concerning the ability of climate models to predict the next 10 years, etc. Gavin’s point is that a) there are many different meanings to “the models ability to reproduce temperature observations” and b) the ability cannot be determined, since it would require us to be around 10 years later to assess what actually happened to what was modeled…point b is kind of silly and nitpicking at words, since it doesn’t seem to be open to any other interpretation, however point a is very reasonable. He does make a good point however– models do projections, not predictions. This means an entirely different thing, since a projection assumes a given scenario (e.g., a rise of CO2 by 2% per year, no asteroid impacts, the sun doesn’t explode, etc) while a prediction is meant to forecast all possible socio-economic and physical conditions, so unlike a projection where you have “if x, then y” a prediction guarantees x, therefore y. We cannot predict volcanic eruptions or asteroid impacts, and scientists who answer this survey may be nitpickers and give an answer to that question which would have been different than had you said “projections.”

   Clearly Gavin was incorrect in saying that no scientist would be in favor of “exaggerating” in a public setting, but he also makes good points on what “extreme events” are, etc. There were clearly very poorly phrased questions in the survey (e.g., what are things well enough for in Q15?). In my opinion, the effects of a doubling of CO2 are understood well enough for policy action on mitigation, but not good enough to stop research funding and going on to something else. So I’d have to assume what you mean. The essence of the RC post is this– The less assumptions your respondents need to make, the better. There is no way you can argue with that, or the fact that certain questions in this survey require assumptions. You need to see the forest a bit and not the trees in your response of their critique of your survey– Quibbling over who was not contacted or what he said, she said, etc is cute, but RC wanted to assure that questions are properly phrased, not open to question on meaning, and everyone here is hoping that they will not be misused by special interests (denialists or alarmists). I’d hope we all agree with this, so it’s not an attempt by them to discredit everyone else’s opinion as you say in your last paragraph.

   By the way, Gavin is certainly right in that a few of his words were definitely misquoted, and those misquotes lead to a much different context. I hope that you can correct these mistakes (and why a [sic] next to “since?”). I do hope that you guys come over to RC and discuss this directly, so these types of backs and forth don’t get too out of hand. It seems Roger Pielke simply wants to insult Gavin which is non-productive, and certain RC commenters apparently have not understood the RC post, or have not bothered to read the survey. I doubt the aim was to “discredit your work in a public manner” as Roger Pielke thinks, but the public needs to be aware that polling can be a tricky business, and your piece had great intentions but a few of the questions they made examples out of. However, the overall procedure and intent deserves kudos, and I hope we can get reasonable assessments of what the mainstream community thinks. If anything, let’s hope respondents to this survey read the RC post to catch a few of the subtleties.

   Chris

5
5. bray Says:
   October 14th, 2008 at 4:52 am

   As it seems that never the twain shall meet, this is a cross posting, also posted on RC, at least in part.

   As on RC

   I would like to briefly comment on some of the cyber shenanigans of late concerning the survey of climate scientists.

   First, to give Hank Roberts (on RC) peace of mind: The first survey (1996) was a hard copy mailout in multiple languages. The second survey (2003) was on-line.

   Gavin (RC) wrote: ‘I really do think that these surveys could be done much better – and the idea proposed above of having an open wiki where the questions could be hashed out ahead of time is still I think a good one. Bray and von Storch did not respond to that, nor to the issues of precision in the questions. These are important issues to get right.’

   On the Wiki suggestion …

   The idea of the Wiki proposal raises a couple of concerns which seem to predominate many of the comments on the survey: How would you control the respondents, i.e. would the questions be open for discussion by anyone so wishing to participate or would the participants have to be a practitioner of that particular specialty? Would there be community censorship on sensitive questions? (Just as an aside, haven’t some of the more damning criticisms – some of which might be valid – come from outside of the climate science community in the last couple of years?) What if questions were posed that could not be answered, would they simply be ignored? Who would be the intended audience of the responses? Would it be an in-house assessment with no room for critical comment? Might it turn out to be little more than asking the members of the Vatican to evaluate the significance of the Vatican? So first I would suggest it is necessary to establish the PURPOSE of the survey. Ours was to get scientific OPINIONS concerning broad aspects of the climate sciences. We clearly stated the intention of the survey in the letter if invitation:

   1 identifying areas in need of increased research and/or focus
   2.suggesting funding priorities for research
   3.providing the opportunity for the science community to express an opinion concerning the dissemination of scientific results

   It will provide a collective perspective of the climate science community on matters of climate science and climate change from the perspective of climate scientists.

   On precision …

   The precision necessary to finely tune questions to each specialization in climate science would further result in a much lengthier questionnaire than any of the original surveys, with a much reduced sample size with the ability to respond to such specific questions, so what would be the representativeness of the survey?

   Still on the matter of precision …

   Gavin also comments concerning the precision of a question asking as to the acceptance of presenting extreme scenarios. He says ‘A more specific question involving a real case would have been a much better thing to ask about.’

   A ‘more specific real case’ might be fine for a more specific sample. Unfortunately, the global climate science community does not all read the same newspaper nor watch the same TV station. We simply wanted to know if climate scientists agreed with this practice and the question as posed captures this opinion, in my assessment, without any problem. Or, is it the case, that you feel some cases should be exaggerated and some not?

   precision continued …

   We had in mind to address a general broad spectrum of climate scientists and to do so the questions needed be be somewhat general.

   question content and preemptive conclusions

   Back to the general posting, Gavin stated ‘One set of questions (Q68+Q69) obviously come from a social rather than a climate scientist: Q68 asks whether science has as its main activity to falsify or verify existing hypothesis or something else [Gavin, you were very critical of my misquoting you in one of your comments. Please note that the question does NOT read ‘verify existing hypothesis or something else’, it reads ‘verify existing CONDITION and OTHER. A minor point to be sure and no less petty than your own comment]; and Q69 whether the role of science tends towards the deligitimization or the legitimization of existing ‘facts’ or something else. What is one to make of them? [I am a sociologist and oddly as it may seem, I have an interest in sociology] There are shades of Karl Popper and social constructivism in there, but we’d be very surprised if any working scientist answered anything other than ‘other’.

   Well, once more your preemptive assumptions prove to be incorrect:

   The frequencies of responses are as such:

   68. Concerning what science is in general, what would you say is its main activity?

   to falsify hypothesis – 88
   to verify existing conditions – 100
   other – 179

   69. Concerning science in general, the role of science tends towards

   deligitimization of existing ‘facts’ – 45
   legitimization of existing ‘facts’ – 132
   other – 179

   Surprise (again), but that is what the data says.

   This part I do not understand:

   Gavin concludes: ‘Thus I urge Dennis Bray (who I think wrote the response) to engage more directly, and maybe ask himself (and the readers here) why the responses were as they were? There may well be lessons to learn there too.’

   Why the responses were as they were? Are the responses incorrect? Or is it perhaps they simply do not fit with preconceived notions and are therefore incorrect? Furthermore, little in the way of results have been made public, so how on earth do you know what the responses are?

   Now on to Prom. comments – this section not on my post to RC

   To Chris Colose: (I can’t find the link to the cheerleaders!) You do though provide good advice in that one should ignore most of the comments – fun, but far from productive. But there are some things that warrant a response. So I will respond to your comments (well, some of them).

   Question 16 seems to raise some confusion although I am not sure why. For the sake of clarification the question reads:

   How would you rate the ability of climate models to:
   (response options – 1 – very poor 7 – very good)

   reproduce temperature observations [I am not sure what is not clear about this part of the question]; reproduce temperature VALUES for the next 10 years; for the next 50 years; to model sea-level rise for the next 10 years; to model extreme events for the next 10 years?

   If the respondent felt that the model had the a good ability to detect extreme events for the next ten years then he or she could respond with very good. If he or she doubted the output of the model then he or she could respond with very poor.

   There is no use of the word ‘predict’.

   The question asks nothing about the ability of future MODELS.

   Chris, referring to Gavin says ‘He does make a good point however– models do projections, not predictions.’

   I would agree, but as I said, I did not use the word prediction nor projection for precisely that reason – I used ‘model’. In the survey we did indeed ask climate scientists what they felt models produced. Contrary to what again appears popular belief, I would like to present some of the data.

   The question reads ‘From the output of global climate models, climate scientists are more inclined to make:

   frequency of responses

   projections – 256
   prediction – 88
   other – 17

   Concerning regional models, the results are even more startling

   projections – 217
   predictions – 120
   other – 20

   So to me it seems perhaps climate scientists with preconceived notions of the opinions/perspectives of their peers are perhaps not the optimal group to compile a questionnaire to address the more general aspects of the opinions/perspectives of the practitioners of climate science, i.e. as someone mentioned, can’t see the forest for the trees. Perhaps to assess the intricacies of their specialties yes, but a majority opinion, no.

   Again, in trying to address generalities Chris say ‘Clearly Gavin was incorrect in saying that no scientist would be in favor of “exaggerating” in a public setting, but he also makes good points on what “extreme events” are, etc.’

   I would like to ask how much precision would be necessary? Temperatures of this magnitude, waves of this height, winds of this speed, etc. etc.? Exaggerating is the key word, not the magnitude of the extreme event.

   Question 15 The current state of scientific knowledge is developed well enough to allow for a reasonable assessment of the effects of …

   turbulence; … sea ice … green house gases …

   This is simply asking the state of art of the science. No one would expect 100% perfect understanding. The intention of the series of questions it to identify RELATIVE weak points, not an absolute evaluation.

   Chris says ‘The essence of the RC post is this– The less assumptions your respondents need to make, the better. There is no way you can argue with that, or the fact that certain questions in this survey require assumptions.’

   The questions were pre tested a number of times with climate scientists and the questions raising the most concern and subsequently removed from the survey were the more specific questions.

   Let’s take an example. If I was to pose the question ‘are your shoes red’ with a yes no response option, a response of yes would provide some very specific detail (although no doubt there would be arguments over what shade of red qualifies as red i.e. pink, puce, etc). But for those answering no, we would have no information whatsoever, only that the shoes were not red and in fact, they might even be pink or puce or whatever but were not considered red.

   So I would like to propose a little exercise for the critics of the questions before they venture into wiki question construction. Design a question and a set of responses designed to capture the simple concept of color of shoes from 100 people. After collecting the data attempt to identify the shoes according to the respondent’s response.

   The goal of our survey was to capture as much information as possible. In the case of climate science, too much specificity would appear like a test of personal knowledge and would most likely be very detrimental to response rates. I suppose if one is in a particular philosophical mood he or she could ponder just about any question or any statement looking for a range of possible meanings.

   Chris says ‘everyone here is hoping that they will not be misused by special interests (denialists or alarmists). I’d hope we all agree with this… ‘

   I think this is indeed an agreed upon point

   Chris again ‘… so it’s not an attempt by them to discredit everyone else’s opinion as you say in your last paragraph.’

   Here I assume you mean the following comment:

   The frightening thing about the Real Climate post and the quality of debate on its site is the rush to conclusions by people insistent that any opinion other than their own is wrong.

   Well, all I can say is that there certainly seemed to be a rush to criticize, the survey, draw conclusions, claim the invalidity of the questions and responses, and the sample etc. all before any results were made public. In fact, these comments were made before the survey was even closed. Perhaps they were made based on assumptions. If one happens to be in the pondering of assumptions mood, this might be a good way to pas the afternoon.

6
6. EDaniel Says:
   October 14th, 2008 at 6:08 am

   It seems to me that this post by Pielke Sr., which links to this post, which is partly based on this report which is found on this page is related to the present discussions.

7
7. Tamara Says:
   October 14th, 2008 at 6:58 am

   Question 16 only seems to raise the hackles of those who are predisposed to defend the model predictions. It is clear that this question is asking the scientists to give their opinions as to the usefulness of the tool, not the specific results. Gavin’s objection seems a bit absurd to me. If you asked a group of doctors to rate the usefulness of a particular medical procedure you wouldn’t expect them to say “Well, we can’t answer that until we know what happened to all of the patients.”

8
8. Roger Pielke, Jr. Says:
   October 14th, 2008 at 6:59 am

   Gavin likes to teach people “lessons”, apparently now their expertise includes the design of social science surveys. Here is Gavin again admonishing sociologist Bray on his survey:

   ‘Thus I urge Dennis Bray (who I think wrote the response) to engage more directly, and maybe ask himself (and the readers here) why the responses were as they were? There may well be lessons to learn there too.’

   Presumably by “responses” he is referring to (and excusing) the vitriolic responses to his original post (or at least the ones that RC allowed through). As John McCain learned last week there is a difference between intelligent criticism and inciting the masses against your opponents.

9
9. Roger Pielke, Jr. Says:
   October 14th, 2008 at 7:09 am

   Dennis-

   On the results of the survey that you have released thus far, it seems clear that there is some considerable texture to the views of climate scientists and considerably more uncertainty than commonly represented.

   How else to understand the responses to the following question?

   How convinced are you that climate change poses a very serious and dangerous threat to humanity? (response range: 1 = not at all, 7 = very much)

   More than 20% of responses were either agnostic on this question or leaned toward the “not at all” side.

   I am not surprised that 1/7 of your responses thought that exaggeration was OK to get the public’s attention.

10
10. Paul Biggs Says:
    October 14th, 2008 at 10:06 am

    You have to feel for RC – they know the King has no clothes – but they continue to try and shield him hoping no one sees. RC want to perpetuate the ‘2500 scientists’ myth – the last thing they want is for someone to actually ask scientists individually for their opinion.

11
11. Carrick Says:
    October 14th, 2008 at 10:29 am

    ChrisColose:
    

    I do hope that you guys come over to RC and discuss this directly, so these types of backs and forth don’t get too out of hand.

    Why would anybody want to do that? I mean that seriously.

    Responses are often edited or even suppressed at the whim of the maintainers of that site (literally “whim”, I doubt they have any standards for what gets included, edited or outright rejected, let alone an appeal process for inappropriate behavior on the part of the editors). Were RC actually a scientific blog, instead of a advocacy website, I would have considered such editing and suppression an outright breach of scientific ethics.

    Since it is not, they certainly have the right to edit or suppress opposing view points. But that hardly makes it the the appropriate site for an even-handed exchange of any sort.

    I don’t have problems with people like Chris or Gavin choosing to advocate a particular policy position. I just think it is important to note that there is a difference between policy advocacy and scientific discussion. And that unfortunately the two really mix about as well as oil and water.

12
12. Gary Says:
    October 14th, 2008 at 12:41 pm

    Chriscolose recommends that the discussion of this survey be carried out over at RC. For readers who have posted at RC we know that besides the rudeness that is common there, questioning of the work of Dr. Mann or Dr. Hansen results in posts being rejected. Not exactly some place for serious discussion unless you agree with Gavin.

13
13. Deltoid Says:
    October 14th, 2008 at 12:44 pm

    Bray again…

    Remember Dennis Bray’s useless survey of climate scientists? The URL and password were posted to the climatesceptics mail list, so the results were biased and included responses from people who were not climate scientists. Bray refused to concede that…

14
14. PollsFacts » Blog Archive » Bray again [Deltoid] Says:
    October 14th, 2008 at 1:42 pm

    [...] (and his co-author Von Storch) has a response to Schmidt over at Prometheus. The bit that concerns me is this: Read the rest of this post… | Read the [...]

15
15. Doug White Says:
    October 14th, 2008 at 2:58 pm

    Oh my heck, Roger, your response on RC as to Gavin’s question “What do you think this post is about” absolutely floored me. You’re right, of course, it WAS a pre-emptive hit piece designed to discredit; I doubt anyone (Gavin included) expected you to answer him honestly. Thanks for pointing out the obvious to RC. Someone had to do it.

16
16. TokyoTom Says:
    October 14th, 2008 at 9:41 pm

    Dennis Bray says: Our questionnaire can be viewed at on the RealClimate website where it was reproduced without notification or a request for permission to post.

    Without addressing the question of whether it was improper or discourteous of RealClimate to post the questionnaire, can I suggest that you post a copy at Prometheus and link to it? That way the reader doesn’t have to go searching for the survey over at RS (and you don’t by default leave RS as the authority of what the survey says).

17
17. TokyoTom Says:
    October 14th, 2008 at 9:53 pm

    Paul: RC want to perpetuate the ‘2500 scientists’ myth – the last thing they want is for someone to actually ask scientists individually for their opinion.

    1. Assuming arguendo that RC has the intent you specify, what’s the relevance of the 2500 number? Isn’t it rather clear that most climate scientists believe that human activities are exerting a significant influence on climate? Or are you trying to balance of these climate scientists with the larger numbers produced by extremely sloppy surveys like OISM of non-climate scientists?

    2. How do you derive your second assertion, when Gavin specifically suggests ways that surveys such as that by Bray and Von Storch could be fine-tuned?

18
18. TokyoTom Says:
    October 14th, 2008 at 10:05 pm

    Roger, sometimes I have very impressed with your ability to help others to move past ruffled feathers and mutual misperceptions and on to substantive issues.

    By the way, I note that there is a discussion of another recent survey of scientists regarding climate here: http://stats.org/stories/2008/global_warming_survey_apr23_08.html. (h/t commenters Joel Shore and John Cross at Deltoid: http://scienceblogs.com/deltoid/2008/10/bray_again.php).

19
19. bray Says:
    October 15th, 2008 at 12:57 am

    Re 13 – 14 from Tim Lambert over at Deltoid

    Well, it’s nice to see that Tim Lambert over at Deltoid has finally jumped into the fray. A little late though. Too bad he provides somewhat of a useless comment on the survey of climate scientists drawing from his five year old posting of useless survey of climate scientists. I could say much more but I have neither the time nor the inclination.

    My foray into blogworld has been what … amusing, I guess. But I must bid farewell to the Hatfields and the McCoys. Alas, I must return to more responsible duties – I am milk monitor of the week!

    Before I go I would like to offer a little prayer, penned by Roberston Davies long before Digital Daze, but perhaps particularly apt for blogging. It goes ‘God give me oblivion from the small small voices of small small people.’

    Amen to that

20
20. solman Says:
    October 15th, 2008 at 11:27 am

    Real Climate censors the vast majority of posts that disagree with Gavin.

    It is absurd to suggest that any meaningful dialog can exist in an environment in which one party has the ability to censor all dissenting opinion, and makes liberal use of this ability.

21
21. Mark Bahner Says:
    October 16th, 2008 at 10:32 am

    Hi,

    I hope Dennis Bray (or Hans von Storch) reads this.

    It’s difficult to separate the wheat from the chaff in blogs, in part because there’s a whole lot of chaff. But that doesn’t mean there isn’t wheat in there.

    Dennis, you misunderstood Gavin Schmidt’s comment about a wiki for your survey. I hope you now understand that the wiki was to formulate the *questions*, not to get answers. I think that is actually a pretty good suggestion. Another possibility would be to simply send the survey to some people (who won’t be responding) for comments about the questions.

    For example, right off the top of my head, just looking at the questions you’ve posted here, I could think of some suggestions for changes:

    “How convinced are you that most of the recent or near future climate change is, or will be, a result of anthropogenic causes?”

    First, to which aspect of “climate” are you referring? Global average temperatures? Regional temperatures? Precipitation? Frequency and intensity of hurricanes, typhoons and cyclones?

    Second, there is no time given for “recent.” For example, there hasn’t been a year where the global average surface temperature has been significantly hotter than 1998 since that year. On the other hand, if one goes back to 1979, both global average surface temperatures and satellite-measured tropospheric temperatures have increased. So what is “recent”…since 1998 (the last 10 years) or since 1979 (the last 30 years)? Or some other period, such as even the past 100 years (which is very “recent” in geological time).

    So a better question would have been something like, “How convinced are you that most of the global average temperature change of the past 30 years, or next 30 years, will be a result of anthropogenic causes?”

    It would also be interesting to break that question into two questions: one for the past 30 years, and one for the next 30 years. And it would be interesting to ask a question about the expected global change in surface temperature by 2030, e.g.:

    “Relative to a 5-year global average centered around 2005, what do you think the increase or decrease in global average temperature (in degrees Celsius) will be for a 5-year average centered around 2030?”

    All of the new/revised questions I proposed could probably be improved upon. That’s what a wiki has the potential to do.

    Best wishes,
    Mark

22
22. eadler Says:
    October 19th, 2008 at 7:29 pm

    I don’t see what all the fuss is about.
    Based on the survey results, if my mental arithmetic is correct.

    92.6% agree that climate change is happening now.
    83.5% are convinced that climate changes are anthropogenic
    85.5 are convinced that climate change is a threat to humanity

    In my calculations I have taken 4 as the dividing line between positive and negative responses.

    This is indicates a consensus that AGW is happening and is something that needs to be dealt with. It is what one would expect given the scientific research results as I understand them.