Comments on: Hypotheses about IPCC and Peer Review http://cstpr.colorado.edu/prometheus/?p=3713 Wed, 29 Jul 2009 22:36:51 -0600 http://wordpress.org/?v=2.9.1 hourly 1 By: kevin http://cstpr.colorado.edu/prometheus/?p=3713&cpage=1#comment-2826 kevin Wed, 01 Feb 2006 20:41:53 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=3713#comment-2826 The grey lit issue was one of my strongest criticisms in the two AR4 WGII chapters I reviewed. Perhaps 'appalled' is too strong of a word, but I was unimpressed to the point of annoyance by the reliance on NGO reports to make some points and a conspicuous lack of citation in making others. The message I have been taking from this (although I am waiting on the revised chapters) is that the knowledge base just isn't there on some issues that were tagged to be covered in AR4. But rather than report, "We have no idea," the lead authors must feel that they have to give *something*. The grey lit issue was one of my strongest criticisms in the two AR4 WGII chapters I reviewed. Perhaps ‘appalled’ is too strong of a word, but I was unimpressed to the point of annoyance by the reliance on NGO reports to make some points and a conspicuous lack of citation in making others. The message I have been taking from this (although I am waiting on the revised chapters) is that the knowledge base just isn’t there on some issues that were tagged to be covered in AR4. But rather than report, “We have no idea,” the lead authors must feel that they have to give *something*.

]]>
By: James Annan http://cstpr.colorado.edu/prometheus/?p=3713&cpage=1#comment-2825 James Annan Tue, 31 Jan 2006 03:48:18 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=3713#comment-2825 Actually I think a (controlled) wiki between the relevant people could possibly be a very much more efficient and effective method than the current one. But OTOH I'm going to play my standard card that it's not reasonable to blame the "at the coal-face" scientists for merely getting on with the task at hand, and deadlines and peer-reviewed "quality control", although not perfect, are not obviously wrong in that respect. Actually I think a (controlled) wiki between the relevant people could possibly be a very much more efficient and effective method than the current one. But OTOH I’m going to play my standard card that it’s not reasonable to blame the “at the coal-face” scientists for merely getting on with the task at hand, and deadlines and peer-reviewed “quality control”, although not perfect, are not obviously wrong in that respect.

]]>
By: Rabett http://cstpr.colorado.edu/prometheus/?p=3713&cpage=1#comment-2824 Rabett Mon, 30 Jan 2006 23:16:39 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=3713#comment-2824 Roger, for a guy who posts weekly reading lists, you sure do play a mean game of you do it. Come on, you know that the questions you asked have been pretty much addressed, but you tried to put James off by giving him a list to wade through. The one about doing a Wiki for assessment was particularly amusing in a "is he silly enough to think that anyone would fall for that" sort of way. Roger, for a guy who posts weekly reading lists, you sure do play a mean game of you do it. Come on, you know that the questions you asked have been pretty much addressed, but you tried to put James off by giving him a list to wade through.

The one about doing a Wiki for assessment was particularly amusing in a “is he silly enough to think that anyone would fall for that” sort of way.

]]>
By: Roger Pielke, Jr. http://cstpr.colorado.edu/prometheus/?p=3713&cpage=1#comment-2823 Roger Pielke, Jr. Mon, 30 Jan 2006 15:53:19 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=3713#comment-2823 Jeff- Excellent point! Jeff- Excellent point!

]]>
By: Jeff Norman http://cstpr.colorado.edu/prometheus/?p=3713&cpage=1#comment-2822 Jeff Norman Mon, 30 Jan 2006 14:04:55 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=3713#comment-2822 Roger, An addition to your list might be recommendations from the IPCC to policy makers regarding climate research in their countries and/or regions that could be funded or otherwise facilitated to promote a better understanding of climate systems. This would help fill in some of the holes in our understanding. It is apparent to me that numerous weather station records ended or became incomplete since the start of the recent warm period. At the very least the IPCC could be recommending to policy makers that these records be maintained. Jeff Roger,

An addition to your list might be recommendations from the IPCC to policy makers regarding climate research in their countries and/or regions that could be funded or otherwise facilitated to promote a better understanding of climate systems. This would help fill in some of the holes in our understanding.

It is apparent to me that numerous weather station records ended or became incomplete since the start of the recent warm period. At the very least the IPCC could be recommending to policy makers that these records be maintained.

Jeff

]]>
By: Roger Pielke Jr. http://cstpr.colorado.edu/prometheus/?p=3713&cpage=1#comment-2821 Roger Pielke Jr. Sun, 29 Jan 2006 03:30:18 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=3713#comment-2821 Eli- Thanks for your comments. I'm sure our readers would welcome your summaries of this literature and your views of its relevance to this issue. Eli- Thanks for your comments. I’m sure our readers would welcome your summaries of this literature and your views of its relevance to this issue.

]]>
By: Rabett http://cstpr.colorado.edu/prometheus/?p=3713&cpage=1#comment-2820 Rabett Sun, 29 Jan 2006 03:05:56 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=3713#comment-2820 AFAIK Science and Nature consider seeing it in the press as "prior". They extend this to press releases and maybe even blogs, and reporters do walk around AGU meetings, believe it or not, and university press officers are always looking for stuff to brag on. Roger, your questions are shallow. Let us look at the first three: *Why have working groups? *Why these three? *Why not focus on policy recommendations? Have you perhaps read: Structure and Agent in the Scientific Diplomacy of Climate Change: An Empirical Case Study... by Tora Skodvin - or perhaps Confronting Climate Change: Risks, Implications and Responses by Irving Mintzer and any number of article in various journals or are you simply trying to blow us off by generating an endless list? AFAIK Science and Nature consider seeing it in the press as “prior”. They extend this to press releases and maybe even blogs, and reporters do walk around AGU meetings, believe it or not, and university press officers are always looking for stuff to brag on.

Roger, your questions are shallow. Let us look at the first three:

*Why have working groups?
*Why these three?
*Why not focus on policy recommendations?

Have you perhaps read:

Structure and Agent in the Scientific Diplomacy of Climate Change: An Empirical Case Study…
by Tora Skodvin -

or perhaps

Confronting Climate Change: Risks, Implications and Responses
by Irving Mintzer

and any number of article in various journals or are you simply trying to blow us off by generating an endless list?

]]>
By: Roger Pielke Jr. http://cstpr.colorado.edu/prometheus/?p=3713&cpage=1#comment-2819 Roger Pielke Jr. Fri, 27 Jan 2006 23:23:42 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=3713#comment-2819 James- Thanks for these comments. There are a lot of potential ways that things could be done differently, and maybe these would be better, maybe not. But every organization should routinely be evaluated and compared to possible alternatives. We should not fall into the status quo trap -- that things are the way they are because they represent the best possible outcome. As examples of questions that might be raised: *Why have working groups? *Why these three? *Why not focus on policy recommendations? *Why focus on massive assessments? *Why not a "Wiki" model for assessment, updated continuously? *Why not start fresh each time with questions solicited from policy makers, and then issue smaller reports as appropriate? *Why not use expert elicitation as an alternative/complement? *Why not have people uninvolved in producing the research charged with summarizing it? *Why not oganize assessments under policy bodies? *Why not use assessments to do research not just summarize it? *Why not use the NOAA RISA model? US NACC model? NRC model? Others? In short there are many, many questions that might be asked about the IPCC, but aren't mostly because everyone who cares has either a vested interested in preserving the status quo or attacking it. The answers to the questions above implicate a whole spectrum of different approaches to the time from/peer review issue. As far as you last point, the Science/Nature embargo is relevant to the IPCC because scientists use it as an excuse not to engage in scientific debate that might result in perspectives other than their being aired, and thus not represented in the peer reviewed liertature and not available on the time schedule of the IPCC. But you are ight that it is a much broader issue than IPCC or climate science. James-

Thanks for these comments.

There are a lot of potential ways that things could be done differently, and maybe these would be better, maybe not. But every organization should routinely be evaluated and compared to possible alternatives. We should not fall into the status quo trap — that things are the way they are because they represent the best possible outcome.

As examples of questions that might be raised:

*Why have working groups?
*Why these three?
*Why not focus on policy recommendations?
*Why focus on massive assessments?
*Why not a “Wiki” model for assessment, updated continuously?
*Why not start fresh each time with questions solicited from policy makers, and then issue smaller reports as appropriate?
*Why not use expert elicitation as an alternative/complement?
*Why not have people uninvolved in producing the research charged with summarizing it?
*Why not oganize assessments under policy bodies?
*Why not use assessments to do research not just summarize it?
*Why not use the NOAA RISA model? US NACC model? NRC model? Others?

In short there are many, many questions that might be asked about the IPCC, but aren’t mostly because everyone who cares has either a vested interested in preserving the status quo or attacking it. The answers to the questions above implicate a whole spectrum of different approaches to the time from/peer review issue.

As far as you last point, the Science/Nature embargo is relevant to the IPCC because scientists use it as an excuse not to engage in scientific debate that might result in perspectives other than their being aired, and thus not represented in the peer reviewed liertature and not available on the time schedule of the IPCC. But you are ight that it is a much broader issue than IPCC or climate science.

]]>
By: James Annan http://cstpr.colorado.edu/prometheus/?p=3713&cpage=1#comment-2818 James Annan Fri, 27 Jan 2006 22:07:26 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=3713#comment-2818 Roger, While I agree that these issues are interesting ones, the real question is, how could it be done better? Given that an assessment report on this scale is desired, the authors need a timetable to work to, and some parameters to work with. I have commented previously on how disproportionate I think the influence of the IPCC is on the science that is performed, especially since they don't actually fund it. But on the plus side, the timetable and pull for contributions has surely pushed forward the rate of scientific research beyond that which would have otherwise existed. And it's simply not fair of you to bring up the Science/Nature embargo in the same article - this is an entirely separate business. Roger,

While I agree that these issues are interesting ones, the real question is, how could it be done better? Given that an assessment report on this scale is desired, the authors need a timetable to work to, and some parameters to work with.

I have commented previously on how disproportionate I think the influence of the IPCC is on the science that is performed, especially since they don’t actually fund it. But on the plus side, the timetable and pull for contributions has surely pushed forward the rate of scientific research beyond that which would have otherwise existed.

And it’s simply not fair of you to bring up the Science/Nature embargo in the same article – this is an entirely separate business.

]]>
By: William Connolley http://cstpr.colorado.edu/prometheus/?p=3713&cpage=1#comment-2817 William Connolley Fri, 27 Jan 2006 19:20:43 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=3713#comment-2817 Embargoed wok? Hot stuff! Embargoed wok? Hot stuff!

]]>