That being said, unusual results will attract more methodological scrutiny than ordinary results, so dodgy methodology is more likely to be discovered in experiments that give funny answers.

I wrote a post on how, and how not, to approach results outside the mainstream in (non-climate-related) geochronology here:
http://lablemminglounge.blogspot.com/ 2006/09/zircon-that-predates-universe.html

with part 2 at:
http://lablemminglounge.blogspot.com/ 2006/10/zircon-that-predates-universe-2.html

How are studies selected for IPCC?

Are some studies discarded because they don’t show the same conclusion of others?

(Example:If 10 paleoclimates studies are available. Of them 9 have similar conclusion that the MWP was cooler than and 1 place it hotter. Does the IPCC conclude that MWP was cooler or do they leave the door open to the one that doesn’t show similar result. Do they try to replicate studies results to see which one is more likely.)

I hope that my question are clear enough.

Thanks for the clear answers to my questions. This is helpful and I think I understand better what you’re saying.

Your point about needing some process however imperfect is good. I agree with you there.

It seems that for all the heated words from Andrew Dessler last week that you’re not saying anything radically different from him except that without a transparent process at IPCC, it’s asking a lot for those of us on the outside to accept on faith that politics had little to do with the evolution of the WG1 document. If this is what you’re saying, I fully agree.

I’ll get back to my last question in a separate comment because I’m having a hard time writing it both clearly and concisely just now.

Thanks much for these thoughtful comments and questions. Let me try to respond to each of your points, and let me know if I haven’t!

1. I agree with you 100% about the notion of “infinite regress” (a la Jasanoff) but my point is a practical one — we can’t engage these issues absent a process. So rather that suggesting that such a process will eliminate such debates (it won’t) I am suggesting that it will instead bring them into the open which is where I think they should be.

2. You ask: “if the scientists participating in IPCC believe the back-end process worked and that the report fairly represents objective science (the statement of the twenty), even then, would there be cause for concern that the answers to the front-end questions have nonetheless been influenced in a nontrivial way by the participants’ social, political, or moral norms?”

My answer – Yes. Absolutely. Because the IPCC is explicitly designed to be “policy relevant” I think that it is absolutely fair and legitimate to ask the IPCC to reveal what criteria that it used to determine what science is relevant, and what is not, even within the front and back ends. For instance, how did the IPCC determine what survives from the full report into the SPM? I don’t know, and no one elese seems to either (it may have been ad hoc which I’d bet as most liklely), but it seems reasonable to want to know.

3. You ask: “How is your position different from the notion that there is objective science and that a suitable back end can facilitate using science to inform subjective, normative politics without itself overtly politicizing the science?”

I’m not sure that my position is any different. In my book I call this relationship “science arbitration” (it is also the model of science advice recommended by A. Dessler in his recent book). This model offers the best hope for science and politics to remain distinct (but it only works in some contexts, which I argue in my book). For it to work requires (among other things) a transparent and effect reconciliation of the demand for knowledge and its supply (see Sarewitz and Pielke 2007 linked above). The IPCC falls short of this because it has no systematic way to elicit the demand, so this is in effect black boxed. The IPCC does a good job characterizing the supply of knowledge. The lack of focus on the demand side is a problem in the implementation of the IPCC, and not in the model of science arbitration, which is a valuable one. (For practical consequences of the IPCC’s confusion of its role see Pielke 2005, “Misdefining Climate Change …”).

4. I don’t get the question in your last paragraph, can you restate?

Thanks!

On the back end, don’t you get into infinite regress on who gets to define legitimacy? You propose an “accepted process” without addressing who gets to veto it as unacceptable or who gets to rule on what’s a request for clarification and what’s interference with the science? The distinction is not always clear.

Jasanoff’s studies of the use of peer review adn other back-end processes in the EPA and FDA show that any back-end process almost inevitably becomes the ground for further disputes (boundary work) on who has the right to define acceptability. If you say peer review defines legitimate science, then the boundary wars move to the selection of the reviewers (Tom’s question gets at this).

I don’t think you address Andrew’s major question from the comments on the other post. Let’s leave Michael Mann’s inflammatory statement out for the moment and ask whether, if the scientists participating in IPCC believe the back-end process worked and that the report fairly represents objective science (the statement of the twenty), even then, would there be cause for concern that the answers to the front-end questions have nonetheless been influenced in a nontrivial way by the participants’ social, political, or moral norms?

Your post here seems to accept the notion that the scientists’ work is reasonably objective and only claim that at the back end, politics may have intervened and distorted the objective science produced in the middle. How is your position different from the notion that there is objective science and that a suitable back end can facilitate using science to inform subjective, normative politics without itself overtly politicizing the science?

You hint at something more when you write that “Many scientists do not like this assertion because it suggests that the IPCC is not accountable to anyone, and stands as a technocratic exercise far from any sort of democratic governance of science,” but you don’t help the reader with any sense of whom these scientists are or why they find democratic governance of scientific results (the back end) as important as democratic governance of scientific goals (the front end). I have my own ideas about this, but I’d like to learn yours.

http://www.heatisonline.org/contentserver/objecthandlers/index.cfm?id=3925&method=full

We don’t hear much about that now! In my view, the same logic I discussed above about the “double bind” applies to the selection of participants as to the contents of the report.

Thanks!