Invitation to McIntyre and Mann – So What?
October 31st, 2005Posted by: Roger Pielke, Jr.
Over the weekend I read with mild amusement and increasing frustration the latest exchange in the “hockey stick” battle. Given that the “hockey stick” has occupied the attention of climate researchers in professional journals, been discussed on the pages of leading newspapers, and been the subject of a Congressional investigation, you might think that the latest exchange would be over something important. If you happen to be thinking along these lines, you’d be wrong.
Increasingly the back-and-forth over hockey sticks is beginning to look like a testosterone-fueled fight between different cliques of pimple-faced junior high school boys, egged on by a loud group of close observers who for various reasons want to see a brawl. And just like those boys on the playground, these guys are too wrapped up in their own vanity to see that they are making us all look bad, and are risking having our recess cancelled. From my perspective — that is, one focused on decisions about climate policies or climate science — the continuing pettiness of the debate on the “hockey stick” suggests that the time might be appropriate for the participants to explain to the rest of us why we should care about their continuing smarminess, else they should all be sent to detention where they can continue their bickering while the rest of us stay on the playground.
So what was the latest exchange about anyway? Given the attention devoted to this subject, perhaps it was about the future of world’s energy policies? Perhaps it was about policies for data archiving and peer review of climate research? Maybe even the technical details of using bristlecone pines in paleoclimate temperature reconstructions? The role of politics in climate science? Nope, none of these things. It was about who gets to post comments on whose website. To get a sense of this juvenile exchange, see this post and comments at ClimateAudit and this post and comments at RealClimate. You’ll see a lot of finger pointing, chest-thumping and pique over claims back-and-forth over issues like “dishonesty” and deeply wounded egos. And the folks chiming in from the sidelines are just having a ball.
I have written previously here that “I think that the debate over the so-called “hockey stick” temperature reconstruction is a distraction from the development and promulgation of climate policy.” But perhaps I am missing something. So I’d like to issue an invitation to both Steve McIntyre and Michael Mann (or another member of his team) to each prepare a short commentary (<1,500 words) which we will post here at Prometheus explaining why those of us not involved in the hockey stick fight should care about the continuing battles. Let me be clear, I am not asking why each of *you* are engaged in this battle. And we don’t want to hear about any of the technical issues involved, all of that stuff is readily available. I want to know why *others* should continue to care about it, from your perspective. Why is this fight important to the rest of us?
We will publish both pieces on November 14, 2005, which means we’ll need your pieces by November 11, 2005. If you would like to participate but would like more time, simply email me (pielke@colorado.edu) saying as much and we’ll extend the deadline. If you don’t respond (and so there is no doubt that you’ll each get this, as I’ll also email this invitation to each of you), we’ll take that as a clear indication that there is not in fact a compelling reason that you can offer for us outsiders to care about this debate. And given the way that you’ve been spending your collective time of late, a response of “too busy” or “over-committed” just won’t fly, particularly given our willingness to extend the deadline in a way that suits your schedules.
Please view this as an opportunity to clearly explain the significance of this debate that you’ve both invested so heavily in. And please apply prisoner’s dilemma logic to our request, and assume that the other has accepted our invitation. I am sure that there are others who are looking forward to each of you engaging on this issue – hockey sticks, so what?
October 31st, 2005 at 3:28 pm
Well Roger, if you wanted to put peoples backs up “pimple-faced junior high
school boys” was a good place to start. As for the offer… I don’t speak for Mann or anyone else, but I think you’ve misunderstood. Several comments from us on the recent RC post have indeed been of the “this is all a distraction” type.
October 31st, 2005 at 5:04 pm
Thanks William for the reponse. For what it is worth, Steve McIntyre wasn’t too happy with the analogy either. But all you guys should know that I’ve also received some positive responses about the characterization. Apparently I’m not alone in how I see your latest spat.
The offer still stands for Micheal Mann, or someone else from your group, to prepare a essay addressing the “so what?” question. I hope that you’ll take us up on it, as I am sure that a lot of people would be interested in seeing what you have to say.
October 31st, 2005 at 6:53 pm
I understand your frustration with the debate, though your challenge and even your initial assumption are misguided. The discussion you picked upon from a moral high ground was about openness of the process. That seems worth bickering over.
There is a better challange to issue though that would eliminate a great deal of the discussion and/or bickering. Eight years after Mann’s work, articles and replies are being published arguing over how the hockey stick was created. The challenge you should issue is to Mann. Challenge Mann to publish the data and the process — in detail. Sufficiently detailed that von Storch and McIntyre aren’t still guessing eight years from now on what process he used. Write a cook book if need be, but get the information out. Talk about childish: the fate of the world hangs in the balance and Mann won’t provide the information required to duplicate his work. The scientific method is all about repeatability. If the results can be replicated then many criticisms will be laid to rest quickly and neatly. Then real issues can be debated. Either way the science will improve when the data and process are open.
October 31st, 2005 at 6:57 pm
At the end of the day, the debate over statistical methods and temperature reconstructions is an interesting science debate. It doesn’t change the nature of the science questions what testosterone or emotionalization of the debate is happening from either side (and the way you raise your topic here is a part of emphasizing the “struggle” of opposing camps rather then the debate of opposing views.)
On the does it matter: I think yes. If the hockey stick is correct, it is a significant result. Showing that temp changes recently are abnormal in persepective to history. Thus helping to build a case for a view that CO2 is warming things up. (And if not, then the lack of such a case.)
Also, the issue of efficacy of research methods is also more significant than you think. Remember reading Feynman’s essay about cargo cult science and the guy who didn’t get famous on rat running experiments but who figured out all the relevant controls to actually make those experiments worthwhile?
October 31st, 2005 at 9:26 pm
IMO, the issue is not the 98 paper any more, but the proxy reconstructions that have followed, which is why the parsing of minutia from 8 years ago is an irrelevant distraction except as it moves the discussion of methods forward. That is why I liked Huyber’s paper.
A lot of the attack on Mann, Bradley and Hughes does not confront the fact that it agreed well with even contemporanious reconstructions such as Jones and Biffra and Crowley and Lowery.
Since then, if you look in detail, there have been some reconstructions with higher variability (Moberg, Huang, Esper, etc.) but higher variability in response to forcings only makes one more concerned with increasing greenhouse forcing.
As far as I can see all of the reconstructions show the late 20th century rise to be unprecidented in the last 500, 1000 and even 2000 years, where, of course everything beyond ~ 150 – 200 years are proxys.
http://en.wikipedia.org/wiki/Temperature_record.
November 1st, 2005 at 2:45 am
If the the temperatures in the past were more variable then something must have caused the variation.The something must be variation of the sun.
In Crowley’s analysis of the “Hockey Stick” he assigns factors for (among other things) the effects of “Insolation” and “Greenhouse Gases”(mainly CO2) on Global Temperature.
If the factor for the “Insolation” is increased to account for a greater variation of temperature in the past then then factor for “Greenhouse Gases” must be reduced to account for the present variation in the “Insolation”.
If this exercise is carried out the effect of CO2 on temperature is reduced to a miniscule amount.
That is why this argument matters.
One could say that the future of of mankind depends on the which viewpoint prevails.
November 1st, 2005 at 3:18 am
Hi Roger,
Speaking for myself, I do understand your frustration. I must admit that I also do not see why all this fuzz has been made about the shape of the principal component (PC) – or the so-called ‘hockey stick shape’. It is important to keep in mind that this has only involved one stage of the analysis, as it is followed by a regression that gives the various principal components appropriate weights. The importants questions, the way I see it, should be how many principal components were involved in the regression analysis and what fraction of variance do these account for. Furthermore, what skill does an independent validation give the predictions? Whether the leading PC has a bananashape or is flat, doesn’t really matter, as the regression analysis should find the best weighted combination of PCs to match the northern hemispheric temperature over the calibration period. Furthermore, it is also too easy to get a wrong impression about the ‘hockey-stick’ shape, as most of the recent up-turn in the curve is due to modern instrumental observations, and hence not part of the principal component analysis. So, I don’t either understand all this fuzz that Steve makes about the shape of the leading PC. I think that it is true, as other independent scholars also have concluded, that this really is of minor importance.
Rasmus
November 1st, 2005 at 10:19 am
Roger, I think a lot more is going on here than your take would suggest. First we have the problem that if one is not committed for or against AGW, but trying to keep an open mind and follow all sides of the “science” one is automatically an AGW skeptic, and second the arguments are being developed on the internet, because the peer reviewed scientific journal process is too slow. Worse the hockey stick debate is even casting into question the issue of peer review, at least in the field of climatology, where the process seems to be much less rigorous than in eg pharmacology. Given these problems, we need some web sites that try to keep a balance with regard to the arguments, in that they present good arguments contrary to the basic view they espouse, and they try not to present spurious arguments in support of their position.. One need only spend a short time at RealClimate to observe that they are in fact a site for highly selective advocacy rather than one of balanced discussion, in spite of the fact thet they claim to be scientific. It is important that this posture be illuminated so that the less critical observer can beware, or perhaps the practise can be changed. However this is not the issue that you chose to address. Rather you jumped to the value of the hockey stick argument itself. The specific issue of the hockey stick has now become symbolic of a couple of much larger and more important issues as has been noted above. The first is full disclosure of data and methods to enable replication. The second is the whole issue of the use of proxies to demonstrate “unprecedentd” climate change (specifically global warming)in recent decades. If our recent experience is not global, or is not unprecedented, then the issue of anthropogenic cause is cast into question, and that is a very important issue. What MacIntyre has done is to show that the proxy reconstructions presently used to demonstrate “unprecedented” are the result of clear selectivity in the choice of data, questionable correlations, doubtful methods of statistical analysis and even evidence that the proxies do not represent the changes they are being used to illustrate. It is very easy for those not deeply versed in statistical analysis, or not disposed to critical thinking, to drink the resulting koolaid, so MacIntyre is providing a very valuable check on runaway advocacy that the scientific community needs. Let me illustrate with the comment from Rasmus just above. What he seems to have failed to recognize is the simplest of the issues at hand. The hockey stick is the result of tacking the instrumental record on to the end of a centuries long proxy analysis, without any evidence that the match where they are joined is valid, and without confirmation by bringing the proxy record up to the present, even though data is available. Even if the proxy analysis, as far as it goes, were valid this failing invalidates the resulting picture. Let’s address the real issues, which are biased advovacy in the guise of science, failure of full disclosure, selective analysis presented as scientific proof, inadequate methodology, and refusal to examine sound alternative theories/evidence/analysis such as solar forcing. Murray
November 1st, 2005 at 10:29 am
Roger’s just upset cos no-one is talking about hurricanes for once
I look forward to yawning at – I mean reading the resulting articles. I have to say I agree with his basic premise, although I wouldn’t have used quite the same endearing terms. Their behaviour doesn’t seem so far removed from that of certain hurricane researchers, either…
November 1st, 2005 at 2:13 pm
Fret no more. The answer is here.
November 1st, 2005 at 2:14 pm
Oh great. The silly software eats hrefs. The answer is http://mustelid.blogspot.com/2005/11/big-picture.html
November 1st, 2005 at 2:33 pm
That’s all well and good, William, but it doesn’t explain why RC has had a remarkable 10 posts (at least) directly focussing on this niche of climate science that doesn’t matter
November 2nd, 2005 at 2:13 am
I think part of the issue is that there is a difference between science and accounting (the name ‘climateaudit’ is a bit of a give-away…): whereas the former concerns seeking universial truths, the latter is about abuse or errors in a particular case. When these objectives get mixed up, I think your’re on a wild goose chase. What is crucial is that past results are replicable and that the method is transparent. As William showed, a number of other scientists have managed to produce similar results as the hockey stick, using slightly different approaches (the data has been available). This supports the notion that this is of more universial character. Scientific results must be robust, and shouldn’t hinge on a particular set-up of piece of code. Conversely, the accounts of a particular company – the way I perceive it – depend on each steps in particular (I’m not an accountant, so please correct me if I’m wrong); the numbers are indeed sensitive to the local choices of how things have been done.
It’s ironical that the critisism of the hockey stick can be taken as a critisism of the science associated with increased greenhouse effect – the hockeystick doesn’t really show the warming, but rather a slow decline in temperatures over a long time, until an enhanced greenhouse effect kicks in. One interesting aspect of this study was that it reproduced the predictions based on Earths orbital parameters (Milankovich), which also suggested a very slow and gradual change in the orbit that eventually should lead to a gradual gooling – if humans had not interferred. The enhanced greenhouse effect has been dealt with in two IPCC reports prior to the publication of the hockey stick, and thus an enhanced greenhouse effect is not based on the hockey stick.
Thus, I hope that these views and my earlier comments illustrate why I think it’s strange that the hockey stick has made such a big fuzz. I do, however, understand the need to defend the curve, as it has been critisised for what I see as wrong reasons by individuals who have convinced me that they have misunderstood some central aspects about analytical methods (eg see http://www.realclimate.org/index.php?p=41).
November 2nd, 2005 at 7:47 am
To pick one point from Rasmus Nov 1 at 3:18. Rasmus criticizes over-fussing about PC calculations and suggests that a better question is:
Furthermore, what skill does an independent validation give the predictions?
That he should even pose this is very frustrating as we spend a lot of time in our GRL on this very topic. We stated that the MBH98 results failed important statistical cross-validation tests (such as the R2 test, or CE test for that matter) and argued that the seemingly high RE statistics was “spurious” – hence the title: Hockey Sticks,Principal Components and Spurious Significance.
The discussion of spurious significance of the RE statistic was an essential part of our part. We showed that the pattern of high RE statistics and negligible R2 statistics could be obtained by red noise modeling. In our first GRL paper, we simply showed the existence of spurious RE statistics in such circumstances, but did not carry out a full-blown MBH simulation, as there were then still uncertainties about exact MBH details that we did not want speculations about nuances of MBH methodology to distract from our point. Since then, as a result of the Barton Committee, more details about MBH methodology have emerged.
Huybers criticized us for not fully emulating all MBH re-scaling details in his Comment, but then failed to emulate MBH methods himself. In our Reply to Huybers, we extend our previous results, showing that the spurious RE statistics arise under MBH emulation.
Since there is a total failure in cross-validation R2, one may conclude that the MBH index cannot be used to reconstruct past temperature with any confidence. So I think that we’ve answered Rasmus’ question.
Rasmus made a snippy personal comment about competence, linking to a paper by Michaels and McKitrick. While there is a coauthor in common between the two papers, I had nothing to do with that paper. There are goofy methods used in Hughes and Diaz [1994], but I don’t hold that against Mann, Bradley and Hughes.
November 2nd, 2005 at 5:56 pm
Dear Roger
I can understand your amusement at the comments on the blogs, but there is a serious side to it.
Realclimate portrays itself as ” a commentary site on climate science by working climate scientists”, but the reality is that the site shuts down scientific discussion, shamelessly censors dissenting views and makes personal attacks.
Their recent post on “Hockey sticks: Round 27″ takes a partisan view on its subject matter, which are comments on the paper of M&M, and provides an excellent example. When Steve McIntyre (of M&M) posted a reply to this, addressing the science, it was not posted, while several other posts were. It wasn’t till climateaudit made an extremely public exposure of this “posting approval” policy, that realclimate suddenly found the time to approve McIntyre’s post.
The bottom line is that realclimate cannot claim that it is a science site, when it only allows scientific discussion on one side of a scientific argument.
yours
per
November 2nd, 2005 at 6:45 pm
Per- Thanks for your comments. I had a back-and-forth with the RC folks on whether their focus is science or politics. Have a look here:
http://sciencepolicy.colorado.edu/prometheus/archives/climate_change/000316the_uncertainty_trap.html
http://sciencepolicy.colorado.edu/prometheus/archives/climate_change/000317a_response_to_realcl.html
November 2nd, 2005 at 9:50 pm
Roger, I had a look at some of your older comments linked above. I think you’ll agree than RC has considerably broadened the scope of its posts from the first couple of months. Regarding the posting policy at RC, perhaps we won’t agree on this, but because RC’s major purpose is to serve as an educational resource for the public, media and policy-makers with regard to climate science, it wouldn’t be a very good idea to let the trolls run amok. Actually the RC authors tend to be quite a bit more liberal in this regard than I would be.
As to the importance of the fight over the hockey stick, I don’t think it is especially, and I don’t think it makes sense for Mike to take up your offer. That said, whenever there is activity on it that acquires a high public profile (as with the recent exchange in GRL), RC needs to update the issue. If in the future they do a proactive hockey stick post, I’d be real surprised.
November 2nd, 2005 at 10:52 pm
Here is a post that realclimate refused to post. This is a precise on-topic reply to an issue raised in the post. Other similar posts have been rejected.
“A point about Huybers’ Comment, to which we had a very precise Reply not discussed in the head post.
“In our GRL article, we pointed out that the MBH reconstruction failed important cross-validation tests (such as the R2 test, but not only the R2 test) and that these failures were unreported.
“There is no benchmark theory for RE significance. We argued that the seemingly high RE statistic in MBH98 was “spurious” – using the term “spurious” in the statistical sense of Granger and Newbold [1974] and Phillips [1986], not in an argumentative sense and showed that the biased PC1s could be used to create “reconstructions” which had high RE and low R2 statistics.
“Huybers argued that this model of a spurious RE mechanism did not replicate a re-scaling procedure, now known to be in MBH98 from the source code release. Huybers did some new simulations, purporting to show that the benchmark should be a low one, rather than the high benchmark that we had proposed.
“In our Reply, we pointed out that Huybers had not implemented other important aspects of MBH procedure. We re-did our RE simulations, applying information from the source code, and once again obtained a high RE benchmark.
“The most important point is the failure of the cross-validation R2 statistic. We have never argued (contrary to some characterizations of our work) that the cross-validation R2 statistic is sufficient for statistical significance; however, we do argue that it is necessary.”
This is not “trolls running amok”. This is the “serious discussion” that realclimate says that it is intended to foster, but seldom does. Surely it’s reasonable to question the honesty of the realclimate commitment to “serious discussion” without invoking snide comments. Aside from the censorship, simple equity demands that anyone being criticized be given an opportunity to respond – a policy practiced here at Prometheus and at climateaudit, but not at realclimate.
November 3rd, 2005 at 1:32 am
If RC feels that discussion of nuance and debate on open issues distracts from their mission of public outreach, then they should just be upfront about it in their posting policy. Just don’t say that debate is allowed, if it’s supressed.
November 3rd, 2005 at 8:34 am
P.s. Just so noone thinks I am purely anti-hockey team, I was on their side in defending the principle of conservation of momentum in the inelastic collision cockup with you or your old man. I bet you can get Lumo to stick up for them on that also.
Pss. glad that there is some person who can bridge the gap between the camps. However, I see most of it as a form of haughtiness from the stickers which is not justified based on their math abilities versus Steve. Also, I share more the take of the guy in the anti-creationist blogosphere who said “you lose!” when he was prepared to back Mann as expert but found out that the sticker response was based on refusing to share methods. I think you should move into this view of the stickers too. Enough time has gone by.
November 3rd, 2005 at 2:44 pm
I agree Steve M.’s rejected RC post above isn’t trolling in the usual sense, but it does have another problem that makes it inappropriate: Anybody who doesn’t understand the statistics involved, which I would submit is nearly everyone likely to visit RC, will be completely lost in trying to follow such arguments. I think it’s a fair statement that RC tries to keep things on a less technical level, and a couple of their own posts have even been re-written with this perspective in mind. Anybody who wants to see the technical stuff will have no problem finding it.
My big-picture perspective, FWIW: Multi-proxy climate reconstructions are difficult and necessarily involve some degree of interpretation. The fact that arguments arise about any particular study is thus not surprising. Indeed, the fact that there have been multiple additional reconstructions subsequent to MBH, and that this work continues, is proof that nobody in the field believes that any of these studies are perfect or definitive. That said, the fact that all of the studies draw broadly similar conclusions seems significant, and in my semi-informed amateur opinion conclusive.
Steve M.’s work has gotten vastly more traction than it deserves simply because MBH was identified by the skeptics/contrarians as the “weak spot” in the TAR. If it hadn’t been MBH, it would have been something else. As is very obvious from some of the comments above and elsewhere, the attacks on MBH are simply a stalking horse for those who are predisposed to want to believe that the current warming is a result of solar forcing (or natural cycles, or whatever).
It’s important to not forget why the IPCC decided to feature MBH: That nice flat shaft punctuated by the sharply rising blade was graphically effective with the public, the media and policy-makers, even though the flatness itself never meant much in a broad sense. If the result had been a bit bumpier (as with one of the more recent reconstructions), it still would have been featured, although perhaps less prominently. In any case, being featured at all inevitably meant attack.
Bringing things back to the present, Steve M. finally got the scientific hearing he’s been clamoring for (probably as a consequence of the crap from Barton and the WSJ earlier in the year), and it didn’t work out to his liking. Now he has resorted to attacking the entire field of paleoclimatology. I suspect I know what sort of reception he’ll get.
November 3rd, 2005 at 6:50 pm
Steve B, your suggestion that the comment is too “technical” is a joke. They say that they want “serious discussion”. Now you say a serious comment is too technical. LOL. ROTFLOL. You guys are just too funny.
The Comments by von Storch and Huybers were submitted long before Barton. Both Comments have serious flaws, which realclimate doesn’t want to hear about. I’m not in the least worried about the final outcome of these studies, if anyone reads the Replies as well. What is pathetic is the excessive readiness of so many climate scientists to grasp at such straws.
As to discussing other studies, this is something that I’ve been working on for a long time and have been posting about all year at climateaudit. I wrote about MBH98 initially because it was the most prominent study. People have said that errors in MBH98 don’t “matter” because the various other studies confirm it. People have challenged me on this for some time. I would like to have been further along in these studies by now, but each one is time-consuming because of the poor traditions in the discipline of archiving data and methods, making each analysis take more time than should be required. Any close examination of these other studies shows many problems with each study individually. The fact that this has not been observed publicly to date is simply because no one has gone through these studies carefully.
Your suggestions as to my motives are completely off-base. I became interested in MBH because it seemed promotional (as you point out) and I was simply interested in how the promotion worked. I did it entirely for personal interest without any plans or expectation of writing academic papers.
I am not personally advocating any alternative theory. At some point in the future, I might, but, for now, I’m trying to completely analyze the existing offerings – something that no one else seems to have done. I would have thought that this would be viewed as a worthwhile undertaking by people in the field, rather than being resented.
I am absolutely not attacking the “entire field of paleoclimatology”. I think that there are many substantial and valuable studies in the field and I find the discipline very interesting. I happen to be very unimpressed with the writings of one small subgroup in the field – sometimes known as the Hockey Team – and it is grandiose to identify their output as being equivalent to the entire discipline.
November 3rd, 2005 at 9:42 pm
Hi Roger,
I think your challenge is a good one. One can go to the Barton website (http://energycommerce.house.gov/108/Letters/06232005_1570.htm#Related) and read in the letters to the scientists why they think folks in general should be interested in the issue. If the reasons stated are genuine, you’d think they would post the replies to those letters. Oddly those replies aren’t presented. (Thankfully all of the climate blogs I’ve visited are better at presenting both sides than that!)
Why should regular people care? The answer is that they shouldn’t — they should only care that the bickering be allowed to continue in a constructive manner until some possible resolution is reached. Next question: how can the back-and-forth be done more constructively? Sadly there are so many ways…. That’s one regular Joe outsider’s perspective, anyway.
November 4th, 2005 at 2:25 am
Come on over to climate audit. debating efficacy of statistics is not bicker in my book. And I’ve held Steve’s ass to the fire a bit…
November 4th, 2005 at 2:26 am
M that is