Comments on: Peer Review – Great Idea, but the Execution Needs Work http://cstpr.colorado.edu/prometheus/?p=5030 Wed, 29 Jul 2009 22:36:51 -0600 http://wordpress.org/?v=2.9.1 hourly 1 By: rephelan http://cstpr.colorado.edu/prometheus/?p=5030&cpage=1#comment-12754 rephelan Fri, 06 Mar 2009 09:26:46 +0000 http://sciencepolicy.colorado.edu/prometheus/?p=5030#comment-12754 I agree with the last sentence: "We can do better." I am becomingly less convinced that "peer review" is the path to "better". The "critical methodology" approach adopted by the social sciences has come to permeate scientific discourse in general: "all issues are political". Today's topic du jour in the climate science blogs, the rejection of the Paltridge et al paper, is one case in point. My old mentor, the late Nicholas C. Mullins, made me read Kuhn's "Structure of Scientific Revolutions", which suggests that science is not free of bias and agendas or even the cultural context in which science is "done". Max Weber, the great German Sociologist, gave an address called "Science as a Profession" - almost a century in the past but well worth the reading today. A dialogue between Weber and Crichton would be absolutely fascinating. In the 60's and 70's researching the latest information, thumbing through card catalogs, was a task of weeks. We may have more journals today, but the task of research can often be reduced to hours. (OK, there is a legititimate niche industry for researching precedents, but the author still has to review them!) There is less excuse in the 21st century for not knowing what has already been published and discussed. One of the fundamental bedrock values of science is transparency. Can it be replicated, are there alternative approaches? Some science blogs give us the impression that free-wheeling discussion of science is possible, but the "trolls" arguing for agendas (either political or paradigmatic) obscure the real issues. I would suggest that the peer-review system (along with the pay-to-view approach of most journals) is no longer functional. Too many scientists have adopted paradigmatic or political positions and will not be budged by evidence from them. Or have gotten too lazy to keep abreast of their literature (e.g.: "93 percent of original authors were not aware of the duplication"). The current peer-review process is too prone to orthodoxy. The current cultural climate is too prone to instrumentalism ("gee, professor, I took this course because I needed three social science credits at a convenient time ... why don't I have an "A"?)... some things are more important than fundamentals. We've been letting this go on for a generation. OK, I've just wiped the foam from my mouth and had my wife force water down my throat... the animal control officer may not need to put me down after all... science needs to be made more public, with tools to assist that, and a re-emphasis by the academy on just what academic integrity means... Michael Crichton would almost certainly agree with me that private resarch, proprietary knowledge, is not science. The issue is not how difficult it is to monitor one's field, it is a question of character and ethics. period. 'nuff said. I agree with the last sentence: “We can do better.” I am becomingly less convinced that “peer review” is the path to “better”. The “critical methodology” approach adopted by the social sciences has come to permeate scientific discourse in general: “all issues are political”. Today’s topic du jour in the climate science blogs, the rejection of the Paltridge et al paper, is one case in point. My old mentor, the late Nicholas C. Mullins, made me read Kuhn’s “Structure of Scientific Revolutions”, which suggests that science is not free of bias and agendas or even the cultural context in which science is “done”.

Max Weber, the great German Sociologist, gave an address called “Science as a Profession” – almost a century in the past but well worth the reading today. A dialogue between Weber and Crichton would be absolutely fascinating.

In the 60’s and 70’s researching the latest information, thumbing through card catalogs, was a task of weeks. We may have more journals today, but the task of research can often be reduced to hours. (OK, there is a legititimate niche industry for researching precedents, but the author still has to review them!) There is less excuse in the 21st century for not knowing what has already been published and discussed.

One of the fundamental bedrock values of science is transparency. Can it be replicated, are there alternative approaches? Some science blogs give us the impression that free-wheeling discussion of science is possible, but the “trolls” arguing for agendas (either political or paradigmatic) obscure the real issues. I would suggest that the peer-review system (along with the pay-to-view approach of most journals) is no longer functional. Too many scientists have adopted paradigmatic or political positions and will not be budged by evidence from them. Or have gotten too lazy to keep abreast of their literature (e.g.: “93 percent of original authors were not aware of the duplication”).

The current peer-review process is too prone to orthodoxy. The current cultural climate is too prone to instrumentalism (“gee, professor, I took this course because I needed three social science credits at a convenient time … why don’t I have an “A”?)… some things are more important than fundamentals. We’ve been letting this go on for a generation.

OK, I’ve just wiped the foam from my mouth and had my wife force water down my throat… the animal control officer may not need to put me down after all… science needs to be made more public, with tools to assist that, and a re-emphasis by the academy on just what academic integrity means… Michael Crichton would almost certainly agree with me that private resarch, proprietary knowledge, is not science.

The issue is not how difficult it is to monitor one’s field, it is a question of character and ethics. period. ’nuff said.

]]>