Comments on: The assessors assessing the assessments http://cstpr.colorado.edu/prometheus/?p=4138 Wed, 29 Jul 2009 22:36:51 -0600 http://wordpress.org/?v=2.9.1 hourly 1 By: Sylvia S Tognetti http://cstpr.colorado.edu/prometheus/?p=4138&cpage=1#comment-8496 Sylvia S Tognetti Mon, 12 Mar 2007 23:39:10 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=4138#comment-8496 I generally refer to this as a problem of "getting the right science." Unfortunately, many of the end users of national and global level assessments still think it is just about "getting the science right." And they are the ones that order up the science. The NAS is one of the few institutions that can produce a study that says the wrong questions are being asked, and still get paid for it. Ultimately it is an institutional problem, of getting science to support a place-based approach - an area where the natural sciences could learn a few lessons from the social sciences about dealing with complexity and still assuring quality of information. I will write more on this eventually but, I always wonder what the Millennium Ecosystem Assessment might have looked like had it been initiated by geographers, like Gilbert White... I generally refer to this as a problem of “getting the right science.” Unfortunately, many of the end users of national and global level assessments still think it is just about “getting the science right.” And they are the ones that order up the science. The NAS is one of the few institutions that can produce a study that says the wrong questions are being asked, and still get paid for it. Ultimately it is an institutional problem, of getting science to support a place-based approach – an area where the natural sciences could learn a few lessons from the social sciences about dealing with complexity and still assuring quality of information. I will write more on this eventually but, I always wonder what the Millennium Ecosystem Assessment might have looked like had it been initiated by geographers, like Gilbert White…

]]> By: Fergus Brown http://cstpr.colorado.edu/prometheus/?p=4138&cpage=1#comment-8495 Fergus Brown Thu, 08 Mar 2007 12:42:42 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=4138#comment-8495 I don't think it is controversial to say that this problem will always arise where there are two or more spheres of action (in this case, the scientific, academic, political, media and public), which are distinguished by both the methods and criteria of evaluation and by the language registers they use. What constitutes a successful communication act to a scientist is fundamentally different to the same for a policy maker, or almost any non-scientist. Likewise, a report writer might be satisfied that the report expressed what was intended, even when the consumer has misunderstood or misinterpreted the material. But all of the players in the science/policy/media debate are in a double-bind: make it too detailed, and your consumer may well lose the ability to make meaning; make it too simple, and you stand open to challenges of incompleteness, inaccuracy or unfairness, as well as the use of linguistic 'tricks' to take advatage of the different usages and meanings of language in different registers. And a clever 'player' in any one sphere of action can take advantage of the uncertainties inherent in language use difference to cast doubt on the veracity of statements where doubt probably doesn't really exist. if this is so, then the burden of responsibility - and the moral burden - for effective communication passes to the intermediary, the fortunate individual who can straddle the different spheres of action, manage the various language registers, and operate outside the confinement of evaluation by any one peer-group. For those whose place it is to make the meanings for the consumers, the imperative thus exists to offer up such interpretations as honestly, fairly and with as little prejudice as is possible within the bounds of their own personalities. You must say it as you see it and not be afraid to be challenged. Regards, I don’t think it is controversial to say that this problem will always arise where there are two or more spheres of action (in this case, the scientific, academic, political, media and public), which are distinguished by both the methods and criteria of evaluation and by the language registers they use.

What constitutes a successful communication act to a scientist is fundamentally different to the same for a policy maker, or almost any non-scientist. Likewise, a report writer might be satisfied that the report expressed what was intended, even when the consumer has misunderstood or misinterpreted the material.

But all of the players in the science/policy/media debate are in a double-bind: make it too detailed, and your consumer may well lose the ability to make meaning; make it too simple, and you stand open to challenges of incompleteness, inaccuracy or unfairness, as well as the use of linguistic ‘tricks’ to take advatage of the different usages and meanings of language in different registers. And a clever ‘player’ in any one sphere of action can take advantage of the uncertainties inherent in language use difference to cast doubt on the veracity of statements where doubt probably doesn’t really exist.

if this is so, then the burden of responsibility – and the moral burden – for effective communication passes to the intermediary, the fortunate individual who can straddle the different spheres of action, manage the various language registers, and operate outside the confinement of evaluation by any one peer-group. For those whose place it is to make the meanings for the consumers, the imperative thus exists to offer up such interpretations as honestly, fairly and with as little prejudice as is possible within the bounds of their own personalities. You must say it as you see it and not be afraid to be challenged.

Regards,

]]> By: Jim Angel http://cstpr.colorado.edu/prometheus/?p=4138&cpage=1#comment-8494 Jim Angel Wed, 07 Mar 2007 14:55:45 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=4138#comment-8494 This is the perennial challenge with all assessments. In 20 years, I can't point to any major assessment that was truly end-user driven and/or particularly useful to end-users. I work a lot with end-users on a daily basis and its challenging. Each one has specific needs, many of which aren't or can't be addressed by GCMs. For example, corn producers have about a two-week window in heat stress that is very critical, right at pollination. That window depends on corn variety and when it was planted. So you have a moving target and a focus on extreme events during a short time frame. A few years ago Stan Changnon did a study with a local utility to understand how they might use seasonal forecasting. You had to identify the decision-makers in the company, then you had to understand their operational needs, and then you had to try to match what you had available to their needs (and there is no guarantee that there is a match). It was very time consuming and that was only one company. So, it takes a lot of work and requires continuous two-way communication to keep the end-users involved. I don't think that happens too often. Jim This is the perennial challenge with all assessments. In 20 years, I can’t point to any major assessment that was truly end-user driven and/or particularly useful to end-users. I work a lot with end-users on a daily basis and its challenging. Each one has specific needs, many of which aren’t or can’t be addressed by GCMs. For example, corn producers have about a two-week window in heat stress that is very critical, right at pollination. That window depends on corn variety and when it was planted. So you have a moving target and a focus on extreme events during a short time frame.

A few years ago Stan Changnon did a study with a local utility to understand how they might use seasonal forecasting. You had to identify the decision-makers in the company, then you had to understand their operational needs, and then you had to try to match what you had available to their needs (and there is no guarantee that there is a match). It was very time consuming and that was only one company.

So, it takes a lot of work and requires continuous two-way communication to keep the end-users involved. I don’t think that happens too often.

Jim

]]> By: Jim Lebeau http://cstpr.colorado.edu/prometheus/?p=4138&cpage=1#comment-8493 Jim Lebeau Tue, 06 Mar 2007 20:15:52 +0000 http://sciencepolicy.colorado.edu/prometheusreborn/?p=4138#comment-8493 Quis custodiet ipsos custodes? Quis custodiet ipsos custodes?

]]>