Class Copenhagen Consensus Exercise: Feedback Requested
November 24th, 2006Posted by: Roger Pielke, Jr.
This Post Will Stay at the Top through 24 Nov, New Posts Will Still Appear Below
This semester in my graduate seminar Policy, Science, and the Environment we have spent a good share of the semester replicating and critiquing the Copenhagen Consensus exercise. With this post we’d like to solicit some feedback on the class term projects reporting and justifying their results
For those of you unfamiliar with the Copenhagen Consensus, its homepage describes its efforts as follows:
The Copenhagen Consensus Center (CCC) is a center under the auspices of the Copenhagen Business School. Through the commissioning and conveying of research, we work to improve the foundation for prioritizing between various efforts to mitigate the consequences of the World’s biggest challenges. In particular we focus on the international community’s effort to solve the World’s biggest challenges and how to do this in the most cost-efficient manner.
The idea is simple, yet often neglected. When financial resources are limited you need to prioritize your effort. Everyday, from policymakers to business leaders, at all levels, priorities are made between investing in one project and not another. However, many times, and particularly at the political level, decisions on priorities are made not based on facts, science or calculations but on which issue gets the most media coverage or is most politisized. The Copenhagen Consensus approach works to improve the foundation of knowledge, to get an overview of research and facts within a given problem, so that the prioritizing of efforts to solve this problem is based on evidence and is comparable with solutions across problems.
We are focusing on repeating the Copenhagen Consensus analysis in my class. This is the first time I’ve attempted this exercise, so this year I am very fortunate to have an extremely hard-working and thoughtful set of students in my class. Most importantly they have been extremely gracious in playing along as guinea pigs with a complete redesign of this course.
Here is what we’ve done. In our class we divided up into four groups – the Wolfpack, Troika, Great Danes, and the Savvy World Affairs Troubleshooters (SWAT). The first task for each group was to identify the two most important topics that were not on the Copenhagen Consensus list of 10 world problems. Each group presented two subjects and the students then voted among the recommendations to identify the two we would add to the list. We added Energy and Land Degradation.
The next task for the groups was to allocate $50 billion among the now 12 issue areas. Their assignment was to produce an allocation as well as a justification for their allocation. We have spent much of the semester focused on two tasks. One was learning from each other about the substantive issues involved with each problem area. The second was discussing the nature of cost-benefit analysis as a tool for producing information relevant to establishing priorities. We focused in particular on valuing human lives and discounting.
In producing their allocations and justifications, the groups were free to use whatever approach or method that they saw fit. We are posting the group reports here online to stimulate some feedback from our readers to the class on their reports. Note that the dictator professor disallowed efforts to spend the money over time or investing it in hopes of gaining a larger return, among other rules put in place to simplify and standardize the assignement. You will find a range of approaches to the allocation and a range of results.
Great Danes final report webpage
Troika final report webpage
SWAT final report webpage
Wolfpack final report webpage
Here is a spreadsheet summarizing the group allocations and comparing the class averages to three exercises run by the Copenhagen Consensus in 2004, 2006, and by the UN.
Whether or not you agree with their allocations, we would find useful any feedback on the group reports. How understandable are they? Are their arguments well supported and well justified? Are their reports credible?
It is a lot of work to read through the class projects, so we are grateful for whatever responses that our readers provide.
In closing, I wish to emphasize that in posting the reports the class is very interested in feedback but also wanted me to emphasize that many of the students are uncomfortable with the notion of cost-benefit analysis, and a few simply reject it as a legitimate basis for decision making altogether. We use the Copenhagen Consensus exercise as a pedagogical tool, not as an endorsement of the approach as a means of setting priorities. If I had $50 billion to spend, I certainly wouldn’t allocate it using the Copenhagen Consensus approach. Nonetheless, as an exercise for learning about global problems, the challenges of priority setting, and the difficulty of trade-offs, at least from the standpoint of the professor, the Copenhagen Consensus has some worthwhile qualities in the classroom. After the semester I’ll be happy to editorialize a bit more on the class and the Copenhagen Consensus, but for now I’d like the attention focused on the work of our students.
November 16th, 2006 at 2:24 am
I was disappointed that the CC didn’t include the “War on Terror” in their cost-benefit type analysis. So now I’m disappointed that you didn’t either.
November 16th, 2006 at 3:24 am
I’ll have a read… I remember doing this in university on health care. That was a real eye opener for many students at the time.
November 16th, 2006 at 8:06 am
Roger,
It would help to learn whether the group names are arbitrary (like football mascots) or are supposed to represent an organized viewpoint.
Also, by what date are you seeking feedback?
November 16th, 2006 at 8:14 am
Richard-
The group names were selected by the groups and are arbitrary.
We’ll be discussing any reactions to their reports in class November 28, so feedback before then would be much appreciated. Sooner is probably better than later.
Thanks!
November 16th, 2006 at 10:11 am
Hi Roger,
I have a couple of comments before even reading the reports:
1) Good stuff. Good idea.
2) I found it problematic in the Copenhagen Consensus that they looked at such broad and different areas (e.g. “conflicts,” “financial instability,” and “governance/corruption” versus “sanitation/clean water,” and “climate change”).
It seems to me it would have been better if the Copenhagen Consensus had focused only on environmental problems, or at least only on environmental problems and communicable dieseases.
I’d recommend your future classes focus more narrowly on the environment and human health (especially since your class name has “…the Environment” in it).
For example, I’d love to end “conflicts” and “poor governance/corruption”…but how does someone with an environmental background (like me) know anything about how much money spent will yield how much reduction in “conflicts” and “poor governance/corruption”?
So it would be good to compare things closer to apples versus other types of apples, rather than apples versus oranges.
November 16th, 2006 at 10:15 am
Thanks Mark- A few quick replies.
1. The point of the exercise is explicitly to make trade-offs across areas. Like in the real world. Feedback from past classes led me to this broader framing. It would be easier to constrain the focus, but that wasn’t my goal this term.
2. You ask: “but how does someone with an environmental background (like me) know anything about how much money spent will yield how much reduction in “conflicts” and “poor governance/corruption”?”
Good question!!
November 16th, 2006 at 1:59 pm
I don’t mean to rain on the parade of your benevolent world order, but have you looked into the question of whose money is being spent? Or is that a topic for next semester?
November 16th, 2006 at 2:22 pm
LL- A mythical $50B.
November 16th, 2006 at 2:27 pm
Frankly I’m suprised that trade barriers/subsidies aren’t higher on anyone’s list as they leave developing countries worse off and therefore with less of an ability to deal with all of the other problems listed. Having said that, I’m not sure how you could “spend” $50 billion to remove trade barriers/subsidies — although maybe the per diems for Washington lobbyists are higher than I think
.
November 16th, 2006 at 7:19 pm
Hi Roger,
One additional comment before reviewing the presentations (I’m not sure I can make it through all four):
“1. The point of the exercise is explicitly to make trade-offs across areas. Like in the real world.”
Actually, I’m not sure that is so much like the real world. The EPA works on Environmental stuff, Dept of Energy does energy stuff, and so on.
Even charities have specialties: Grameen Bank, Doctors Without Borders.
So I still think a more narrow focus would be better.
November 16th, 2006 at 7:19 pm
Environmental analyst Mark prefers to focus on technical issues rather than infrastructure/incentive issues. He asks: I’d love to end “conflicts” and “poor governance/corruption”…but how does someone with an environmental background (like me) know anything about how much money spent will yield how much reduction in “conflicts” and “poor governance/corruption”?
I would turn his question around and observe that most of the environmental and developmental problems in the third world result from poor governance/corruption and conflicts over resources. Unless these underlying issues are addressed, throwing money at supposedly discrete environmental/health issues may be simply throwing money away, as Ron Bailey noted recently in commenting on the meeting in Nairobi:
“Although the new funds would be devoted to projects to offset CO2 emissions, the experience of foreign aid over the past 50 years is sobering. During that period rich countries have spent more than $2.3 trillion dollars on aid and due largely the kleptocrats that have run many of the world’s poorest countries, their people are poorer than ever. Unless that changes, pouring money into Africa and other developing countries to offset carbon emissions will produce neither development nor actual reductions in carbon emissions.”
http://www.reason.com/news/show/116751.html
I find this Copenhagen-type analysis to be fatally flawed in that it fails to consider either HOW problems should be addressed and by WHOM.
November 16th, 2006 at 7:38 pm
Thanks Mark- I understand your point. It is not either/or, but both. In the real world there are indeed decisions like 302(b) appropriations allocations:
http://www.rules.house.gov/archives/RS20144.pdf
November 16th, 2006 at 7:41 pm
Comments on Great Dane:
First off, I’m a critical guy. Welcome to the real world, kiddies.
Overall, I got a fairly good idea how they made their decisions. (This is my first review, so I don’t have anything to compare with.) The website was slick, but navigation was confusing. I didn’t understand what the “Preliminary Presentations” had to do with the whole project. It seems to me those Preliminary Presentations should have gone in an Appendix, rather than the ranking system.
1) I didn’t understand the allocation system…why there was more funding for middle-ranked projects. It was briefly explained, but I’m a ‘zecutive (as Mickey Mantle used to say in his Brill Cream commercials…that probably dates me).
2) The Climate Change writeup had the phrase “green houses gases.” I’m all for spending money spraying green houses with gases, but still…
To paraphrase Jerry Seinfeld, “And you want my $50 billion!”
3) The Climate Change ended up with funding for specific communities for disaster preparation. That seems like it would be a “drop in the bucket” proposition. Tell them to fund my project to develop a storm surge protection system that can be deployed anywhere in the world. Much better bang for the buck.
4) The ranking system didn’t seem at all transparent…until I found the actual ranking table in Appendix 2. That’s a pretty important thing to be stuck in an appendix!
5) One of the reasons they didn’t use a cost-benefit analysis seemed to be because Copenhagen Concensus did, and they thought CC did it badly. Why not keep the cost-benefit, but do it well?
6) There were three subcategories for sanitation and clean water, but they weren’t even explained.
November 16th, 2006 at 7:41 pm
Mark-
It is like in the real world when every agencies propose the most alarmist report possible to gain the attention and a larger piece of the subsidies pie provided by the current administration.
Every year these agencies say that the situation will get better, but the conclusions always seems worst. Surprisingly they still need more money.
November 16th, 2006 at 8:34 pm
Dr Pielke,
Are the priorities set by the ammount of money spent to solve the issue or by which one should be solve to start with.
Some issue are more important than others although they require less money to be solved.
I give for example the SWAT group which ranked education 5th and yet they alloted it more money (I agree with them on that matter).
November 16th, 2006 at 8:50 pm
Sylvain-
Some groups indeed ranked priority for spending separately from the amount allocated. In class discussion, some explained that some high priority projects simply required less funding.
So yes please do keep that in mind when providing comments.
Thanks!!
November 16th, 2006 at 9:21 pm
Hi Roger,
I’m part way done through Troika. I won’t be able to finish tonight. (This is very hard slogging.
)
One comment I do have now that I’ve seen two rather than just one:
Both groups need to do a better job of presenting their methodology…Troika especially so. If one’s group is presenting the results of a multicompent numerical analysis, it has to be done in such a manner that a busy boss or a less-than-expert reader can follow what is being done.
One way to do this would be take one or two options (say the top-rated option, and the bottom-rated option), and walk the reader through how the numbers were estimated and crunched crunched out for those particular options.
For example, Great Dane’s evaluation table, while buried in the Appendix 2, at least has enough keys and explanations that I can follow what they’re doing. Troika’s spreadsheet is a total maze to me.
Troika and the rest of your class should understand that something as basic as the fact that Troika’s spreadsheet requires scrolling, rather than being visible on a single page, is a significant problem.
It may seem strange to them, but in the “real world” (at least the real one I’m currently in) such aspects as whether the final summary table is clear or muddy makes a huge difference. Lots of people can deliver pages and pages of obscure text…a single summary table may be all most of the audience ever sees.
Back to the real world,
Mark
November 17th, 2006 at 1:15 am
Hey Mark,
Welcome to university.
In my experience, students almost always skimp on their methodological descriptions- until they are required to reproduce a published study using only the description of the method in the paper (no advice, no lectures, no web). Practical learning roolz.
November 19th, 2006 at 12:16 am
Roger
This is a cost\benefit analysis. I have read your earlier paper (which was excellent!) about the problems when values judgments are used when making policy decisions. Isn’t cost benefit analysis a values judgment? What are the values and costs that are important? How do you reconcile the differences in what different groups consider are important?
For example the national parks need money to maintain the parks. Surveys of visitors of the parks show that they are willing to pay increased fees if it will go to the parks, but the park service will not increase fees because they want to make sure there is open access to the parks.
How do you reconcile the different groups differt value judgments? You can give a link to a paper you have written ( I’m sure it will be insightful
)
November 19th, 2006 at 5:13 am
A quick perusal of the Wofpack’s efforts, as nobody else has yet.
The most obvious limitation of this paper is the absence of the benefit part of the cost-benefit formula. The group is eager to invest in projects, but does not always clearly demonstrate the benefit of the investment or speculate as to a long-term outcome.
Another difficulty is that the group’s decision appear to be more subjective than analytical; this is why they end up with the majority of their allocation going to projects which match their initial mission statement. The implication of this is that there is something missing in the objective analysis and intercomparison of opportunities. As they state that their initial ranking was subjectively-based, perhaps the group needs to find a more dispassionate means of assessing priorities.
Part of the problem may lie in the pairing procedure, which did not seem to add a great deal to the evaluative process, and the initial value estimates, which were insufficiently varied. A glance at the appended tables shows too little differentiation between different options. The group needs to reconsider its underlying bias in relation to the range of opportunities, make estimates of real benefits and allocate a value to these, and take consideration of the long-term implications of their choices, such as the likely increase of population in marginal development areas due to improved disease control & hunger/food programs.
My conclusion overall is that the wolfpack, in spite of its name, has allowed its heart to rule its head, reaching the conclusions which are implied by its initial subjective evaluations, rather than looking coldly and ruthlessly at best opportunities.
My recommendation is to go back to the start and look at how you can evaluate the benefits of each opportunity.
I hope you find this useful and not too destrucitve; good luck!
November 19th, 2006 at 10:04 am
My comment on the presentation would be thanks those who used .PDF. It makes everything easier to find, print and read.
My comment on the project itself:
I support the SWAT group who prioritized project not by the amount of money allotted but by the urgency of the problem.
To spend money on some of those project without the people having access to food and water would be a huge waste of effort and time. A quick look at the Pyramid of Maslow tells us that only when people have fulfill sufficiently their physiological need can they focus on their security needs. In that views it is very important to solve the basic need of hunger, water and communicable disease.
The groups did a great job describing the problems but not such a great job at describing the benefit of their proposed action. I believe that action are worth as long as the benefit exceed the cost.
I hope that my comment were helpful.
November 23rd, 2006 at 2:18 am
Roger,
it could be an interesting excercise to allow the groups to negotiate between them and finally propose an all-groups priority list, as in real life..
eduardo
November 23rd, 2006 at 7:10 am
Thanks Eduardo- We did discuss something along these lines in class. While several of the groups had similar apporaches, most were pretty satisfied with their own group’s apporach. With more time, I’d be we could negotiate an outcome. In the summary chart I simply took the averages of the groups. Thanks!
November 25th, 2006 at 4:23 am
CRITICISM OF CC ON LOMBORG-ERRORS
Let me please draw your attention to the extensive criticism of the Copenhagen Consensus idea which may be read at the Lomborg-errors web site on the following address:
http://www.lomborg-errors.dk/CopCons.htm.
It would be nice to hear comments from the students to the text there. It may be printed as a word-document.
KFog.
November 27th, 2006 at 7:58 pm
Some final comments, starting with Troika:
1) Troika’s discount rate example for a windmill has the price of electricity increasing from 8 to 14 cents/kWh in 29 years. That’s very unrealistic. The trend in electricity prices (adjusted for inflation) has been downward, not upward. (And definitely not almost doubling in 29 years.)
2) The same discount rate example covers years 0 to 5, and years 25 to 29, but nothing in between. I realize it’s just an example, but how difficult would it be to use an example that includes all 29 years?
3) The energy document says the world will “need” a “global investment” of $16 trillion in energy development. Well, what happens when the world doesn’t get that “need” met? The end of the world as we know it?
4) The windmill example says that a 500 amp wind power station (presumably at 110 volts, so 50 kW) could be built for a “$1000 kit” if done “on an industrial scale with automotive components”. But that is obviously ridiculously low. A much better estimate would be about $2000 per PEAK kilowatt, or $100,000 for a 500 amp PEAK installation. And such an installation would typically have a capacity factor of only about 33%…so it would be $100,000 for an *average* output of about 17 kW.
Therefore, the estimate is off by approximately a factor of 300(!). Even for non-technical students, graduate students shouldn’t be making such huge mistakes. If the world really could get $50 kW continuously for a capital cost of $1000, there wouldn’t be any energy problem anywhere. We certainly wouldn’t have coal-fired and nuclear-fission power plants if wind was that inexpensive.
November 27th, 2006 at 8:04 pm
Oh, one final comment on Troika:
“We will spend $1.5 billion to support planning for local amelioration and adaptation projects. We will fund studies to show where coffer dams can protect population centers from rising sea levels and violent storm surges…”
Sounds sort of like water tubes. If so….A++++++!
November 27th, 2006 at 8:22 pm
Comments on SWAT:
1) Overall, the website of SWAT is by far the “cleanest” and easiest to understand. The column with the issues listed on the left is good, as is the the “click for more information” after each summary.
2) SWAT is totally devoid of calculations that help to explain why specific amounts were allocated to each category. It appears that SWAT was one of the groups that doesn’t “believe in” cost/benefit analysis. Well, welcome to the real world, kiddies.
3) It’s disappointing (to me) to see the lack of willingness of SWAT to consider funding organizations outside the U.N. Everything is World Bank, UNICEF, UNESCO, etc. For example, (Nobel-prize-winning) Grameen Bank, in lieu of the World Bank?
November 27th, 2006 at 8:59 pm
Comments on Wolfpack:
1) Good animated wolf.
2) I wonder about the decision to limit the maximum amount to any one crisis to $8 billion. (Note: Page 11 of 22 of the paper mistakenly says, “$8 million.”)
3) The ranking system is clear…and I like rankings (rather than SWAT’s lack of numerical rankings). But in some cases, the rankings don’t make too much sense. For example, how is it that climate change rates a “2″ on Direct Life-Saving Capacity? How many people are directly killed by climate change each year?
4) There is a reference to a cost/benefit analysis. But I don’t see the cost/benefit analysis calculations for each crisis…? In an appendix that didn’t make it to the website?