IPCC Fraud Solutions

Concern is often expressed about the way the IPCC has been conducted, and I want to suggest a constructive solution. Recently, a climate scientist was critical of freedom of information inquiries reported at ClimateAudit, but made some points that help to illustrate the errors in the current state of thinking. Below is my solution, after extracting quotes from a couple of Michael Tobis’ posts, made in a slightly different context.

Most established climatologists didn’t seek controversy in their career choice. The model of the culture as secretive, manipulative and selfish is not a good explanation of the facts.

Formal processes are intrinsically expensive, and they also reduce the attractiveness of the work, implying greater compensation.

Such a change will in fact attract different people as well as different methods. Something will be lost in the process, and an eye to preserving as much of the collegial culture as possible is also worth considering.

Am I saying, “we are in trouble, send money”? Not really. I don’t think climate science is first order important as of now, as the big picture is pretty clear. It’s those of you who don’t trust us who should be willing to invest in the matter.

I suggest recruiting people from other sciences who don’t have a dog in the hunt. But I’m afraid you’ll get the same answer you always do. The sensitivity to CO2 doubling on a century time scale is about 3 C. Sloppy methods or not we have this thing nailed. Now you can let us keep thinking about our angles and pins, or you can hire somebody to replicate our work.

But if you insist on your sport of sniping at our informality, if you insist that we become more formal, you need to invest a lot of money to train us and/or replace us, because we weren’t trained as MDs or pharmacologists or (a few exceptions like myself notwithstanding) as engineers.

The state of academic science is what it is for a number of reasons. Climatology is unexceptional except in having to deliver some very disconcerting news. You may argue that the nature of the news is such that climatology becomes higher stakes and needs to be reorganized and formalized. I have a great deal of sympathy with that position, and in that regard among others I’m an outlier within the field. Note, though, that such endeavors are expensive and prone to failure.

The ‘opus’ that exists, the response to a need for an organized presentation, is the IPCC WGI reports. For all its flaws, the IPCC consensus process and its reports are an interesting and useful achievement.

The network of trust on which human progress is based is badly frayed these days. I don’t think Climate Audit has made matters any better, but I understand that trust can;t be manufactured on demand. All I can do is state that I have complete confidence in the intellectual competence and moral integrity of those leading figures in the field I have been privileged to work with…

It’s a problem. People are demanding forms of “proof” that aren’t well suited to the problem area. Atmospheres are complicated and interesting beasts; atmosphere-ocean-ice systems (of which we have only one non-simulated instance) the more so. They aren’t unknowable, but predictions about large experiments on a specific system will always be contingent.

Suppose rather than sneering at what is wrong you make some suggestions as to how to set it right, what scale that would require, and who should pay for it.

That said, I believe that the concept of an outside audit is sound and I advocate one for the field of economics, so I can’t consistently argue against one for climatology. I’d be interested in constructive ideas as to how we could improve our credibility if our understanding is sound, or test our understanding if it isn’t.

In my view the UN IPCC report is simply a review of the literature, useful but unsystematic, incomplete and unremarkable apart for the hype surrounding it. Medical science conducts reviews all the time, and they have found that some guiding principles of Evidence Based Practice are essential:

1. The review must be systematic. The type of evidence to be used is explicitly stated by the commissioning agency and the procedures adhered to, with a view to minimizing personal bias.

1. The review must be without conflict of interest. It must be done by people with nothing to gain from the promotion of specific studies.

3. The review must pay particular attention to the relative quality of evidence contributed by each study.

In respect to the third point, a number of different systems have been set up. Adaptation of one of the most well known, from the Centre for Evidence-Based Medicine, Oxford, leads to a hierarchy of evidence something like the following, highest level first:

Level 1. Blind and randomized studies with publicly archived data and code.
Level 2. Comprehensive and independently tested studies with publicly archived data and code.
Level 3. Observational evidence and correlation studies with publicly archived data.
Level 4. Theory, models, and case studies.
Level 5. Expert opinion.

The application of such a system to climate science is not necessarily regulatory, but methodological for evaluating existing evidence in focussed systematic reviews. This would not be expensive, or require an overhaul or massive retraining of climate scientists as Michael Tobis fears. It would provide a positive incentive for studies to be structured in ways that have been proven to yield more reliable results, with less personal bias. It would help to build a climate of trust.

For example, a review commission might stipulate that all evidence is to be level 3 and above, requiring at least publicly archived data. This would eliminate a great deal of studies cited in the IPCC review. It would however provide a strong incentive for archiving data the next time around.

A blind trial of climate models need not be any more expensive than comparison trials as they are currently conducted. As an example, accuracy of a range of niche modeling methods were evaluated in a blind trial reported here.

It would be recognized that the IPCC is just another review, and an unstructured and biased one at that. Its main in-scope goal is to find a human influence on climate, and the range of reasons for climate change are out-of-scope, creating a systematic bias against natural explanations for climate change. This predeliction is clearly stated in its reward of a Nobel Peace Prize for:

“for their efforts to build up and disseminate greater knowledge about man-made climate change, and to lay the foundations for the measures that are needed to counteract such change”

So my solution is not one thats tend to one extreme or the other, from cries of “fraud” to blind acceptance of IPCC as gospel truth. The solution is just to keep it in perspective, and for those who are financially impacted by the implications to conduct their own structured reviews of key components of the case, and let these be a guide to their policy decisions.