The process behind the Mann “hockey stick” paper, featured by Al Gore in his movie “An Inconvienient Truth” has been damned by the US National Academy of Sciences Report. (the Wegman Report.) But how much difference should this make to belief in global warming?
The Wegman report addresses one of the questions asked by Barton’s office:
How central is the work of Drs. Mann, Bradley, and Hughes to the consensus on the temperature record?
Ans: MBH98/99 has been politicized by the IPCC and other public forums and
has generated an unfortunate level of consensus in the public and political sectors and has been accepted to a large extent as truth. Within the scholarly community and in certain conservative sectors of the popular press, there is at least some level of skepticism.
How could we answer this question quantitatively?
Consider the status of Anthropogenic Global Warming (AGW) claims of unanimous certainty by Naomi Oreskes
that “the scientific community believes that the Earth is warming and that human activities are the principal cause”.
Yet the IPCC claims AGW is “likely”. IPCC defines the terms â€œlikelyâ€ (meaning 64-90%) and â€œvery likelyâ€ (meaning >90%). Under this definition, AGW would not rise to the level of ‘significant’ in a scientific test based usually on a threshold of 95% levels of confidence. Classical science operates through such definitive high confidence beliefs. In this way the possibility of being wrong in a claim is reduced to very low probability, an event in the ‘long-tail’ of the probability distribution.
On listening to the general arguments in the Wegman hearing it seemed that the scientific consensus for anomalous warming of the 20th century comes by accumulating multiple lines of plausible evidence (e.g. glaciers, models, proxies, biota, etc).
Application of simple probability theory leads to some interesting observations about how multiple lines of low confidence evidence stack up against a single high confidence claim. Simple probability theory gives us a handle on it.
Claims that are ‘plausible’, that is 2-1 odds they are right, have about a 40% chance of being wrong. OTOH the scientific standard for significance is 5% chance an hypothesis is wrong, while 1% chance a claim is wrong is ‘highly significant’.
For comparison breadth vs depth of evidence, we estimate the probability that all the lines of plausible evidence are wrong. To do this we multiply 0.4 by itself according to the number of independent sources of evidence. The chance that a claim based on n multiple independent lines of evidence are incorrect will be 0.4n.
A quick calculation shows that it takes 4 independent lines of evidence to be equivalent to one claim with 5% significance, and 6 lines of independent evidence to be equivalent to one claim with 1% significance.
In quantifying claims, claims from multiple sources can be combined to give estimates of the real level of confidence in the composite whole. Then one could evaluate the claims against a level of confidence one requires to inform policy. If greater confidence is required, then an individual ‘plausible’ claim evaluated in more depth. If the claim falls apart under scrutiny, as was the case with the hockey stick, then the overall confidence in the theory must be adjusted accordingly.
So the answer is that demonstrating one line of plausible evidence is flawed, does not completely falsify the whole theory of anomalous warmth. Nor does it have no effect, as the removal of one line of evidence must reduce overall confidence in the conclusion. Partcularly if the AGW is at most ‘likely’ 90% level of confidence, changes in the probability of the component hypotheses can have a significant effect. Thus, expressions of excessive certainty in AGW would seem to be based on lower levels of certainty than are typically used in qunatitative experimental science and perhaps more indicative of a qualitative discipline such as history.
Controversial topics often swing opinions from one extreme or the other. Careful quantification of the impact of breaking the hockey stick on global warming using elementary probability suggests a more moderate approach. It also suggests that there would be benefit in implementing the Wegman recommendation that other lines of plausible evidence be audited by independent statisians, as these could also affect the ultimate level of confidence.