Risky Statistical Prediction Methods

A couple of days ago, Luke, a frequent commenter, sent in a number of links to a new Australian Government drought initiative. The Minister Tony Burke has appointed an Expert Panel to examine the social impacts of drought as part of its national review of exceptional circumstances (EC) funding, which argues for a major change, based on incentives rather than emergency aid. In a recent speech, Peter Kenny, chair of an expert panel looking at the social impact of drought said of the Drought Exceptional Circumstances Report (DECR):

The Bureau of Meteorology and the CSIRO predicted there was an increased risk of hotter and dryer seasons over the next 20 to 30 years, compared to the last hundred years, across many parts of Australia.

The same sort of restrained language is shown in the various news articles linked below. This is a far cry from the lurid claims of imminent drought apocalypse, encouraged by unvalidated (and completely inaccurate historically) climate model simulations produced by the DECR. After wringing the data out of them with the support of numerous blogs, I wrote a review showing that the frequency and severity of drought had actually declined over the whole of Australia, while the climate models show an increasing trend. This simple observation was not reported in the DECR. Contrary to the message in the report, the lead author later said in an interview that “a long term trend its not very clear in terms of exceptional low rainfall years.” CSIRO have been ‘statuesque’ in defense of their report. While the director promised a reply to my critique on Sept 16, I have yet to receive anything. In legal circles, no reply within 30 days can trigger a tacit admission that the accusations are true.

I have no issue at all with the way Peter Kenny seems to be reporting the CSIRO findings, and the latest report validates what I had always suspected about the political motivations for the dodgy statistical prediction methods in the DECR. In my post Scientists Biasing Research I proposed:

Could it be that climate scientists are biasing the detrimental effects of manmade global climate change to suit the review of EC funding by the Rudd government?

You have to wonder why people are listening to climate code red models when we know they are inherently flawed and useless at prediction. This speaks to the credibility of the media and the scientists involved. The skeptical bloggers seem to have it right and mainstream experts and the media have totally got this wrong.

The similarities with the sub-prime financial crisis are amazing. We don’t have to look far to find numerous financial bloggers warning about the dangers of unrestrained credit expansion while the mainstream economists and fund managers have been completely wrong-footed. They were probably using a lot of risky statistical prediction methods too. A recent paper, Forecasting the Depression: Harvard Versus Yale, tried a range of modern models at predicting the Great Depression, and finds that both the Harvard and Yale forecasters were systematically too optimistic. It seems that statistical models have not improved forecasting skill.

In the latest fiasco, The Federal Government initiated a bank deposit guarantee plan, and by Friday of last week 13 of the top 20 funds in Australia had frozen redemptions to investors to stop money flowing out of their businesses. I tend to agree a bit with the farewell letter of resignation from hedge fund manager Andrew Lahde, that some people are truly not worthy of the education they received.

My opinion based on many years in modeling, is that the mainstream scientists exaggerate the predicted global warming effects on purpose. They have vested interests, as bad news sells science. Governments get to blame their present land management mistakes on something in the future outside their control. This is why validation is so important and statistical prediction methods need to be way down the feeding chain on sources of evidence.

Who does this remind you of?

Below are a set of links to the recent policy initiative on drought management.

Wiki: The new climate theory of Dr. Ferenc Miskolczi

I have converted the draft of the introductory document The new climate theory of Dr. Ferenc Miskolczi by Dr. Noor van Andel into a Wiki. The permissions are set for registered users of the Wiki to freely edit it. There are a great deal of areas where it could be improved and added too, and an opportunity to learn more about Wikis.

Register here to edit.

Wiki Syntax is here.

Continue reading

Linear Regression R Squared

One of the tests of climate models predicting drought in my review of the Drought Exceptional Circumstances Report was the correlation of predicted area under drought with actual observed area under drought. Lazar criticized my inclusion of the R-Squared (r2) coefficient, an issue I didn’t follow up at the time.

… correlating model predictions for individual years of exceptional rainfall with observed years of exceptional rainfall! This ignores noise (internal variability in the climate system and GCM climate simulations) and that the CSIRO report predicted frequency. Steve MicIntrye and the auditors repeat this mistake here, with the obligatory snark from Steve

The objection is that it unreasonable to expect climate models to predict ‘year-to-year variation’ with drought using this test. To set the record straight, I have run a small test demonstrating conclusively that the r2 does detect trends in frequency of intermittent events, (as opposed to trends in actual values) and consequently the test does not only rely only on year-to-year variations.

Below is a short R script where I represent a trend of increasing drought frequency with two independent sequences of numbers (0,1). These are plotted below. The results of fitting a linear regression to the sequences follow.

o runif(100)
e runif(100)
l<- lm(e~o)

> source("rtest.R")

lm(formula = e ~ o)

Min 1Q Median 3Q Max
-0.6739 -0.3148 -0.3148 0.3261 0.6852

Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.31481 0.06412 4.910 3.64e-06 ***
oTRUE 0.35910 0.09454 3.798 0.000253 ***
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4712 on 98 degrees of freedom
Multiple R-Squared: 0.1283, Adjusted R-squared: 0.1194
F-statistic: 14.43 on 1 and 98 DF, p-value: 0.0002527

The run shown produced a non-zero R-squared of 0.128, and significant slope (***), demonstrating that a similar trend in frequency of values does produce a positive correlation r2. In comparison, the r2 between modeled and drought data was essentially zero, indicating no detectable common trend in drought frequency using this method.

CSIRO Progress in Science Law

Geoff Sherrington has been drawing attention to some changes in the legal language attached to various emails and reports associated with the CSIRO and the Climate Adaptation Flagship (CAF). Since I have been posting up emails in an attempt to hold people accountable, I have been looking into the legality. In the case of reports, to what degree are the authors accountable for the accuracy of the contents? (Disclaimer: This post makes no representations or warranties regarding merchantability, fitness for purpose or otherwise with respect to the following assessment.)

DISCLAIMER (Email from Director Dr Andrew Ash, Sept 16 2008)

The information contained in the above e-mail message or messages (which includes any attachments) is Confidential / Commercial-in-confidence and may be legally privileged. It is intended only for the use of the person or entity to which it is addressed. If you are not the addressee any form of disclosure, copying, modification, distribution or any action taken or omitted in reliance on the information is unauthorised. If you received this communication in error, please notify the sender immediately and delete it from your computer system network.

This communication is intended for discussion purposes only and does not constitute a commitment by CSIRO to any agreement, memorandum of understanding, obligation, course of action or any other undertaking. Any transaction will be subject to contract and such contract will require approval in accordance with the Science and Industry Research Act, 1949. CSIRO will not be legally bound until these approvals are obtained.

On first reading you would think this prohibited publishing the email in blogs. However, as shown by the conditional emphasized, most of the terms are intended for a recipient other than the intended addressee. The prohibition on disclosure is therefore irrelevant to the recipient and doesn’t limit the them from doing anything they like with it, unless the sender were to claim that the email was not intended for the recipient. The advice I have received is that the language would NOT limit publication on a blog.

To correct this limitation, the CAF appears to have recently enlarged the scope of non-disclosure provisions in the following email, received by Geoff Sherrington.

DISCLAIMER: (14 Oct 2008, from James Davidson, Information Officer.)

This communication is for Discussion Purposes Only. It is not an agreement, memorandum of understanding, proposal, offer or the like, and is solely intended for informal discussion of ideas. For any agreement to be binding on CSIRO, it must be in writing, and executed on behalf of CSIRO by a person with proper authority, and in accordance with the Science and Industry Research Act 1949.

To the extent permitted by law, CSIRO does not represent, warrant and/or guarantee that the integrity of this communication has been maintained or that the communication is free from errors, virus, interception or interference.

The information contained in this email may be confidential or privileged. Any unauthorised use or disclosure is prohibited. If you have received this email in error, please accept my apologies and delete it immediately and please notify me.

Here, we are not told it is “Confidential / Commercial-in-confidence” but “may be confidential or privileged” and that “Any unauthorized use or disclosure is prohibited.”

First, what could constitute an “unauthorized use”? One would assume that permitted uses are spelled out in the first paragraph. That is, that use is authorized for “Discussion Purposes Only”. Unauthorized use would be for an “agreement, memorandum of understanding, proposal, offer or the like”.

So it seems that primarily what is prohibited is only using the email as a contract. This would be fair enough, as some people do send email as contracts. Clearly, normal emails are not contractual commitments, so it doesn’t apply to posting on a blog. However, the statement that “any disclosure is prohibited” would seem to contradict that, and be worrying.

Open to question is whether such a disclaimer would have any force without explicit agreement to terms and conditions by the addressee. Perhaps the next legal advance we may see from CSIRO CAF is a little button “I Agree” to click before we can open an email from them.

When we look at the the issue of accountability for a report, the Drought Exceptional Circumstances report (DECR) shows another expansion of scope. While the disclaimer in the DECR covered only CSIRO and BoM for “any liability for any opinion, advice and information”, and liability in the use of the information, it didn’t exclude the publishers of the report, the client organization DAFF.

Disclaimer (DECR)
CSIRO and the Bureau of Meteorology (BoM) make no representations or warranties regarding merchantability, fitness for purpose or otherwise with respect to this assessment. Any person relying on the assessment does so entirely at his or her own risk. CSIRO and the Bureau of Meteorology and all persons associated with it exclude all liability (including liability for negligence) in relation to any opinion, advice or information contained in this assessment, including, without limitation, any liability which is consequential on the use of such opinion, advice or information to the full extent of the law, including, without limitation, consequences arising as a result of action or inaction taken by that person or any third parties pursuant to reliance on the assessment.Where liability cannot be lawfully excluded, liability is limited, at the election of CSIRO and the Bureau of Meteorology, to the re-supply of the assessment or payment of the cost of re-supply of the assessmentl [sic].

The next report from CAF on the effect of global warming on fisheries covers the whole of the Australian Government. Whereas before there was “no liability for any opinion, advice and information”, this has been expanded to provide “no liability for the accuracy of or inferences from the material contained in this publication”. In other words, before there was “no liability for information” now it explicitly claims no liability for the “accuracy of the information”.

Important Notice – please read
This document is produced for general information only and does not represent a statement of the policy of the Australian Government. The Australian Government and all persons acting for the Government preparing this report accept no liability for the accuracy of or inferences from the material contained in this publication, or for any action as a result of any person’s or group’s interpretations, deductions, conclusions or actions in relying on this material.

It would seem they have tried to expressly remove liability for disseminating inaccurate information. Why? Various Acts forbid the dissemination of misleading and false information. The Trade Practises Act is the primary consumer protection legislation and states:

A person shall not, in trade or commerce, engage in conduct that is liable to mislead the public as to the nature, the manufacturing process, the characteristics, the suitability for their purpose or the quantity of any goods.

Goods could include reports, models, and possibly even the predictions of climate models, and may be covered by the TPA. However, could the new disclaimer by the CAF that tries to remove liability for inaccurate statements also immunize it from the TPA? I don’t think so.

Congratulations to the CAF for their advances in legal language attempting to protect them from being discussed on blogs, and bound to the standard consumer protections that bind every other corporate body. Following their lead, I will be including the following stupid disclaimer on all my emails:

By sending an email to me, to any of my aliases or to any of my addresses you are agreeing that:
1. I am by definition, “the intended recipient”
2. All information in the email is mine to make such financial profit, political mileage, or a good joke with as I see fit. In particular, I may quote it on the internet.
3. I may take the contents as representing the views of your organization.
4. This overrides any disclaimer or statement of confidentiality that may be in your disclaimer.

Theory of the Bailout

Sinfest has captured the essence of the bailout in comic form. What brave new theory permits contradictory gems like:

The US Government is to spend up to $250 billion buying direct stakes in banks and other financial institutions under a controversial emergency plan which President Bush insisted today was “not intended to take over the free market but to preserve it”.

One does not take over free markets to save them. It was and still is government intervention in the first place that is destroying the free markets. The root cause of the mess we are in is fractional reserve lending, an unsound currency, and interest rate micro-mismanagement by the Fed.