It was described by the ABC and Agriculture Minister Tony Burke as like “a disaster novel”, but a simple test shows the drought predictions are not statistically significant, apart for the SW of Western Australia. The conclusions should read:

“Despite the significant increase in exceptional temperatures to 2050, apart for SWWA the modelling predicted no significant increase in areas experiencing exceptionally low rainfall or soil moisture.”

The Exceptional Circumstances Report models changes in the areal extent and frequency of exceptionally high temperatures, low rainfall
and low soil moisture for seven Australian regions. It is predicting a doubling of drought declarations from reduction in rainfall and soil moisture across much of Australia’s farming regions.

First there was confusion over the release and a hurried re-edit, recorded here.
The supplementary data for the Drought Exceptional Circumstances Review did not
contain the data from the 13 climate model predictions.
Despite this, it is possible to estimate the 95% confidence intervals, and evaluate the statistical significance of the results.

Tabulated below are the data from tables 4, 7 and 9, the percentage area affected by exceptionally low soil moisture, rainfall and temperatures. The caption for the soil table is as follows:

Table 9: Simulated percentage area having exceptionally
low annual-average soil moisture for 1957-2006 and 50
years centred on 2030, based on 13 climate models.The
low and high scenarios represent the lowest and highest
ten per cent of the range of model results.

As the high and low scenarios are an average of the low and high 10% of model results, these would be approximately equal to the 95% confidence intervals. So, assuming the observed historic value (over a range of years) is equivalent to a realization of a model result, (the Gavin Schmidt assumption) significance occurs if the observed value lies outside the modeled range. The next three columns show the 95% CI interval, as the average of high and low interval, the difference between the observed value (O) and the predicted value (P), and the significance at the 95% level.

Almost all regions (SWWA is the exception) show no significant change in % area of exceptionally low soil moisture and low rainfall. Exceptional temperature is significant in all regions, but is not an indicator of drought.

The lack of significance of the results contradicts the conclusions of the report, lower rainfall and soil moisture and more drought.

Observed (O) Low Mean (P) High 95% CI Diff. O-P Significant
Qld 6.5 4.4 7.4 10.6 3.1 0.9 FALSE
NSW 6.3 4.5 7.1 10 2.75 0.8 FALSE
Vic&Tas 6.6 7 11 15.2 4.1 4.4 TRUE
SW 6.2 6 8.5 11.3 2.65 2.3 FALSE
NW 6.3 4.4 7 10 2.8 0.7 FALSE
MDB 6.2 4.6 7.3 10.2 2.8 1.1 FALSE
SWWA 6.1 12.8 15.9 20 3.6 9.8 TRUE
Qld 5.5 1.7 5.8 9.7 4 0.3 FALSE
NSW 5.6 2.2 5.6 10.9 4.35 0 FALSE
Vic&Tas 5.4 3 9.7 13.9 5.45 4.3 FALSE
SW 5.6 5 8.4 12.1 3.55 2.8 FALSE
NW 5.6 1.8 5.3 8 3.1 -0.3 FALSE
MDB 5.6 2.7 6 11.1 4.2 0.4 FALSE
SWWA 5.5 9 18.4 26.5 8.75 12.9 TRUE
Qld 4.6 48.9 62.2 73.8 12.45 57.6 TRUE
NSW 4.5 43.5 62.1 81 18.75 57.6 TRUE
Vic&Tas 4.6 60.5 76.1 95 17.25 71.5 TRUE
SW 4.6 49.1 68.4 86.3 18.6 63.8 TRUE
NW 4.6 50 63.5 82 16 58.9 TRUE
MDB 4.5 45.2 64.9 90.1 22.45 60.4 TRUE
SWWA 4.6 63.1 81.9 97.1 17 77.3 TRUE

Two ways the report biased the results are as follows:

1. No statistical significances were quoted, only averages. That way, insignificantly small increases in averages could be interpreted as increases in droughts. As far as I know, political correctness has not released scientists from the responsibility to disclose when their results are not significant — or has it?

2. The 10% of high model results were treated as high scenarios. From the methods section:

Low, mean and high
scenarios are given, where the mean is the 13-
model average, and the low and high scenarios are
the lowest and highest 10% of the range of model
results, respectively.

That is, variation between different climate models was treated as equivalent to differing climate change scenarios, such as different levels of CO2 emission. That way, the high statistical tail results could imply even more disastrous scenarios. So far as I can see, the two have no relationship to each other.

Hmm… disaster novel or just a disaster?

About these ads