Australian Temperature Adjustments II

Paz pointed out that the normalization used previously might not remove geographic biases introduced by fugitive weather stations, so here is another approach. I have differenced each of the records, averaged the differences and then cumulative summed the result.

fig2

This way we are dealing only in annual increments, up until the final summation. The result differs from the previous, as the pop-up in 1914 is lowered, and the temperatures post 2000 are raised. However, it still seems to depart somewhat from the official version.

Code: script.R

Advertisements

0 thoughts on “Australian Temperature Adjustments II

  1. I am not sure I understand this: “..differenced each of the records, averaged the differences and then cumulative summed the result.”You and everyone else would like to use as many stations as possible despite length of measurements.I dont think that is possible in the way You explained. You dont know short lived stations “level” form “normal average” I belive the long lived stations need to be the base and then add the short lived one as (minor) adjustments.Use the long lived = the one covering the whole period, and average those as in your example 1. Then add each short lived station one by one with the longest short one first.The short one should be averaged one by one and then used best fit to the main curve. That fit is minimum sum of divergence to the main curve over the period.The last short station, even if it just one measurement, would then have no implication on the average or result.

  2. The problem with long records approach is that you are open to the charge of cherry-picking the records. How long is long? Check out the code, but I get a matrix of annual changes for each station, then average them for each year, giving the overall year-to-year changes. Then I accumulate the differences to get back to the original.

  3. I am not sure I understand this: “..differenced each of the records, averaged the differences and then cumulative summed the result.”

    You and everyone else would like to use as many stations as possible despite length of measurements.
    I dont think that is possible in the way You explained. You dont know short lived stations “level” form “normal average”

    I belive the long lived stations need to be the base and then add the short lived one as (minor) adjustments.
    Use the long lived = the one covering the whole period, and average those as in your example 1. Then add each short lived station one by one with the longest short one first.
    The short one should be averaged one by one and then used best fit to the main curve. That fit is minimum sum of divergence to the main curve over the period.
    The last short station, even if it just one measurement, would then have no implication on the average or result.

    • The problem with long records approach is that you are open to the charge of cherry-picking the records. How long is long?

      Check out the code, but I get a matrix of annual changes for each station, then average them for each year, giving the overall year-to-year changes. Then I accumulate the differences to get back to the original.

  4. David, I agreed with Paz, and I think your second approach is better. It actually gives about the same trend as the BOM version. It differs in detail – I suspect that is because BoM uses an area-weighted average, whereas yours is just an arithmetic mean.

  5. David, I agreed with Paz, and I think your second approach is better. It actually gives about the same trend as the BOM version. It differs in detail – I suspect that is because BoM uses an area-weighted average, whereas yours is just an arithmetic mean.

  6. David, thanks for this — using differentiation is an elegant way of doing the pure trends recorded at the stations.It is very interesting where the remaining differences come from. As Nick said, area weighting might be one influence. The other one is, of course, as you say, absolute (not differenced) temperature that is recorded in the stations.

  7. David,Your reconstruction of the data seems to confirm the BOM presentation (although not in detail as N. Stokes points out). It points to a rise in the “trend” average since the 1920s of 1.1-1.2 degrees: about the same as the adjusted BOM trend. This is at about 50% higher than e.g. the HADCRUT3 “global” trend rise over the same period. Also, your reconstruction, like the BOMs shows almost all the warming occurring since 1980.What do you think we learn from your re-construction of the data? That the warming trend is (really) in the data, is not trivial and invites an explanation? That Australia has warmed faster than the rest of the world (if HADCRUT accurately represents the rest of the world)? (Obviously, it does not tell us that the future will be like the past).

  8. David, thanks for this — using differentiation is an elegant way of doing the pure trends recorded at the stations.

    It is very interesting where the remaining differences come from. As Nick said, area weighting might be one influence. The other one is, of course, as you say, absolute (not differenced) temperature that is recorded in the stations.

  9. David,

    Your reconstruction of the data seems to confirm the BOM presentation (although not in detail as N. Stokes points out). It points to a rise in the “trend” average since the 1920s of 1.1-1.2 degrees: about the same as the adjusted BOM trend. This is at about 50% higher than e.g. the HADCRUT3 “global” trend rise over the same period. Also, your reconstruction, like the BOMs shows almost all the warming occurring since 1980.

    What do you think we learn from your re-construction of the data? That the warming trend is (really) in the data, is not trivial and invites an explanation? That Australia has warmed faster than the rest of the world (if HADCRUT accurately represents the rest of the world)?

    (Obviously, it does not tell us that the future will be like the past).

    • Peter, I will do a post on the comparison with the official version,
      but I would say that this quick method largely agrees with BoM, except
      for the recent (~10 yr) difference, so there appears not to be the
      issue with the raw adjustments as noted in NZ here, unless the data is
      already adjusted.

  10. Peter, I will do a post on the comparison with the official version,but I would say that this quick method largely agrees with BoM, exceptfor the recent (~10 yr) difference, so there appears not to be theissue with the raw adjustments as noted in NZ here, unless the data isalready adjusted.

  11. Is it possible that this result is consistent with Pielke Sr.'s theory that land use changes have caused warming in Australia?

  12. I believe Geoff Sherrington has looked at the Australian data and cometo a similar view. To me, the data I have is a small 103 site'rehabilitated' data set, and all I can say is that the quick methodlargely matches the official version. Next step is to go back and tryit on the raw data, as the issue seems to be whether the 'adjustments'are a large part of the observed trend.

  13. Is it possible that this result is consistent with Pielke Sr.’s theory that land use changes have caused warming in Australia?

    • I believe Geoff Sherrington has looked at the Australian data and come
      to a similar view. To me, the data I have is a small 103 site
      ‘rehabilitated’ data set, and all I can say is that the quick method
      largely matches the official version. Next step is to go back and try
      it on the raw data, as the issue seems to be whether the ‘adjustments’
      are a large part of the observed trend.

      • I think there is no question that the substantial land use changes Australia has experienced has effected it’s climate. A good question is how, where, and how much.

        Do you know of a paper by Pielke Sr which looks into this? I would enjoy having a look at that!

  14. I think there is no question that the substantial land use changes Australia has experienced has effected it's climate. A good question is how, where, and how much.Do you know of a paper by Pielke Sr which looks into this? I would enjoy having a look at that!

  15. I know it's bad form to self-reference, but this work of David's has stations that seem to be the same as some I used in May 4th 2009 “40 Years of some BOM temperature Data”. I started this database in 1968 to partially overcome the problem of length of observation period that has been discussed above.The main finding was that coastal stations had barely changed in Tmax or Tmin in that 40 years, while inland stations had a trend upwards of up to 2 deg C per century. This has to be a transient or instrumental effect, because if you project it for a few decades forwards or back, you get seemingly impossible numbers.Then I wondered if the data were adjusted. There is relevant information in the CRU leaks, for example see this trail from “Marine Shale” on CA, who is a mystery name to me.Marine_Shale (16:54:20) : The official Australian temperature record is also based on significant “adjustments’ of raw historical data.The main adjustments were made by Torok and Nicholls in 1996 and then a few more adjustments were done by Paul Della-Marta in 2000.to my knowledge this process has never been properly audited, but from a few stations that I have checked I see that there is a similar lowering of temps in the early part of the records and a raising later on.All the data can be found in this folder.ftp://ftp2.bom.gov.au/anon/home/bmrc/perm/climate/temperatureWithin the last URL there is a section named 08/02/1999 12:00AM 26,328 alladj.utx.Z (This is one higher directory level up from the page that opens, under “annual”) It gives the dates and magnitudes of step changes to raw data for many stations that became part of the Reference Climate Station Network. Changes range from +7 deg C to -5.5 deg C (though many of the bigger ones are in the 1800s), seemingly often for the reason that they looked better eyeballed that way. At least, I think that code “o” allows for eyeballing. The alladj.utx.Z opens up with this at the start (then a whole lot of data, small bit given here) Key Station Element (1021=min, 1001=max)YearType (1=single years, 0=all previous years)AdjustmentCumulative adjustment Reason : o= objective test f= median r= range d= detect documented changes : m= moves= stevenson screen suppliedb= buildingv= vegetation (trees, grass growing, etc)c= change in site/temporary siten= new screenp= poor site/site clearedu= old/poor screen or screen fixeda= composite movee= entry/observer/instument problemsi= inspectiont= time change*= documentation unclear 01013 1021 1968 0 -1.3 -1.3 fda 01013 1021 1925 0 +0.3 -1.0 od 01013 1001 1980 0 +0.2 +0.2 odb 01013 1001 1968 0 -0.6 -0.4 orda 01013 1001 1942 0 +1.3 +0.9 ord 01013 1001 1921 0 -0.4 +0.5 ord 02012 1021 1985 0 +0.6 +0.6 ord 02012 1021 1959 0 -0.3 +0.3 od 02012 1021 1947 0 +1.3 +1.6 oda 02012 1001 1982 0 -0.4 -0.4 or 02012 1001 1975 0 +0.2 -0.2 o 02012 1001 1966 0 -0.5 -0.7 ord etc etc for 2,800 linesStation 01013 is Wyndham WA. Station 02012 is Halls Creek Airport WA. These are Australian station numbers, different to World numbers. They are on BOM sites online. This is the first time I have had a chance to get an eye an Australian data before it was adjusted. The moves are huge and sometimes questionable.The Torok & Nichols paper has a longer explanation for adjustments.Torok, S. and Nicholls, N., 1996. An historicaltemperature record for Australia. Aust. Met. Mag. 45, 251-260. “Abstract A high-quality historical surface air temperature data set, for mean annual temperatures, has been prepared for Australia by adjusting data for inhomogeneities caused by station relocations, changes in exposure and other discontinuities. An objective procedure was developed for determining the necessary adjustments. Station history documentation was also used for this purpose. Time-series of annual mean maximum and mean minimum temperatures have been produced for 224 stations. Trends in annual mean maximum, minimum, the mean of the maximum and minimum, and the range between maximum and minimum, have been calculated at each site. The data set provides adequate spatial coverage of Australia back to 1910 for the production of all-Australia average temperatures. Maximum and minimum temperatures have increased since about 1950, with minimum temperatures increasing faster than the maximum temperatures. “(Much more detail is in the body of the paper, of course).So far as I can tell, the data that are online for BOM and Australian Weather News are adjusted and are NOT raw. To the extent that data sets are adjusted, you will get a different analysis each time you use a different set.The 1940s “bump” might coincide with a lot of airfields built in the tropics and weather stations shifted there. Also, in the 1989-94 period there was a changeover from mercury max-min thermometers to thermocouple types with half-hourly recording. I hope this helps with understanding.

  16. I know it’s bad form to self-reference, but this work of David’s has stations that seem to be the same as some I used in May 4th 2009 “40 Years of some BOM temperature Data”. I started this database in 1968 to partially overcome the problem of length of observation period that has been discussed above.

    The main finding was that coastal stations had barely changed in Tmax or Tmin in that 40 years, while inland stations had a trend upwards of up to 2 deg C per century. This has to be a transient or instrumental effect, because if you project it for a few decades forwards or back, you get seemingly impossible numbers.

    Then I wondered if the data were adjusted. There is relevant information in the CRU leaks, for example see this trail from “Marine Shale” on CA, who is a mystery name to me.

    Marine_Shale (16:54:20) :

    The official Australian temperature record is also based on significant “adjustments’ of raw historical data.
    The main adjustments were made by Torok and Nicholls in 1996 and then a few more adjustments were done by Paul Della-Marta in 2000.
    to my knowledge this process has never been properly audited, but from a few stations that I have checked I see that there is a similar lowering of temps in the early part of the records and a raising later on.
    All the data can be found in this folder.
    ftp://ftp2.bom.gov.au/anon/home/bmrc/perm/climate/temperature

    Within the last URL there is a section named 08/02/1999 12:00AM 26,328 alladj.utx.Z

    (This is one higher directory level up from the page that opens, under “annual”)

    It gives the dates and magnitudes of step changes to raw data for many stations that became part of the Reference Climate Station Network. Changes range from +7 deg C to -5.5 deg C (though many of the bigger ones are in the 1800s), seemingly often for the reason that they looked better eyeballed that way. At least, I think that code “o” allows for eyeballing.

    The alladj.utx.Z opens up with this at the start (then a whole lot of data, small bit given here)

    Key

    Station
    Element (1021=min, 1001=max)
    Year
    Type (1=single years, 0=all previous years)
    Adjustment
    Cumulative adjustment

    Reason : o= objective test
    f= median
    r= range
    d= detect

    documented changes :
    m= move
    s= stevenson screen supplied
    b= building
    v= vegetation (trees, grass growing, etc)
    c= change in site/temporary site
    n= new screen
    p= poor site/site cleared
    u= old/poor screen or screen fixed
    a= composite move
    e= entry/observer/instument problems
    i= inspection
    t= time change
    *= documentation unclear

    01013 1021 1968 0 -1.3 -1.3 fda
    01013 1021 1925 0 +0.3 -1.0 od
    01013 1001 1980 0 +0.2 +0.2 odb
    01013 1001 1968 0 -0.6 -0.4 orda
    01013 1001 1942 0 +1.3 +0.9 ord
    01013 1001 1921 0 -0.4 +0.5 ord
    02012 1021 1985 0 +0.6 +0.6 ord
    02012 1021 1959 0 -0.3 +0.3 od
    02012 1021 1947 0 +1.3 +1.6 oda
    02012 1001 1982 0 -0.4 -0.4 or
    02012 1001 1975 0 +0.2 -0.2 o
    02012 1001 1966 0 -0.5 -0.7 ord etc etc for 2,800 lines

    Station 01013 is Wyndham WA. Station 02012 is Halls Creek Airport WA. These are Australian station numbers, different to World numbers. They are on BOM sites online.

    This is the first time I have had a chance to get an eye an Australian data before it was adjusted. The moves are huge and sometimes questionable.

    The Torok & Nichols paper has a longer explanation for adjustments.

    Torok, S. and Nicholls, N., 1996. An historical
    temperature record for Australia. Aust. Met. Mag. 45, 251-260.

    “Abstract
    A high-quality historical surface air temperature data
    set, for mean annual temperatures, has been prepared for
    Australia by adjusting data for inhomogeneities caused by
    station relocations, changes in exposure and other
    discontinuities. An objective procedure was developed for
    determining the necessary adjustments. Station history
    documentation was also used for this purpose. Time-series
    of annual mean maximum and mean minimum temperatures have been produced for 224 stations. Trends in annual mean
    maximum, minimum, the mean of the maximum and minimum,
    and the range between maximum and minimum, have been
    calculated at each site. The data set provides adequate
    spatial coverage of Australia back to 1910 for the
    production of all-Australia average temperatures. Maximum
    and minimum temperatures have increased since about 1950,
    with minimum temperatures increasing faster than the
    maximum temperatures. “(Much more detail is in the body of the paper, of course).

    So far as I can tell, the data that are online for BOM and Australian Weather News are adjusted and are NOT raw. To the extent that data sets are adjusted, you will get a different analysis each time you use a different set.

    The 1940s “bump” might coincide with a lot of airfields built in the tropics and weather stations shifted there. Also, in the 1989-94 period there was a changeover from mercury max-min thermometers to thermocouple types with half-hourly recording.

    I hope this helps with understanding.

  17. The Pielke snr paper on the effect of land use on climate is here;

    http://wattsupwiththat.com/2009/08/19/impacts-of-land-use-land-cover-change-on-climate/

    Another paper based in the US is here;

    http://wattsupwiththat.com/2009/11/11/georgia-tech-50-percent-of-the-usa-warming-that-has-occurred-since-1950-is-due-to-land-use-changes/#comments

    Obviously some land use changes would have a cooling effect but given Stewart Franks new paper which shows that an increase in temperature doesn’t cause an increase in evaporation, would a dam have a cooling or a warming effect?

  18. Pingback: tutaj

  19. Pingback: strona firmy

  20. Pingback: zobacz tutaj

  21. Pingback: wszystko o masazu

  22. Pingback: polecam

  23. Pingback: zobacz oferte

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s