National Centers for Environmental Prediction
Washington, D.C. 20233
May 14, 1996 W/NP2:EK
Dr. Abraham Oort
We would like to ask your urgent advice about a problem that we have found
with the Reanalysis. We are sorry to tell you that we have discovered that an error was
introduced in the use of the PAOBS data (SH surface pressure bogus data) produced
by Australia. The error affects 1979-1992 (14 years). For that period we used the
PAOBS data collected by ECMWF, rather than our own decoded data, and in our
BUFR encoding the data was shifted by 180 degrees.
We have made preliminary experiments to assess the impact of this error, and
are enclosing results for your information. First of all the NH is not affected at all, and
the area which is most significantly affected is the SH south of 50-60S, which, as we
know, is the most sensitive to any kind of change (e.g., when we tested dropping the
"black list" of stations over India, the largest impact was also in the SH south of 50-60S;
this is where analyses from different operational centers differ the most, and where our
own analyses with different resolution differ the most as well). This "butterfly" effect is
explored in the NULL experiment described below.
In order to gauge the impact of this error we performed the following comparison
for a winter month, July 1979 (we also did the comparison for January 1979, and the
differences are even smaller because it is summer in the SH):
a) NEW assimilation: with corrected PAOBS
b) OLD assimilation: the Reanalysis with PAOBS in the wrong longitudes
c) NULL experiment: identical to a) (with corrected PAOBS), but with an initial first
guess for just the first day (1 July 1979) coming from the NOSAT experiment (to assess
the "butterfly effect of intrinsic analysis uncertainty).
A comparison of NEW-NULL (which have the same data) shows the impact of
the natural variability that comes from any change (a "butterfly" effect which exists in
the analysis due to the fact that the available data is not enough to determine the state
of the atmosphere). Only the last 21 days are included to allow for the initial
differences between the NOSAT and the regular reanalysis to subside (Fig. 9).
A comparison of NEW-OLD shows the impact of the wrong location of the
PAOBS, plus the butterfly effect, so that it should be compared with the NEW-NULL
in order to assess significance (i.e., changes south of 50-60S do not necessarily
constitute an impact of the PAOBS mislocation).
The following figures show the observed impact on the 21-day average. Only
the anomalies wrt the 17 year climatology are plotted, because on the full fields (not
shown) the differences are not noticeable to the eye:
Fig. 1: 21 day average of 1000 mb analysis anomalies.
a) OLD; b) NEW c) NULL and d) climatological daily standard deviation
Fig. 2: 21 day average of 500 mb analysis anomalies.
a) OLD; b) NEW c) NULL and d)climatological daily standard deviation
Fig. 3: Differences of 21 day average of 1000 mb and 500 mb analyses. a) NEW-OLD
1000mb; b) NEW-NULL 1000mb; c) NEW-OLD 500mb; d) NEW-NULL 500mb
Figs 1 and 2, show that, even when looking at the anomalies, the differences are very
small. When differences between the analyses are taken, they are confined mostly to
South of 60S, and are only marginally larger for NEW vs OLD than for NEW vs NULL.
The following figures show the impact on the daily fields (for a single day, 16
July 1979, which had the worst differences in the whole period, see Fig 9):
Fig. 4: 16 July 1000 mb analysis. a) OLD; b) NEW c) NULL and d) CLIM
Fig. 5: 16 July 500 mb analysis. a) OLD; b) NEW c) NULL and d) CLIM
Fig. 6: Differences of 1000 mb and 500 mb 16 July analyses. a) NEW-OLD 1000mb; b)
NEW-NULL 1000mb; c) NEW-OLD 500mb; d) NEW-NULL 500mb
As could be expected, the impact of the mislocation of the PAOBS is larger for
daily fields than for monthly averages. Looking at the synoptic maps, one can see
small differences between the NEW and the OLD, about twice as large as the
differences in the NULL experiment (but much smaller than, for example, between SAT
and NOSAT). So we can conclude that we doubled the intrinsic uncertainty in the
daily fields, but only south of 45S or so.
Fig. 9a, b shows the daily rms differences between the NEW-OLD and the NEW
-NULL for the whole month, between 21S and 80S. It takes about a week for the
influence of the the initial first guess to disappear. Again, on the daily fields, the rms
difference introduced by the PAOBS mislocation varies between 0% to about 80%
above the intrinsic difference expected from the "butterfly" effect, and this difference is
essentially confined to south of 50-60S.
This is also confirmed by Fig. 8 in which we plotted with a very small contour
interval (5m) the differences between the NEW and OLD global 21 day means, and Fig.
9, where we plotted the difference between the NEW and OLD analyses on 16 July
1979 (the worst day) with a 10m contour interval.
Fig. 10 shows the numbers of different types of data available (solid), and the
number that were actually accepted by the QC system. The four columns are NH
extratropics, SH extratropics, tropics (+/-30), and poles (>60N,S). The top is for the
NEW experiment and the bottotm for the OLD. It can be seen that in OLD, we rejected
about half the PAOBS, because of large differences wrt the first guess. Even in the
NEW analysis there is a high rate of rejection near the poles, due to the butterfly effect.
The Reanalysis team met yesterday to discuss this problem and how to proceed.
We see basically two choices and would like the advice of the Advisory Committee:
a) To stop the reanalysis and recompute the 14 years affected by this unfortunate
mistake. This would represent a major delay in our production of the reanalysis,
probably 9 months or more, on top of the delay already introduced by VTPR etc.
b) To recompute one or two years (let's say July 1979-July 1980 or July 1981)
with the correct location of the PAOBS, and offer this data base to researchers
that want to assess the extent to which the error affect their studies. We would
also do a corresponding NULL experiment. We plan to immediately post online
in the Reanalysis homepage results of the NEW, OLD and NULL experiments for
access by outside researchers. In the meantime we will continue with the pre-FGGE reanalysis.
We also considered a partial correction of the pressure fields (by reanalysing
the surface pressure and correcting hydrostatically the upper levels), but the
consensus was that it was going to bring more problems than solutions. It should be
noted that there are three reasons why the impact of the mislocation of the buoys is
relatively small: 1) Geostrophic adjustment indicates that surface pressure is a type of
data "hard to swallow" by the model; 2) the QC system rejected about 50% of the
PAOBS (Fig. 10); 3) we gave them less weight than ship and buoy data.
We are sorry about this mistake. It's effects are small enough that we did not
discover it earlier, and we believe that there may be other errors lurking in our system.
We look forward to your guidance in this matter.
Eugenia Kalnay, Director
Environmental Modeling Center
cc: R. McPherson
R. Jenne, R. Kistler, C. Ropelewski, J. Woollen, M. Kanamitsu, S. Saha, G.
White, W. Ebisuzaki, M. Chelliah, K. Mo