Reanalysis PSFC problem 1948-1967

Evaluation - Reruns for Jan, Jul 1953

By pure serendipitity, a problem has been uncovered with the encoding of surface and mean sea level pressure that affect the period 1948-1967. The problem originated in the erroneous conversion of a portion of the TD13 dataset into the bufr ADPSFC type messages. Surface pressures in the past have suffured from a "Y2K" like problem in that values were stored with 3 digits ,pp.p, and a flag that inicated which hundreds units should be added (700,800,900,1000,etc). The discovery arose from a case study of the Reanalysis handling of the North Sea Gale of Jan 31:Feb 2, 1953 In addition to the forecast and verication grids, Suru made observation plots where the data rejected by the Reanl were plotted in red. The PSFC 03Z Feb 1, 1953 revealed the problem. Most of the obs within the 1000mb contour were rejected for being unphysically high pressure values. The increment plots revealed they were ~100 mb too large. If one mentally replaces the leading "10" with a "9", the values create a reasonable picture of the cyclone.

Once we saw that pattern, we told Jack Woollen what we had found, so that he could research the origin of these reports and the decoding procedures. Jack called back the next day and said he had found a tell-tail error in translating dataset TD13. Furthermore, it apeared that 4-6% of the surface obs in the set were affected.

In order to assess the geographical distribution over time, I have made two sequences maps. I had created grib files of the obs with surface pressures greater than 1050mb. I then used these in conjuction with the files Wesley has made available for plot and for download, and computed the percentages of obs <1050mb. This approach gives a sense of the location and volume of obs that were miscoded.

This error has good news and bad news. The good news is that the "bad data" were omitted. The analyses are not contaminated. The error is one of omission, not comission.

The bad news is that the omission of data is correlated with Northern Hemispere extratropical cyclones, thereby requiring the dynamics of the data assimilation process to completely determine the details of the surface pressure without the benefit of direct measurement when the pressure lowered below 1000 mb. In addition (and the source of the serendipity) European obs seems to bear the brunt of the problem, unlike US obs which were unaffected. Evaluation - Reruns for Jan, Jul 1953

Evaluation - Reruns for Jan, Jul 1953

A methodology to identify the miscoded sfc press and mslp obs proved more involved that I first thought.

1. Read all sfc obs for all synoptic times
2. Store sfc obs sorted by observed hour and location
3. Do a time series analysis of locations that reported either pressure value over 1049.9 mb.
4. Look for tendencies of adjacent times (restricted to no more than 24 hrs) of 75mb. This kept cold Siberian highs from being viewed as mis-coded.
5. Flag the "excesive" sequence.
6. When flagged mslp/ unflagged sfc press found, check the mslp d-value minus sfcp d-value difference.
   1. Small (<25mb): flag psfc for correction
   2. Large (>75mb, usually ~100mb) psfc is ok
   3. Indeterminate (>25 and <75): flag for rejection.
7. Write a synoptically time sorted file for the assimilation bufr editor to read and apply