An NCEP/EMC Perspective
26 Feb 2002
a) The NCEP Stage II analysis and its usage
The real-time, hourly, multi-sensor national precipitation analysis (the NCEP Stage II analysis - Baldwin and Mitchll, 1997) , developed at National Center for Environmental Prediction (NCEP) based on the multi-sensor precipitation (MPE) algorithm developed in the Office of Hydrological Development (OHD; Seo, 1998), has been produced at NCEP and archived at UCAR since 1996.
The Stage II analysis has been used by NCEP and others for model forecast verification and precipitation data assimilation. In Jul 2001 the precipitation assimilation in the Eta model became operational. The assimilation of hourly precipitation data improves the model’s soil moisture and short-term precipitation forecast.
b) The Stage III/Stage IV analyses
The 12 NWS River Forecast Centers over CONUS also produce hourly analyses over each of their domains, using algorithm similar to the one used at NCEP (Seo, 1998. Actually the RFCs are ahead of us, they are, or will soon be, using a much-improved version of MPE. NCEP will upgrade its MPE software). Starting 1 Dec 2001, NCEP has been quilting (mosaicking) these Stage III products into a national product (the Stage IV analysis - see http://wwwt.emc.ncep.noaa.gov/mmb/ylin/pcpanl/stage4/). The RFCs produce an automated version of the Stage III analysis, available shortly after the top of the hour, followed by a manually quality-controled version, several hours later. At NCEP, due to man power constraints, the Stage II analysis is not manually QC’d.
c) Timeliness requirement of data:
In the earlier years the NCEP Stage II analysis, produced directly from gauge and radar data, was the only national real-time multi-sensor hourly precipitation analysis available. Now that the RFCs are routinely producing the manually QC’d Stage III, the resulting mosaicked Stage IV is superior to the NCEP Stage II. NCEP will continue to produce the Stage II analysis (with appropriate software upgrades along the way), chiefly to fill any ‘holes’ in the Stage IV data, for the benefit of precipitation assimilation in the operational model, since the in-house Stage II product is more timely than the Stage III/Stage IV. For that reason, we need to improve automated QC of NCEP Stage II.
For verification purposes, QC is not a problem since one can just wait 9-12 hours and use the Stage IV, which already benefits from the manual QC at the RFCs. Timeliness is a model assimilation issue.
II. The gauge QC problem
Radar and gauge precipitation estimates are subject to both random and systematic errors. Systematic errors in radar precipitation estimates have been studied extensively and part of the MPE algorithm is built to correct this problem (Seo, 1999). At NCEP we plan to develop methods to quantify and correct the systematic biases in the multi-sensor hourly analysis before it is used by the model in the precipitation assimilation (Ken Mitchell has outlined a method to perform this bias measurement/correction).
MPE software in use at the RFCs also provides graphic interface to allow manual editing of data that are deemed to be random error (Breidenbach et al., 1998). What concerns NCEP most is random gauge error.
There are two types of gauges data used in the NCEP Stage II, the HADS (Hydrometeorological Automated Data System) data and the METAR data. At the present, the METAR data are used in the “early” version of the hourly analysis, performed at 33 minutes after the top of the hour, and the HADS data are used in a re-run of the analysis performed 6 hours later (the HADS data are transmitted via the GOES DCP (data collection platform) and most stations only send their precipitation data every 4 hours or so). In the next implementation, we plan to use all available gauge data in both the “early” and the “delayed” analyses.
Though we are aware that METAR gauge data have some problems, the biggest gauge QC problem we have encountered is with the HADS data (the METAR has a human component that would likely catch the really egregious errors. Errors in METAR are probably more subtle and more difficult to spot).
Typical varieties of HADS gauge errors include
III. Current approaches to gauge QC
a) At NCEP
We keep a reject-list of apparently faulty gauges that tells the MPE software to avoid using data from these gauges. This list is updated manually – we plot the 24h-total (12Z-12Z) of the NCEP Stage II analysis, and compare it to other 24h precipitation analyses for the same time period (e.g. the 24h gauge-only precipitation analysis from ~8,000 gauges, QC’d by NCEP/CPC, and the 24 sum of the Stage IV analysis). When we notice a bull’s-eye in the NCEP Stage II that is likely to be faulty, we attempt to identify the gauge that’s responsible for it (the FSL gauge page at http://precip.fsl.noaa.gov/hourly_precip.html proved to be very helpful), and add it to the gauge reject-list if we decide that this gauge is likely to be faulty. The only other QC measure is the 1”/hr gross-range check placed on the gauge data. The 1”/hr limit is imposed to prevent sudden appearances of large, erroneous gauge reports from wreaking havoc with the analysis and model precipitation assimilation.
This system is far from ideal, since there is only one person (me - YL) checking the data/updating the reject-list; the list is not kept up-to-the-hour (a misbehaving gauge is usually not flagged until a day or two later, or even later than that). There are currently 60 gauges (approximately 2% of total) in the reject-list. Random check seems to indicate that many of them are still bad after several months, but some seem to be behaving OK. There is no measure in place to rehabilitate the rejected gauges, so thinning of the gauge population is a concern.
Other measures that have been discussed at NCEP: make gauge QC part of the NCEP operations, develop graphic/visual display of gauge data, overlay with current weather/cloud map. Have the SDMs (senior duty meteorologists) monitor the incoming gauge reports and flag bad gauge data (basically duplicating the manual QC done at the 12 RFCs). This is very labor-intensive.
b) At RFCs
Pretty much the same thing (reject-list and manual QC via graphic/visual display). They have more manpower to do this, and they each have a smaller area to deal with.
c) Method under development at OHD
“Buddy check” algorithm that sends red flag to human analyst (under development by Chandra Kontragunta); check against radar reflectivities (conversation with Seo).
IV. Proposed automated gauge QC
For NCEP, the most critical/useful thing would be to develop some automated software to identify bad gauges sooner, and to restore ‘reformed’ bad gauges to so the rank of gauges will not thin out over time (an up-to-the-hour gauge reject-list would be ideal). To that end, we propose the following:
a) Real-time QC: done at the time of the hourly analysis
At each hour, before the gauge data are used in the analysis, they are compared against
i) Satellite cloud data (DiMego’s idea)
ii) Radar reflectivities (conversation with Seo)
In the most obvious scenario, if a gauge is reporting precipitation in an area that we can be reasonably sure to be cloudless/rainless, we can toss out this gauge report. This should lessen the gauge ‘false positive’ (II a-c) problem.
b) Near real-time QC (6-24h later)
i) Check against co-located 24h gauges (the email to Moninger).
ii) Check against Stage III analysis and QC’d 24h gauge-only analysis (DiMego): this is a way to indirectly benefit from the manual QC done by others.
c) Continued monitoring of “bad” gauges
Techniques developed in b) can also be used to continue to monitor gauges that have been flagged as bad. After a period of good behavior these gauges can be removed from the reject-list.
We plan to start testing/implementing a) at NCEP in the near future. We hope that FSL can develop techniques for b) and c).
V. Further possibilities of improving the quality of gauge data
Measures proposed above would go a long way towards improving the quality of the NCEP Stage II analysis and consequently improving the operational precipitation assimilation, so that’s what we seek in the near term. In the long run, as gauge QC is perhaps the thorniest problem in hourly precipitation analysis (faced by NCEP as well as the RFCs), after the above measures have been taken at NCEP and the techniques currently being developed at OHD (Kontragunta) are implemented in the future version of the MPE software for the RFC’s, the community might consider expanding the effort to include
a) Sharing of gauge QC database
Currently each of the 12 RFCs keeps its own gauge reject-list for gauges over its own domain, no sharing of these data (even though domains of RFCs overlap in places). Surely the gauge information noted down by the human analysts at each RFC could be useful to other RFCs and to NCEP, and the RFCs might benefit from an objectively-generated gauge-reject list for the entire CONUS. Maybe there could be a central clearing house - a gauge credibility bureau – where up-to-date gauge ratings are received and maintained.
b) Feed back to gauge keepers
Who is responsible for maintaining these gauges? If we develop a database for the gauges and their keepers, when a gauge misbehaves we can alert the proper gauge-keeper to get it “unstuck”.
Baldwin, M. E., and K.E. Mitchell, 1997: The NCEP hourly multi-sensor U.S. precipitation analysis for operations and GCIP research. Preprints, 13th AMS Conference on Hydrology, Long Beach, CA, 54--55.
Redenbacher, J., Sao, D.J. and R. Fulton, 1998: Stage II and III Post Processing of NEXRAD Precipitation Estimates in the Modernized Weather Service, AMS 78th Annual Meeting.
Seo, D.J., 1998: Real-time estimation of rainfall fields using radar rainfall and rain gauge data. J. of Hydro., 208, 37-52.
Seo, D.J., 1999: Real-time estimation of mean field bias in radar rainfall data. J. of Hydrol., 223, 131-147.