Notice
of Crisis Change to 3DVAR Analysis in Eta/EDAS
Brief
Description of Changes:
Surface-land
observations of temperature and all off-time surface data are being turned off
in the 3DVAR analysis used in the Eta and its Eta Data Assimilation System
(EDAS).
Reasons for
Changes:
For some
time, we have been searching for the reason Eta forecasts initialized off the
GFS analysis are superior to the operational ones initialized with our 3DVAR
analysis. I sent a note to you all
titled "Eta performance" on 8 November 2002 in which I stated we were
running a 32 km parallel (Etay) using GFS initial conditions. Stats from those runs were better than the
control run using the Eta 3DVAR. There
are a rather small number of differences between the Eta and GFS analysis
systems since they are both based on the same 3DVAR concepts. These differences include horizontal domain
& resolution (global ~55km vs No. America ~12km), vertical domain (model
top 0.2 mb vs 25 mb), balance constraints (strong vs weak), use of digital filter
(yes vs no), amount
of data
(more from late data dump vs less from early data dump) and use of surface data
(only surface pressure data over land vs all surface data over land). We've spent a ton of effort on the balance
issue but to make a long story short, we have very strong evidence that the GFS
vs Eta differences are due to the use of surface data and specifically the use
of surface temperature data.
The North
American Regional Reanalysis (NARR) was instrumental in our discovery. The NARR (a 32 km Eta/EDAS system using a
3DVAR identical to the one used in operations) was being ported from its
development platform to our old IBM to make its production runs of 25 year
period 1979-2003. During the port it
was discovered that surface observations of temperature, moisture and wind had
been turned off due to our use of the global reanalysis data files which had
these data flagged for non-use by the global reanalysis. On the IBM, this was corrected but the
performance of the system became degraded compared to the development
system. Numerous runs were made to
isolate the source of the degradation.
First, all surface data whose obtimes were more than 30 minutes
different than the analysis time were excluded. This had a small positive effect (mostly reduced noise). Second, runs were made with only surface
temperature, winds or moisture turned on.
There was one component that accounted for all of the degradation and it
was the surface temperatures. The first
2 attached gifs show the fit of the assimilation first-guess to 00z and 12z
raobs at different levels in the vertical for a system of EDAS runs made 3
hourly for the months of January and July 1993. The runs with surface temperature data show the degradation I
spoke of. While the impact is greatest
at 850 mb and 700mb (1000 mb level has very few raobs relative to upper
levels), it is very important to note that this degradation exists at all
levels in the troposphere. Surface wind
and moisture obs were found to have no negative impact and all of that data
within 30 minutes of analysis time are still used in the NARR.
Upon further
review, it is Dave Parrish's belief that the 3DVAR is handicapped by being cast
in the step-mountain framework. Because
this is NOT terrain following, it is impossible to cleanly limit the vertical
influence of surface data as is done in the current GFS analysis, the RUC 3DVAR
and was done in the previous Eta Optimum Interpolation analysis. Anisotropic covariances with vertical
stability dependence cast in a terrain following coordinate (at least near the
surface) has been our longterm goal, but that is too far away to wait for. Hence, we have decided to turn these data
off for the time being, until ways can be found to limit their influence.
Schedule for
change:
Final
testing : Ongoing since 14 August 2003
Expected
implementation : 9-11 September 2003
Description
of testing:
Extensive
30-day tests were conducted with the 32 km NARR system. A 12 km parallel EDAS system is running in
real-time parallel and we are making periodic 12 km Eta forecasts from 00z from
this test cycle. These parallel runs
(ETAX) can be viewed at Eric Roger's webpage:
http://www.emc.ncep.noaa.gov/mmb/mmbpll/etapll/
Results of
testing + verifications:
The attached
gif file (edas12test.jpg) shows the RMS fit of the 12 km EDAS first guess (ops
and parallel) vs. raob temperatures from 14-25 August 2003. The results are
very similar to those seen in the 32 km NARR, namely, withholding surface
temperatures from the 3DVAR leads to improved fit of the guess to raob
temperatures in the lower troposphere.
Statistical
verifications of the parallel ETAX forecasts have been made for QPF and at
upper-air and surface observations as well.
Check out http://wwwt.emc.ncep.noaa.gov/mmb/mmbpll/pll12stats.etx_3dv/
The impact
on QPF scores is universally positive with impact greatest in the eastern CONUS
and at 60 hour range. While there is
slight negative impact on forecast RH, the RAOB wind, temperature and height
fits are neutral to positive and become more positive with longer forecast
range. Fits to surface data show
surprisingly little impact - even for temperature.
Anticipated
impact of forecasts:
The impact
of this change on a 60-h Eta forecast is shown in the attached 4-panel chart in
"etavsetax.2003082412.gif". The top panels show 60-h forecasts of 500
mb height valid at 12Z 8/24/03 from the ops Eta (left) and the parallel Eta
(right) with the changes described above. The parallel-ops 500 mb height
difference is shown in the bottom left panel.
The feature of interest is the short-wave trough which is located off
the coast of New England. The trough is slower and deeper in the parallel Eta (top right) when compared to the ops
Eta (top left), with height differences parallel - ops 500 mb difference less
than -60 m over New Hampshire and Vermont. The verifying EDAS analysis in the
bottom right panel shows that the parallel Eta run had a better prediction of
the position and depth of the trough.
A recent
case has been pointed out by Phil Schumacher, SOO at Sioux Falls, with too cold
initialization of temperatures at 850 mb.
I have attached 3 gif files prepared by Eric Rogers. #1 opsedas.850.gif
is the 850 temp analysis over the Dakotas from the operational EDAS valid at
12Z 8/30; #2 plledas.850.gif is the same as #1, but for the parallel 12-km EDAS
run with the surface temperature data over land excluded from the 3DVAR, and #3
diff.850.gif is the difference = [parallel EDAS] – [ops EDAS]. Comparing #1 with #2, note the eastward
shift of the 8C isotherm, so that in the parallel EDAS analysis almost all of
North Dakota is between 8C and 12Z (ob at Bismarck at 12Z was 9C), while in the
ops EDAS all of North Dakota was < 8C at 850 mb. This case shows that excluding surface land temperatures from the
3DVAR analysis will help mitigate part of this problem.
Based on the
strength and extent of these impacts, we feel the inability of 3DVAR to
properly use surface temperatures over land is a major reason (if not THE
"smoking gun") for Eta 3DVAR being inferior to the GFS analysis. All other attempts to find comparability
with the GFS have been much less successful than this. Therefore, we expect these changes to result
in a) more accurate data assimilation cycle leading to b) more accurate
forecasts and c) more consistent performance from run to run.
Field evaluation:
Datasets and
webpages from the parallel runs were available for download and review,
respectively, but time has been limited.
Points of
Contact:
David.Parrish@noaa.gov 301-763-8000 ext 7742
Eric.Rogers@noaa.gov 301-763-8000 ext 7227
Geoff.DiMego@noaa.gov 301-763-8000 ext 7221
Future
changes:
New
procedures for use of surface temperatures in the 3DVAR are being considered
for inclusion in our late Fall Bundle.
This bundle of changes will include: a) new shortwave radiation scheme
which will eliminate our excess incoming shortwave and high bias in 2m
temperatures under clear skies, b) mods to BMJ deep and shallow convection will
sharpen the structure of the QPF, c) add GOES 12 radiances and d) modulate
analysed precip to compensate for its low bias in precip assimilation and thus
to produce unbiased soil moisture.
Implementation of these and other changes are planned for the late fall
of 2003.