PURPOSEWorkshop on the

USE OF ENSEMBLES IN DATA ASSIMILATION

April 13-14, 1999, NCEP, Camp Springs, MD

8:30-9:00 Refreshments

*INTRODUCTION*

9:00 Louis Uccellini, Director,
NCEP:

Welcoming Remarks

9:05 Steve Lord, Acting Director, EMC:

Purpose and expectations of the workshop

9:15 John Derber:

Expected advantages of using
ensembles in data assimilation

9:45 Discussion

**ENSEMBLE-BASED ANALYSIS SCHEMES**

10:00 Peter Houtekamer:

Prospects
for an operational ensemble Kalman filter

10:30 Break

10:45 Jeff Anderson:

Application of a fully
non-linear filter and Monte Carlo techniques to ensemble data assimilation
in intermediate models

11:15 Craig Bishop:

The
Ensemble Transform Kalman Filter

11:45 Discussion

12:00 Lunch brought in

13:00 Jan Barkmeijer:

Recent
developments in the ECMWF Ensemble Prediction System

13:30 Martin Ehrendorfer (Presented
by J. Barkmeijer):

13:45 Chris Snyder:

Dynamics
and statistics of forecast errors in a quasi-geostrophic model

14:00 Tom Hamill:

Analysis and forecast errors
and the Ensemble Kalman Filter

14:15 Discussion

14:30 Break

**ENSEMBLES SUPPORTING EXISTING ANALYSIS SCHEMES**

14:45 Dale Barker:

The
specification and use of synoptically-dependent background errors in 3DVAR
using information from an Error Breeding Cycle

15:15 Eugenia Kalnay:

Some
ideas on the possible use of bred vectors in data assimilation

15:45 Jeff Whitaker:

Spread/analysis
error relationships in a simple model

16:05 Zoltan Toth:

Estimating analysis uncertainty
using the NCEP global ensemble

16:35 Discussion

17:00 Workshop Report Structure - Suggested topics:

a) Required ensemble characteristics

b) Use of ensembles with existing
analysis schemes

c) Ensemble-based analysis schemes

17:30 Adjourn for day

18:30 Optional dinner

*Wednesday, April 14*

8:00 Refreshments

**ALTERNATIVES TO USING ENSEMBLES**

8:30 Jim Purser:

Estimates of flow dependent
covariance structures

8:50 Lars Peter Riishojgaard:

Anisotropic flow-dependent
modeling of the forecast error correlations

9:20 Milija Zupanski

Plans for using fully flow-dependent background
error covariance information in the NCEP regional 4-DVAR analysis
system

9:30 Discussion

9:45 Break

10:00 Summary of presented material (Report
Leader and Rapporteurs)

Writing assignments established

11:00 Writing (In 2-3 groups)

12:00 Lunch brought in

13:00 Writing continues (In 2-3 groups)

14:00 Break

14:30 Discussion: Review and Revise Group Reports; Finalize Workshop Report

16:00 Adjourn

Jeff Anderson, GFDL, Princeton, NJ

Stephen Anderson, Metron, Reston, VA

Dale Barker, UK Met. Office, Bracknell, UK

Jan Barkmeijer, ECMWF, Reading, England

Craig Bishop, Pennsylvania State University,
State College, PA

Steve Cohn, NASA GSFC, Greenbelt, MD

Kerry Emanuel, MIT, Boston, MA

Brian Etherton, Pennsylvania State University,
State College, PA

Peter Houtekamer, AES, Dorval, Canada

Tom Hamill, NCAR, Boulder, CO

Eugenia Kalnay, University of Oklahoma, Norman,
OK

Sharan Majumdar, Pennsylvania State University,
State College, PA

Zhao-Xia Pu, USRA - NASA GSFC, Greenbelt,
MD

Chris Snyder, NCAR, Boulder, CO

Jeff Whitaker, CDC, Boulder, CO

From NCEP:

John Derber

Steve Lord

Dave Parrish

Jim Purser

Istvan Szunyogh

Zoltan Toth

Steve Tracton

Wan-Shu Wu

Dusanka Zupanski

Milija Zupanski

Application of a Fully Non-Linear Filter and Monte
Carlo Techniques

to Ensemble Data Assimilation in Intermediate
Models

A probabilistic approach to the fully non-linear
filtering problem is

developed in the context of the problem of data
assimilation for

atmospheric and oceanic models. The goal of this
method is to produce a

probability sample of the state of a dynamical
system that is

consistent with a set of temporally discrete
observations. This sample

can then be used as initial conditions for ensemble
forecasts which

themselves approximate probability samples of
the forecast state of the

system. Both the state of the assimilating model
and the set of

observations available at a given instant of
time are treated formally

as random variables. The state of the system
given a new set of

observations can then be computed, after an application
of Bayes' rule,

as a convolution of the conditional probability
densities associated

with the new observations and the prior density
generated from a

knowledge of the model's dynamics and all previous
observations.

Traditionally, this problem has been simplified
through linearization

leading to the Kalman-Bucy filter. However, in
the approach discussed

here, a Monte Carlo (ensemble) approach is used
to sample the prior

density and an expanded Monte Carlo sample is
generated. This expanded

Monte Carlo sample can then be convolved with
the conditional

distribution from the new observations. An updated
probability sample

of the state of the system is then generated
by subsampling the

convolution of the expanded Monte Carlo sample.
A number of interesting

issues related to applying Monte Carlo methods
in this context are

addressed.

Presentations at previous meetings have shown
results in low order models.

This talk will quickly review the method and
its use of ensembles. The

method will then be extended for application
in models with many degrees

of freedom and results presented for a spectral
barotropic vorticity

model on the sphere and a grid-point PE model.

The Specification and Use of Synoptically-Dependent
Background Errors

in 3DVAR using information from an Error Breeding
Cycle

The specification and use of synoptically-dependent background errors in 3DVAR using information from an Error Breeding Cycle

A study is currently under way at the UKMO to
use 3D synoptically-dependent

background error modes (SBEMs) within 3DVAR.
Current background errors are

`static', derived via the `NMC' method.
An error-breeding cycle is used to

provide SBEMs which are used in 3DVAR via a new
control variable and cost

function. Details of the methodology and early
results will be presented.

Recent developments in the ECMWF Ensemble Prediction System

Perturbations used in ensemble forecasting ask
for a careful computation.

One of the conditions they should satisfy is
that their statistics

resemble what is known of the analysis error.
Prelimenary results on the

EPS performance will be presented of singular
vectors computed

with a Reduced Rank Kalman Filter. Such singular
vectors are constrained

by the analysis error covariance matrix at initial
time. This is achieved

by using the Hessian of the full 4D-Var costfunction.
Also the use of different

analyses in the EPS or the construction a so-called
consensus analysis

will be discussed.

A Survey of Ensemble Kalman Filters

A variety of ensemble Kalman filters are tested by performing

OSSEs on an idealized barotropic vorticity equation model.

For fixed (non-adaptive) observational networks, the effectiveness
of

the schemes is sensitive to the manner in which the ensemble perturbations

are generated. Important issues that one must consider when creating

an ensemble for data assimilation include the following:

(a) the ensemble perturbations need to be

well resolved by the observational network

in order to avoid spurious representations

of the part of the vector subspace

not represented in the ensemble;

(b) spurious long distance correlations need to be avoided; and

(c) perturbations which obey linear dynamics at the assimilation time

may linearly combine into a likely prediction error more readily than

perturbations which are profoundly non-linear at the assimilation time.

Interestingly, ensemble generation schemes such as those that accurately
sample

the PDF or those that "optimally" estimate the leading eigenvectors
and

eigenvalues of the prediction error covariance matrix do not necessarily

have the desirable characteristics mentioned above. In our particular
OSSEs,

ensemble generation techniques optimized for the PDF or prediction
error

variance perform badly as a basis for data assimilation.

We found that the most effective ensemble Kalman filter is one in which
the

initial perturbations are

based on the eiegnvectors of an isotropic correlation matrix scaled
by

the square root of their eigenvalues. This choice of initial perturbations

yields evolved perturbations with the three desirable characteristics
mentioned

above. This Kalman filter significantly out performs 3-D Var with isotropic

error covariances.

In order to avoid the problem of diminishing ensemble spread,

all of our Kalman filters (except that which is based

on the Houtekamer and Mitchell (1998) scheme) include some sort

of on-line estimation of prediction error covariance parameters

(cf Dee 1995, MWR). This on-line estimation is made easier by an ensemble

transformation approach that also obviates the need to invert large
matrices.

Prospects for an operational ensemble Kalman filter

An ensemble of short-range forecasts can provide
the flow-dependent

covariances of the forecast error, needed by
the Kalman filter.

The finite ensemble size causes the estimated
correlations to be noisy.

To filter small forecast-error correlations associated
with remote

observations, a Schur (termwise) product of the
covariances of the

forecast error and a correlation function with
local support is used.

To solve the Kalman filter equations, the observations
are organized

into batches which are assimilated sequentially.
The ensemble of

background fields is updated at each step, and
thus provides a measure

of the improving quality of the background fields
as more and more

batches of observations are assimilated. For
each batch, a Cholesky

decomposition method is used to solve the linear
system of equations.

Observations from several regions of the globe
may be selected for a

single batch, such that information from different
regions

has zero correlation due to the Schur product.
The linear system

then becomes block diagonal.

A prototype sequential filter has been developed
for atmospheric

data assimilation. Application in real time would
appear to be

feasible.

Some ideas on the possible use of bred vectors in data assimilation

I will briefly review several potential uses such as a) minimization
of the

distance between observations and the first guess along the space of
the bred

vectors (Kalnay and Toth, 1994); b) use of the ensemble spread to increase
the

weight of the observations in areas with large short range ensemble
spread (Pu

et al, W&F, 1997); c) use of the ensemble and the quasi-inverse
of the linear

model to identify "the best regional initial conditions" given the

verification

of a short range ensemble forecast.

Dynamics and statistics of forecast errors in a quasi-geostrophic model.

Results from QG model. Time scale
for collapse

of specified initial ensemble to "the attractor",
characteristics

of perturbations on the attractor.
Instantaneous statistics of

analysis and forecast errors for 3DVAR,
in particular influence of

past dynamics through projection of errors
onto leading Lyapnov

subspace. Singular vectors for approximate
"analysis error covariance

norm" and their differences from energy
SV's.

Spread/Analysis error relationships in a simple model

The relationship between analysis error and analysis spread is examined

in the simple 40 variable model described by Lorenz and Emanuel (Feb
1

1998, JAS). Two types of analysis ensembles are used: an optimal
scheme

based on the ensemble Kalman filter, and a suboptimal scheme with

flow-independent forecast error covariances (similar to 3DVAR).
The

ability of these schemes to estimate the quality of the analysis (as

measured by ensemble spread) is investigated as a function of the number

of observations, the ensemble size, and the size of model and

observation error.

Beyond two overhead projectors, the meeting room will also be equipped
with a computer supporting electronic presentations (Corel, Microsoft,
etc).

*ADDRESS:*

National Centers for Environmental Prediction

Environmental Modeling Center

5200 Auth Rd., NOAA Science Center (formerly
WWB), Rm. 207

Camp Springs, MD 20746

Phone: (301) 763-8000

Fax: (301) 763-8545

*DIRECTIONS:*

NCEP is located in the NOAA Science Center (formerly
known as World Weather Building) in the suburbs of Washington, DC. It is
just off the Capitol Beltway (Interstate 95/495) at exit 7B in Maryland
(Branch Avenue, Route 5), southeast of the capital.

Coming on the Beltway (95/495) from the south, take exit 7B (Branch Ave, MD 5) to the North(west). Immediately after exiting, keep right at the first traffic light. This is Auth Rd; NCEP (8-story building) is on the left corner at the first intersection (at Auth Place, where there is a traffic light). You can either go through this intersection and take an immediate left into the parking lot, or instead take a left at the traffic light, and then take the second driveway on your right into the building's parking lot. In either case, at the gate please identify yourself to the guards through the intercom in order to get admittance.

Coming on the Beltway (I95/495) from the north, take exit 7B (Branch Ave, MD 5) to the North(west). At the end of the exit ramp, there is a stop light at Auth Rd.; NCEP (8-story building) is on the other side of Auth Rd, on your right. You can either go through this intersection and take the second driveway on your right into the building's parking lot, or take a right at the traffic light and then take an immediate left into the parking lot. In either case, at the gate please identify yourself to the guards through the intercom in order to get admittance.

We are approximately 45 minutes drive from Washington Dulles International Airport, 45 mins from Baltimore/Washington Intl. Airport and 35 mins from Washington National Airport. If you are driving, any of the three airports is ok - with a cab, Wash. National would be the best choice.

*REFRESHMENTS:*

Light refreshments (pastries and fruit) will
be brought in before the morning sessions start; Tea, coffee and juice
will be provided throughout the meeting. Cost of refreshments is $2 per
day.

*LUNCH:*

Subs or pizza will be brought in both days. Cost
is $5 per day.

*DINNER:*

An optional dinner will be arranged for the participants
at a nearby restaurant. Details will be announced later.

*TRAVEL SUPPORT:*

IMORPTANT! Those of you who will receive travel
support from NCEP will need to have their air tickets, lodging and car
rental expenses be arranged and paid by us directly. Please contact Mike.Pecnick
@noaa.gov if you haven't done so already. Your other expenses will be reimbursed
after the meeting - please save your receipts and send a copy to Mike Pecnick.