Erik Andersson: On the benefits of ensemble techniques used in conjunction with a global 4D-Var system

Thomas M. Hamill: Analysis-Error Covariance Singular Vectors Estimated from an Ensemble Square-Root  ABSTRACT

C. Snyder: Ensemble Kalman filters, radar observations and WRF   ABSTRACT

Jim Hansen: Transformed Lagged Ensemble Forecasting   ABSTRACT

Xuguang Wang: The Ensemble Transform Kalman Filter Ensemble Forecast Scheme

Mozheng Wei: Ensemble forecast experiments using initial perturbations generated by the ETKF method

Zoltan Toth: Tools for evaluating ensembles
 

The ensemble Kalman filter (EnKF) is a four-dimensional data assimilation method. It provides an approximation to the Kalman filter that becomes better as the ensemble size increases. With the arrival of massively parallel computers, it is now possible to use fairly large ensembles and obtain interesting results.

An EnKF is currently being developed at the Meteorological Service of Canada (MSC) for atmospheric data assimilation. Its intended purpose is to generate the initial conditions for the Canadian medium-range ensemble prediction system. Our implementation of the EnKF includes several specific features which do not appear in the traditional Kalman filter. The impact of these on verifications will be shown.     Back to Top

During the 1980's and into the 1990's, operational data assimilation was generally performed using statistical interpolation (SI). The ensemble Kalman filter (EnKF) can be viewed as a generalization of SI. Among its several advantages is that it permits the direct assimilation of radiance data, which SI did not.

An EnKF is being developed for implementation at the Canadian Meteorological Centre. Data assimilation experiments have been performed using the operational (GEM) model and real observations, including microwave radiances from the AMSU-A instrument on the NOAA polar orbiters. The way in which these observations are assimilated and their overall impact will be described.     Back to Top

Studies using idealized ensemble data assimilation systems have shown that the flow-dependent background-error covariances are most beneficial when the observing network is sparse. When observations are very dense, the background-error covariances are not as flow-dependent, and the improvement over schemes with static background-error covariances, such as three-dimensional variational assimilation (3DVar), is not as great. The computational cost of recently proposed ensemble-data assimilation algorithms is directly proportional to the number of observations being assimilated. Therefore, ensemble-based data assimilation should both be more computationally feasible and provide the greatest benefit over current operational schemes in situations when observations are sparse. Reanalysis before the radiosonde era (pre-1948) is just such a situation.

The feasibility of reanalysis before radiosondes using an ensemble square-root filter (EnSRF) is examined. Real surface observations for 2001 are used, subsampled to resemble the density of observations we estimate to be available for 1915. Analysis errors are defined relative to a 3DVar analysis using several orders of magnitude more observations, both at the surface and aloft. We find that the EnSRF is computationally tractable and considerably more accurate than other candidate analysis schemes which use static background error covariance estimates. Of the surface observations, surface pressure provide the most useful information. We conclude that a Northern Hemisphere reanalysis of the middle and lower troposphere during the first half of the 20th century is feasible. Expected Northern Hemisphere analysis errors at 500 hPa for the 1915 observation network are equivalent to current 2.5 day forecast errors.     Back to Top

This talk will discuss numerical aspects of the implementation of a parallel ensemble Kalman filter (EnKF) for a 1/3-degree, 27-layer version of the Neptune quasi-isopycnal global OGCM. This is the version used by the NASA Seasonal-to-Interannual Prediction Project (NSIPP) in its coupled-model seasonal forecasts and the purpose of the implementation is to use the EnKF to initialize the ocean model prior to running a coupled-model ensemble forecast.

Aspects of the implementation that will be discussed include (1) the random-forcing procedure used to simulate the process noise, (2) the ensemble initialization, (3) the parallelization and localization of the global analysis and (4) the estimation of the forecast-model bias using ensemble-derived background-error covariance estimates.

Results from experiments involving the assimilation of altimeter data from TOPEX/Poseidon and of XBT temperature data will be presented. The experiments include both the separate assimilation of the altimeter and temperature data where the other data set is retained for the purpose of cross-validation and their joint assimilation. The performance of the filter is also assessed by measuring the improvements of the coupled-model hindcast skill over the skill of a similar hindcast in which a univariate temperature optimal interpolation (OI) algorithm is used to initialize the ocean model.     Back to Top

Recent studies have shown that, when the Earth's surface is divided up into local regions of moderate size, vectors of the forecast uncertainties in such regions tend to lie in a subspace of much lower dimension than that of the full atmospheric state vector of such a region. In this presentation we show how this finding can be exploited to formulate a potentially accurate and efficient data assimilation technique. The basic idea is that, since the expected forecast errors lie in a locally low dimensional subspace, the analysis resulting from the data assimilation should also lie in this subspace. This implies that operations only on relatively low dimensional matrices are required.     Back to Top

The potential advantages, properties, and implementation requirements of the method of Ott et al. are illustrated by numerical experiments on a 40-variable Lorenz model. It is found that near-optimal performance can be achieved with very modest computational cost. Preliminary results for the NCEP GFS are also presented.     Back to Top

A main advantage of the local ensemble Kalman filter, proposed by Ott et al., is that the data assimilation is done locally in a manner allowing massively parallel computation to be exploited. In this presentation we review potential implementation strategies that can be considered on the different available computer architectures. Timing results for a distributed memory implementation of the method for the T62 28-level version of the NCEP GFS are also presented.     Back to Top

Minimum error variance data assimilation requires the ability to propagate the error covariance of the state estimate and to accurately compute how the assimilation of observations changes the error covariance of the state estimate. For atmospheric data assimilation, adequate error covariance propagation can be achieved via the 6-12 hr integration of an ensemble forecast with as many members as the rank of the error covariance matrix. 2084 1.6GHz Pentium IV processors can produce an ensemble of 100,000 6 hr T79L30 NOGAPS forecasts in 6 hrs. In November 2002, vendors were selling pre-configured Beowolf clusters containing 1000 cpus each with clock speeds of 2.2 GHz for less than a million US dollars. This suggests that full rank moderate resolution error covariance propagation will soon be possible in real time. The purpose of this paper is to investigate and understand highly scalable, computationally efficient techniques by which the mega-dimensional ensemble based error covariance information can be used to compute analysis increments and ensembles of analyses that accurately represent analysis error covariance. A toy model of high dimensional data assimilation is used to illustrate the pros and cons of various approaches. In particular, the toy model is used to illustrate a Scalable Ensemble Transform Kalman Filter (SETKF). The examples indicate that the SETKF has the potential to enable rapid error covariance updates and effective pre-conditioning of the conjugate gradient descent algorithm used to find the analysis increment. It also provides a convenient and relatively accurate framework for data quality control and adaptive sampling.     Back to Top

A common approach in analysis step of an Ensemble Kalman Filter (EnKF) and Ensemble square-root filter (EnSQRTF) is to explicitly obtain the linear solution of the Kalman Filter analysis equation. Since the analysis problem is fundamentally nonlinear, a generalization of existing ensemble data assimilation methodologies, based on the optimization of the non-quadratic cost-function, will be explored. The algorithm can be easily coupled with any EnKF or/and EnSQRTF method.

It will be shown that the effort to obtain the Kalman gain in existing EnKF and EnSQRTF approaches is equivalent to building a "near-perfect" Hessian preconditioing matrix. As a consequence, only a couple of minimization iterations is sufficient to achieve a satisfactory convergence. The analysis results on a simple 1-dimensional domain with highly nonlinear observation operators will be shown.     Back to Top

The use of small ensemble filters in large operational prediction models is now well-established, at least in the absence of model systematic error. Dealing with systematic error continues to present challenges since the underlying statistical models are predicated on unbiased prior estimates. We will examine the results of ensemble filter assimilations in low-order versions of NCEP's global model and discuss methodologies for dealing with systematic error in this context.     Back to Top

Bias in models and observations plague all meteorological data assimilation systems. The only serious attempt at bias correction has been with satellite radiance data, where the bias is explicitly estimated. Here a new method is demonstrated which allows for implicit representation of both model and observation bias for all types of data. The method depends on constructing "floating superobs" by grouping observations in difference pairs, thus focusing on gradients while eliminating by construction the mean value. This allows bias in the primary variables to develop naturally, with no need for explicit bias computation. Of course there are biases in gradients too, but this is an attempt to remove the most visible zero order part of the bias.

Some results will be shown using Anderson's form of an ensemble filter with the NCEP spectral model, run at T62, 28 levels. This is a convenient test bed, because of the simplicity of the method, but floating superobs can be applied to any type of data assimilation procedure. It is not dependent on using ensemble methods.     Back to Top

Initial conditions for ensemble forecasts should sample the distribution of possible analysis states. Suppose we seek to explain the maximum amount of forecast-error variance with a limited ensemble of forecasts. Further, suppose analysis errors are normally distributed and grow linearly. Then it can be shown that the most forecast-error variance can be explained if initial perturbations span the subspace of the leading analysis-error covariance singular vectors (AEC SVs). These AEC SVs represent perturbations that have initial size and structure that are consistent with the analysis error covariances and that grow the most rapidly during the subsequent forecast.

Generating AEC SVs is easier said than done, for current data assimilation schemes are designed to produce an analysis, but they do not directly provide information on the error covariance of that analysis. Consequently, many approximate approaches have been tried. The method that is used operationally at the European Centre for Medium-Range Weather Forecasts is to generate "total-energy" singular vectors (TE SVs) for perturbations. These are the structures that grow the fastest and whose initial and final size are measured in a total-energy norm.

Recent work with ensemble-based data assimilation methods permits us to generate flow-dependent analysis-error covariance statistics and from them approximate AEC SVs. Comparing the structure of these two types of singular vectors, it is apparent that the initial structure of TE SVs look very little like the structure of AEC SVs. For example, the leading TE SV tends to have nearly no amplitude near the surface nor the tropopause; the maximum amplitude is in the middle troposphere. AEC SVs tend to have less amplitude in the middle troposphere and more at the tropopause and the surface.

These differences have consequences if TE SVs are used to generate short-range ensemble forecasts; the forecast error projects much less into the basis of the leading TE SVs than into the basis of the leading AEC SVs.     Back to Top

Some recent results using an ensemble Kalman filter (EnKF) to assimilate simulated Doppler radar observations for both supercells and squall lines will be discussed. The EnKF is effective in both cases. [See http://box.mmm.ucar.edu/individual/snyder/sz.pdf for more details on the supercell experiments.] The squall-line experiments include attempts to infer large-scale information (e.g. the environmental sounding) given the radar observations.

Recent efforts to implement a prototype EnKF for the WRF model will also be briefly reviewed.     Back to Top

Multi-model ensemble forecasts have been shown to outperform single model ensemble forecasts in numerical weather prediction (NWP) for some measures. One interpretation of this result is that multi-model ensembles are better able to sample from the sources of uncertainty present in NWP. But many technical difficulties exist in maintaining the large number of independent models needed for appropriate ensemble sizes within a multi-model environment. To address this, a method of augmenting existing multi-model ensembles to produce a larger forecast ensemble is presented.

The technique of lagged ensemble forecasting (LEF) relies upon gathering forecasts that are verified at the same time, but were launched at sequentially different times. This effectively increases the total forecast ensemble size of each model, but the interpretation of the resulting forecast ensemble is frustrated by different ensemble members possessing different amounts of observational information: forecast ensemble members cannot be interpreted as equally likely draws from the same distribution. This limitation can be overcome without the need of re-running any NWP models. The ensemble transform Kalman filter (ETKF) framework is applied to existing ensemble forecasts each time new observations become available. Resulting ensemble alterations are propagated forward over the forecast period without running any NWP models. This transformed LEF (TLEF) technique can be applied to both single model and multi-model ensembles. Examples are given using simple models and using output from existing NWP models.     Back to Top

Back to Top

AGENDA

Tuesday, March 18
 

8:00-8:30	Refreshments
8:30-8:45	Welcome remarks (Steve Lord, Director of EMC)
TOWARD OPERATIONAL APPLICATIONS
8:45-9:35	Peter Houtekamer Towards an operational ensemble Kalman filter
9:35-10:05	Herschel Mitchell Assimilation of TOVS-1b radiances with EnKF
10:05-10:30	BREAK
10:30-11:00	Jeff Whitaker Reanalysis without Radiosondes using and ensemble filter
11:00-11:30	Christian L. Keppenne Initialization of a high resolution global OGCM with a parallel ensemble Kalman filter and application to seasonal forecasting
11:30-12:00	Discussions
12:00-1:00	LUNCH sandwiches brought in, make selection by morning breaktime
NEW APPROACHES
1:00-1:30	Andrew Lorenc Comparison of EnKF and 4DVAR
1:30-2:00	Ed Ott A Local Ensemble Kalman Filter: Theory
2:00-2:30	I. Szunyogh A Local Ensemble Kalman Filter: Numerical Experiments
2:30-3:00	E. Kostelich A Local Ensemble Kalman Filter: Efficient Implementation Strategies
3:00-3:20	BREAK
3:20-3:40	Craig H. Bishop Mega-dimensional Data Assimilation and a Scalable Ensemble Transform Kalman Filter
3:40-4:10	Milija Zupanski The EnKF/EnSQRTF nonlinear analysis solution
4:10-5:00	Discussion
5:00	Adjourn for Dinner arrangements TBD

Wednesday, March 19

Thursday, March 20

Back to Top