Using a Nature Run as Truth Versus a Succession of Analyses

in Observing System Simulation Experiments.

A nature run is a long, uninterrupted forecast by a model whose statistical behavior matches that of the real atmosphere. The ideal nature run would be a coupled atmosphere-ocean-cryosphere model with a fully interactive lower boundary. Meteorological science is approaching this ideal but has not yet reached it. We still supply the lower boundary conditions (SST and ice cover) appropriate for the span of time being simulated.

The advantage of a long, free-running forecast is that the simulated atmospheric system evolves continuously in a dynamically consistent way. One can extract atmospheric states at any time. Because the real atmosphere is a chaotic system governed mainly by conditions at its lower boundary, it does not matter that the nature run diverges from the real atmosphere a few weeks after the simulation begins provided that the climatological statistics of the simulation match those of the real atmosphere. The nature run should be a separate universe, ultimately independent from but parallel to the real atmosphere.

One of the challenges of an OSSE is to demonstrate that the nature run does have the same statistical behavior as the real atmosphere in every aspect relevant to the observing system under scrutiny. For example, an OSSE for a wind-finding lidar aboard a satellite requires a nature run with a realistic cloud climatology.

A succession of analyses is a collection of snapshots of the real atmosphere. Though (in the case of 4DVAR) the analyses may each be a realizable model state, they all lie on different model trajectories. Each analysis marks a discontinuity in model trajectory. Considering a succession of analyses as truth seems to be a serious compromise in the attempt to conduct a “clean” experiment.

I favor a long, free-running forecast as the basis for defining “truth” in an OSSE.

Tom Schlatter (9/21/2006)

From OSSE chapter in "Data Assimilation: Making sense of Observation" from Springer

Masutani, M., T. W. Schlatter, R. M. Errico, A. Stoffelen, E. Andersson, W. Lahoz, J. S.. Woollen, G. D. Emmitt,L.-P. Riishøjgaard, S. J. Lord : Observing System Simulation Experiments. Data Assimilation: Making sense of observations, to be published from Springer in 2009. The manuscript is posted at

http://www.emc.ncep.noaa.gov/research/JointOSSEs/references/Springer_OSSE_Chapter_PartF_Ch3.081117.doc

3. The Nature Run

The Nature Run is a long, uninterrupted forecast by a NWP model whose statistical behaviour matches that of the real atmosphere. The ideal Nature Run would be a coupled atmosphere-ocean-cryosphere model with a fully interactive lower boundary. However, it is still customary to supply the lower boundary conditions (sea surface temperature, SST, and ice cover) appropriate for the span of time being simulated. Meteorological science is approaching this ideal, but such coupled systems are not yet mature enough to be used for Nature Runs. Although fully coupled systems are available, their usefulness and accuracy for OSSEs is unknown. Preliminary tests, however, suggest that coupled systems may be good enough for operational NWP in near future (Saha et al. 2006; Kistler et al. 2008).

The advantage of using a long, free-running forecast to simulate the Nature Run is that the simulated atmospheric system evolves continuously in a dynamically consistent way. One can extract atmospheric states at any time. Because the real atmosphere is a chaotic system governed mainly by conditions at its lower boundary, it does not matter that the Nature Run diverges from the real atmosphere a few weeks after the simulation begins provided that the climatological statistics of the simulation match those of the real atmosphere. A Nature Run should be a separate universe, ultimately independent from but parallel to the real atmosphere.

3.1 Characteristics of the Nature Run

One of the challenges for an OSSE is to demonstrate that the Nature Run does have the same statistical behaviour as the real atmosphere in every aspect relevant to the observing system under scrutiny. For example, an OSSE for a wind-finding lidar on board a satellite requires a Nature Run with realistic cloud climatology because lidars operate at wavelengths for which thick clouds are opaque. The cloud distribution thus determines the location and number of observations.

The Nature Run is central to an OSSE. It defines the true atmospheric state against which forecasts using simulated observations will be evaluated. This concept deserves more explanation. In 1986, Andrew Lorenc suggested the following definition of the “truth”: the projection of the true state of the atmosphere onto the model basis. As an example, if a spectral model produces a Nature Run, the true atmospheric state might be represented by spectral coefficients corresponding to triangular truncation at total wave number n (Tn) on L vertical levels. Atmospheric features too small to be captured by the model resolution are not incorporated in this truth.

The Nature Run is also the source of simulated observations. For each observing system, existing or future, a set of realistic observing times and locations is developed along with a list of observed parameters. An interpolation algorithm looks at the accumulated output of the Nature Run, goes to the proper time and location and then extracts the value of the observed parameter. If the Nature Run does not explicitly provide an observed parameter, the parameter is estimated from related variables that the model does provide. Because observations extracted from the Nature Run are the same as the defined truth (they are “perfect”), various sources of error must also be simulated and added to form observations with realistic accuracy with respect to the Nature Run itself.

Some OSSEs have used a succession of atmospheric analyses as a substitute for a Nature Run (Keil 2004; Lahoz et al. 2005). A succession of analyses is a collection of snapshots of the real atmosphere. For example, in the case of four-dimensional variational assimilation (4D-VAR, link to theory chapter), although the analyses may each be a realizable model state, they all lie on different model trajectories. The background (first guess) lies on the same model trajectory as the previous analysis because, in 4D-VAR the analysis is a realizable model state (it does not require separate initialization or balancing). Once this background is adjusted by new data in 4D-VAR, the model lies on a new trajectory, which may be close to the old one (the one that the background was on) but is nonetheless different. Each analysis marks a discontinuity in model trajectory, determined by the information content extracted by a DAS from the existing global observing systems (link to Thépaut chapter). Furthermore, residual systematic effects due to the spatially non-uniform and often biased observations, the DAS or the model state, may either favourably or unfavourably affect the potential of new observing systems to improve the forecasts. Thus, considering a succession of analyses as truth seriously compromises the attempt to conduct a “clean” experiment.