EFSO and DFS diagnostics for JMA’s global Data Assimilation system: their caveats and potential pitfalls

Daisuke Hotta
noon May 20 in Room 2155

Diagnostic methods have recently been devised that allow estimation of observational impact on a Data Assimilation System (DAS). Such diagnostics not only enable quantification of the “value” of observations but can also provide clues to improving the diagnosed DAS. Two such diagnostic methods, Ensemble Forecast Sensitivity to Observations (EFSO; Kalnay et al. 2012) and Degrees of Freedom for Signal (DFS; Liu et al. 2009), have been implemented to the EnKF component of JMA’s pre-operational global hybrid DAS. This talk will present findings achieved through these diagnostics. Particular emphasis will be placed on caveats and potential pitfalls in interpreting their results.

The first part of the talk will present EFSO implemented at JMA. It was found that the forecast error reduction estimated by EFSO accounts for only 20% of the actual forecast error reduction. In order to understand mechanism behind this underestimation, we conducted diagnosis where the forecast error vector is decomposed into the column and null space of the ensemble forecast perturbations, recognizing that portions of forecast errors that are in the null space are discarded during EFSO computation. The result indicated that 80% of the forecast errors are in fact in the null space, explaining the mechanism of the underestimation. Reasons for why so little of the forecast error is accounted for by the ensemble will also be discussed.
In the second part, I will show that that the information content extracted from observations by EnKF as quantified by DFS is an order of magnitude smaller than that by 4D-Var. This underestimation is particularly conspicuous to dense observations. Theoretical consideration reveals that, this is because, in EnKF, DFS can never exceed the size of the background ensemble, limiting the amount of information extractable from observations if the number of observations is much larger than the ensemble size. Implications of this DFS underestimation on broad aspect of EnKF, including localization, inflation and observation thinning, will also be discussed.