Forecast Evaluation: Statistical Techniques for Decision Support
Elizabeth C Weatherhead
University of Colorado
2:30 pm Sept 17 in Room 2890
Abstract: Changes in models, observations or even computing
approaches are all likely to result in changes to forecasts. When
the forecast change is small, identifying if it is an improvement can
be challenging and expensive. Decision support statistics can
help identify even very small improvements. By making use of
paired forecasts, and respecting the day-to-day and even hour-to-hour
autocorrelation in weather, forecasts and forecast errors,
identification can be made more efficiently and with higher likelihood
of long-term success.
Perhaps just as importantly, statistical
input can help design evaluation runs to minimize cost while maximizing
the power of the results. For instance, the number of runs can be
reduced significantly, if forecasting evaluation techniques are
determined in advance. These decision support techniques can be
used to determine if an added set of observations is significant,
whether a computer change is systematically more harmful, or the impact
of even small changes to physics packages or model cores.
Techniques will be reviewed and sample results shared.