Forecast Evaluation: Statistical Techniques for Decision Support

Elizabeth C Weatherhead
University of Colorado
2:30 pm Sept 17 in Room 2890

Changes in models, observations or even computing approaches are all likely to result in changes to forecasts.  When the forecast change is small, identifying if it is an improvement can be challenging and expensive.  Decision support statistics can help identify even very small improvements.  By making use of paired forecasts, and respecting the day-to-day and even hour-to-hour autocorrelation in weather, forecasts and forecast errors, identification can be made more efficiently and with higher likelihood of long-term success.

Perhaps just as importantly, statistical input can help design evaluation runs to minimize cost while maximizing the power of the results.  For instance, the number of runs can be reduced significantly, if forecasting evaluation techniques are determined in advance.  These decision support techniques can be used to determine if an added set of observations is significant, whether a computer change is systematically more harmful, or the impact of even small changes to physics packages or model cores.  Techniques will be reviewed and sample results shared.