The ultimate measure of the utility of weather forecasts is arguably the economic benefit associated with their actual use in the daily decision making process of individuals or different organizations. Users of weather forecasts either do, or do not take action (e. g., introduce protective action to prevent/reduce weather-related loss), depending on whether a particular weather event is forecast or not. Cost-loss analysis of different complexity can be applied to evaluate the economic impact of the use of weather forecasts on the users. Here we present a relatively simple cost-loss model that, after some simplifications, can generally be applied in most cases.
If, based upon a particular weather forecast users take a preventive action, and the (predicted) harmful event does not occur, they incur a cost associated with their action. If the event occurs, and following a successful forecast they acted to reduce the loss, they suffer a mitigated loss, while if due to a failed forecast they take no action, they incur a loss. In the event the users take no action because the forecast was negative, and the harmful event does not occur, there is no cost to the users. Assuming that climatological information has zero value, and that perfect forecasts have a value of one, one can plot the value of different forecast systems as a function of the users' cost/loss ratio. On such a plot all potential users are represented with their particular cost/loss ratio. For simplicity, we assume that the mitigated loss is equal to the cost of the protective action (i. e., all weather related losses can be avoided if protective action is taken).
In the example below we compare the economic value of the MRF T126 resolution control forecast to that of a 14-member set of the T62 horizontal resolution ensemble. These two forecast systems use comparable computational resources. The users of the ensemble forecasts have an option of acting if the forecast probability is above any particular level (determined by the number of members predicting the event). It can be shown that their optimal choice of this critical probability level is equal to their cost/loss ratio. The higher their losses with respect to their protective cost, the more inclined they are to act, even when there is only a small chance of a harmful event occurring. In contrast, the user of a single deterministic control forecast has no choice but to act when the model forecast calls for a harmful event. In the illustrative example below, events are defined as the 500 hPa geopotential height at gridpoints over the Northern Hamisphere extratropics being in any of 10 climatologically equally likely bins.
The economic value comparison results indicate that even at short, 24-hour lead time, most users, except those with loss/cost ratios in a relatively narrow band between 2 and 5, can realize more economic value when using the ensemble forecasts. At and beyond 72 hours lead time all users are better off using the ensemble system than the high resolution control forecasts. Furthermore, the range of cost/loss ratios for which the forecasts exhibit value, compared to using climatological information, is substantially widened, indicating that a much larger group of users can benefit from the ensemble forecasts as compared to the high resolution control forecasts.
One can draw the following conclusion from the above results: At and beyond 3 days lead time, the direct model output form the ensemble forecasts offers more value than that from the higher resolution control forecast. It is absolutely critical for the users of weather forecasts to have access to probabilistic information that allows them to optimally tailor the forecasts to their particular cost/loss situtation. A much larger group of users can benefit from using forecasts (as compared to relying on climatological information), and their economic benefit is usually higher when the forecasts are probabilistic in nature. These findings confirm results found earlier at ECMWF and the UK Met. Office. If one has access only to a control forecast, that can also be converted to full forecast probability distributions based on past verification statistics. Such postprocessed forecasts of varying levels of probability would have more value than the dichotomous probabilistic guidance (based on a yes/no forecast, associated with two given probability levels at any particular lead time) evaluated here.
Note, however, that since beyond 3 days lead time all users realize more benefit from using direct output from the ensembles than from the control, this is also true for the users whose cost/loss ratio is exactly the probability associated with the control "yes" forecast (which is the reliability of the control forecasts, i.e., the conditional observed frequency of a weather event given it was predicted by the control forecast). The conversion of the control forecasts into full forecast probability distributions would make no difference for these users, therefore users with this cost/loss ratio are definitely not able to benefit from using the control forecast, not even after their conversion into probability distributions. This result is consistent with earlier probabilistic verification statistics and is due to the ensemble's ability to well indicate flow dependent, day-to-day variations in the expected reliability of the forecasts. How much the economic value of the control forecasts can be increased by postprocessing them via a coversion to full probability distributions is currently being investigated.
Acknowledgements. We benefitted from discussions with Ken Mylne and Mike Harrison of the UK Met. Office and David Richardson of ECMWF.
