MODEL VERIFICATION SYSTEM AT NCEP

                         Keith F. Brill

			 30 March 1999

                       NOAA/NWS/NCEP/EMC/MMB
                       W/NP22, RM 205, WWBG
                       5200 Auth Road
                       Camp Springs, MD   20746-4304

A system for verifying various NCEP operational models at the locations
of observations has been developed.  The system consists of two components:
a generator of mean values over areas and a computation and display
program.  The latter is designed to allow great flexibility in combining
the mean values for various graphical displays.

The mean value generator bilinearly interpolates the forecast gridded
values to the locations of data.  Then, over some defined region of
space, the matching pairs of forecast and observed values are used to
compute the following:

    MEAN [f], MEAN[o], MEAN[f*o], MEAN[f*f], and MEAN[o*o],

where f and o represent forecast and observed values, respectively.
For winds having components u and v, the means are:

    MEAN [uf], MEAN [vf], MEAN [uo], MEAN [vo], MEAN [uf*uo+vf*fo],
    MEAN [uf*uf+vf*vf], MEAN [uo*uo+vo*vo] 

where f and o denote forecast and observed components, respectively.
These number of observation/forecast pairs contributing to each set of
means is stored along with the means themselves in an archive.  Each data
record contains identifying information as well.  These means are
convenient because they can be multiplied by the data count to create
a partial sum that can be added to other partial sums, allowing the
data to be combined as a function of the identifying information and
displayed in many useful ways giving different perspectives on the nature
of model performance.

The second component of the system operates in three steps.  First, the
user provides search conditions describing what data is to be extracted
from the archive and how the means are to be combined at each value of
the independent variable of a display trace.  Second, the search is
launched to find the data in the archive and combine the values, which
are first turned back into sums by multiplying by the data count.  The
third element of this system allows the user to use the means from the
data combination in the second step to compute any one of the following:
standard deviation or variance of forecast values, same of observed values,
root mean square error (RMSE), bias, covariance, or correlation.  Once,
computed, a graphical display of the chosen statistical quantitiy is
plotted.  Of course, the means themselves may be displayed.

A similar treatment allows the second component of this system to
compute and display skill scores.  In this case, the archive consists of
data point counts, fraction of points at which an event is forecast to
occur, the fraction of the former that correspond to an observation of
that event (hits), and the fraction of points at which the event is
observed to occur.