The New Precipitation Verification Package:

Changes and User Guide

 

Last Modified: 19 March 2008

Julia Zhu  Ying Lin

 

The NCEP/EMC precipitation verification package was implemented in production on Dec 11th, 2007. 

 

1.      Changes to the precipitation VSDB on Tempest

24h (12Z-12Z) verification is done for all models verified, and 3-hourly verification is done for selected NCEP operational/parallel models.  Under the old set-up, the 3-hourly and 24h hourly verification database were separate.  In the new set up, the 3-hourly VSDB files are appended to the 24-hourly VSDB files, e.g. nam_20080125.vsdb contains both 24h verification (valid at 12Z 25 Jan 2008) and 3-hourly verifications (valid at 00, 03, 06, ... 21Z 25 Jan 2008).  

 

Data prior to 1 Jan 2008:

3-hourly VSDB: /export-4/tempest/wd22yl/vsdb_prcp3

24-hour VSDB: /export-4/tempest/wd22yl/vsdb

Data from 1 Jan 2008: both 3-hourly and 24-hour VSDBs are in

/export-4/tempest/wd22yl/vsdb

Note: the 24-hour VSDBs used to be in /export-4/tempest/wd22yl/vsdb_prcp24.  The vsdb_prcp24 directory has been renamed 'vsdb'.  Since 1 Jan 2008 'vsdb_prcp24' has been a symbolic link pointing to 'vsdb', so people's old FVS scripts would still work.  This symbolic link will be removed after 30 Jun 2008.  Please point your VSDB_DATA to

/export-4/tempest/wd22yl/vsdb prior to 30 Jun. 

 

2.      Operational scripts, fixed fields, parameter files

2.1.    Scripts

 

The precip verification package contains four steps as shown in the following:

 

1) Extract precip files from model output:

./sms/jverf_precip_getppt.sms

  |_./jobs/JVERF_PRECIP_GETPPT.sms.dev

    |_./scripts/exverf_precip_getppt.sh.sms

      |_./ush/verf_precip_getpptfcst.sh

      |_./ush/verf_precip_getpptndas.sh

 

2) Run 24h verification:

./sms/jverf_precip_verfgen_24h.sms

  |_./jobs/JVERF_PRECIP_VERFGEN_24H.sh.dev

    |./scripts/exverf_precip_verfgen_24h.sh.sms

      |_./ush/nam_cpc2grbrfc8.sh (if necessary)

      |_./ush/verf_precip_verf24.sh

      |_./ush/verf_precip_medley24.sh

 

3) Run 3-hourly verification:

./sms/jverf_precip_verfgen_03h.sms

  |_./jobs/ JVERF_PRECIP_VERFGEN_03H.sms.dev

    |_./scripts/exverf_precip_verfgen_03h.sh.sms

      |_./ush/verf_precip_verf03.sh

 

4) 24h Model vs. Analysis plots:

./sms/jverf_precip_plotpcp.sms

  |_./jobs/JVERF_PRECIP_PLOTPCP.sms.dev

    |/scripts/exverf_precip_plotpcp.sh.sms

      |_./ush/nam_pcpn_plotpcp.sh

 

 

SMS Scripts

 

These are the job submission scripts. These should be the only scripts you will need to modify in order to run your own verification. The following variables will need to be defined if you don’t want to use the defaults:

 

########################################################################

# The following variables can be set by users:

# If not set here, will use the default values set

# in the J-Job scripts

# tmpdir -- working directory

# HOMEverf_precip -- scripts, codes, parm and fix files directories

# COMIN -- input file directory

# COMOUT -- output file directory

# COMCHKPT -- directory to save the checkpoint files

# COMVSDB -- directory to save the vsdb records

# COMHRLY -- directory where the obs precip analysis(rfc or st2) files are copied from

# COMHRLYm1 -- directory where the obs precip analysis(rfc or st2) files are copied

#          from the previous day, should be the same as COMHRLY if not real-time run

# PDY -- the day for the verification, useful for retrospective runs where one

#        needs to set it to a specific date, the format is "yyyymmdd"

########################################################################

#export tmpdir=

#export HOMEverf_precip=

#export COMIN=

#export COMOUT=

#export COMVSDB=

#export COMCHKPT=

#export COMHRLY=

#export COMHRLYm1=

#export PDY=

 

There are four SMS scripts for the precip verification run, where the next-step is dependent on the completion of the previous one, except for step 2 and 3, which could be submitted at the same time.

 

jverf_precip_getppt.sms

jverf_precip_verfgen_24h.sms

jverf_precip_verfgen_03h.sms

jverf_precip_plotpcp.sms

 

J-Job Scripts

 

These are the scripts where most of the environment variables are defined, including the Script/Code directories, the working directory, input/output directories.

 

There are also four J-Job scripts (in capital case as in operations) which are called by the SMS scripts:

 

JVERF_PRECIP_GETPPT.sms.dev

JVERF_PRECIP_VERFGEN_24H.sms.dev

JVERF_PRECIP_VERFGEN_03H.sms.dev

JVERF_PRECIP_PLOTPCP.sms.dev

 

EX-Scripts

 

These are the main driver scripts for the four steps, and they are

 

exverf_precip_getppt.sh.sms

exverf_precip_verfgen_24h.sh.sms

exverf_precip_verfgen_03h.sh.sms

exverf_precip_plotpcp.sh.sms

 

USH-Scripts

 

These are called the utility scripts, they are the scripts that are called by the driver scripts and do the real work. And the following scripts are used by the precip verification run:

 

1) verf_precip_getpptndas.sh and verf_precip_getpptfcst.sh

     They are called by the exverf_precip_getppt.sh.sms to extract precipitation fields from the model forecast output.

 

2) verf_precip_verf24h.sh

    This script generates the 24h precip verification products.

 

3) verf_precip_verf03.sh

     This script generates the 03h precip verification products

 

4) verf_precip_plotpcp.sh

     This script plots the 24h model precipitation using GEMPAK

 

5) verf_precip_indexplot.sh

     This script plots and puts together the precip analysis and model forecast precip files  side-by-side.

 

6) verf_precip_ywgrib.pl

     This is a utility script written by Ying Lin. It reads a "wgrib -V -PDS gribfile" and creates a gribscan-like inventory. It is used in one of the source code.

 

2.1.    Fixed fields

There are five fix files used in this precip verification run:

verf_precip_regmask_211.Z

verf_precip_regmask_212.Z

verf_precip_regmask_218.Z

verf_precip_westmask_218.Z

verf_precip_eastmask_218.Z

2.2.    Parameter files

2.2.1.  Config files

verf_precip_ppt_config

verf_precip_03h_config 

verf_precip_24h_config

 

These config files contain a list of models to run the verification. You can turn on/off a model run by setting the “get_$modelx” to 1/0. If your model (say “modelx”) is not included, add the following line to each of the config files:

 

export run_modelx=1

 

If you only want to run verification for your model, set the value to 0 for the other models

 

2.2.2.  verf_precip_input_domains

This file contains a list of models and tells exverf_precip_getppt.sh.sms how to extract precipitation fields from modelx's output files.  For the nam model, it looks like this:

 

nam| | | |000|/com/nam/prod/nam._DAY_/nam.t%{cyc}%z.egrdsf%{fhr}%.tm00|_DAY_/nam_|00 06 12 18|03|84|03|

 

10 fields separated by the vertical bars are:

1st entry:    model name

2nd entry:   "convert" if it is necessary to do a conversion from the model input grid to another grid

3rd entry:   the grid to convert to if the 2nd entry is “convert”.

4th entry:   grid number of the precipitation field. If the output contains only precipitation fields in one single grid (e.g. NAM), you can use '000'. The NGM precipitation field is outputted on multiple grids in its output file, i.e., ngm.t12z.pgrb.f24, so the 2nd line of verf_precip_input_ngm is '104' because we wanted NGM precipitation field on G104).

5th entry:   model output file format including the path name

6th entry:   prefix for output grid files that contain the extracted precip data

7th entry:    model cyles that are used for the precip data extract

8th entry:    starting model forecast hour for the precip data extract

9th entry:   ending model forecast hour for the precip data extract

10th entry:  hour interval for the precip data extract

 

If this file does not contain your model, you can just add it to the bottom of the file.

 

2.2.3.  verf_precip_verf24_domains and verf_precip_verf03_domains

These files contain a list of the models to run for the 24h and 03h verification, and provide the input fields for the verf_precip_verf24.sh and verf_precip_verf03.sh. Here’s an example for the NGM model:

 

ngm|24|211 212 218|00 12|12|48|

 

Six fields separated by the vertical bars are

1st field: model name

2nd field: verification time period (24h hours)

3rd field: grid(s) over which the verification is to be done (both model forecast and verifying analysis are mapped to the verifying grid(s) using copygb)

4th field: model run cycles (00 and 12Z)

5th field: precipitation 'bucket' length (how often the precipitation bucket is reset to zero).  In the case of NGM, the bucket length is 12 hours, i.e. the 09h forecast file contains 00-09h forecast, the 12h forecast file contains 00-12h forecast, the 15h forecast file contains 12-15h forecast.

6th field: maximum forecast length for NGM (48 hours).

7th field: Nests (optional)

 

And for the NAM model, it looks like this:

 

nam|24|211 212 218|00 12|12|84|

nam|24|211 212 218|06 18|3|84|

nam|24|218|00|12|48|west

nam|24|218|06|3|48|west

nam|24|218|00 12|12|48|east

nam|24|218|18|3|48|east

 

Let's look at the first two lines of this file.  For NAM, the precipitation bucket length is 12 hours for 00 and 12Z runs, 3 hours for 06 and 18Z runs, that is why we need two separate lines to do verification for both cycles.  Lines 3-6 are verification on 'specialty' grids, i.e. the 'western' and 'eastern' domains of HiRes windows runs.  The seventh field in the parameter card, not mentioned in the NGM card above, is an optional field that tells the verification program that the verification is to be done on a special domain.  Without the 7th field, the verification would be performed on ConUS and its 14 sub-regions.

 

3.      Where to get the scripts and Codes

 

You can check out the whole precip verification package (scripts, codes, fix/parm files) from the SVN server on

 

https://svn.ncep.noaa.gov/emc/verif/g2g_precip

 

 

4.      What you need to change to make your own verification run

 

There are only a few files you need to modify to make your own run:

 

1)       The four SMS scripts

 

       If you do not want to use the defaults setup of the input/output directories, you can specify them in the SMS scripts and uncomment the variables.

 

2)      The three config files to turn on/off or add your new model (if it does not exist), and the three domains parameter files to add/change variables related to your model.

 

  verf_precip_ppt_config

  verf_precip_24h_config

  verf_precip_03h_config

  verf_precip_input.domains

  verf_precip_verf24.domains

  verf_precip_verf03.domains

 

5.      How to run the verification

 

You can submit the four jobs to the loadlevelor using “llsubmit” followed by the SMS script name. It is preferable that you submit the next one upon the completion of the previous, except for the 2nd and 3rd jobs (verfgen_24h and verfgen_03h), which could run simultaneously in the system.