The New GRID2OBS Verification System:
User Guide
Last Modified: 24 Oct 2008
Julia Zhu
Email: yuqiu.zhu@noaa.gov
This is a document on how to run the gridtobs verification using this new version of scripts and codes only, for more information on how this verification system works, please refer to Perry Shafran’s document http://www.emc.ncep.noaa.gov/mmb/papers/shafran/gridtob.pdf
This version of codes and scripts will become an official
version to be implemented in NCEP’s production suite.
It contains the gridtobs verifications on
1.
Flow Structure of this system
./sms/jverf_gridtobs_nam_00.sms (the SMS script, an example only)
./jobs/JVERF_GRIDTOBS_FITS.sms.dev (J-Job script, one script shared by all the models)
./scripts/exverf_gridtobs_fits.sh.sms (Ex-script, one script shared by all the models)
./ush/verf_gridtobs_fits.sh (Main script that executes the EDITBUFR, PREPFITS and GRIDTOBS steps)
./ush/verf_gridtobs_getbufr.sh (This script decides which “prepbufr” file to be copied over)
./ush/verf_gridtobs_getbufr_aqm.sh (This script is special to the AQM model only)
./ush/verf_gridtobs_getmodinput.sh (This script obtains the model input grib files)
SMS Scripts
These are the job submission scripts. We should have one SMS script for each model on each cycle. In production, the gridtobs verification are to be divided and run in 8 cycles (00, 03, 06, 09…21). There is only one submission script that will process the EDITBUFR, PREPFITS and GRIDTOBS all together in one job. These scripts I have in the subversion do not assume to the jobs to be submitted through an SMS server, but you will still need to modify the following variables in order to run in your own account:
# @ output = /ptmp/wx22yz/com/output/prod/today/verf_gridtobs_gfs_12.o$(jobid)
# @ error = /ptmp/wx22yz/com/output/prod/today//verf_gridtobs_gfs_12.o$(jobid)
# @ initialdir = /ptmp/wx22yz/tmpnwprd
/meso/save/wx22yz/verif/nwprod/jobs/JVERF_GRIDTOBS_FITS.sms.$envir
You can just substitute “wx22yz” with your own account or change to whatever directory you would like to save those files. You can also change the “class” and “group” if necessary.
J-Job Scripts: JVERF_GRIDTOBS_FITS.sms.dev
This is a generic script to set up various environment variables. At the beginning of this script, I put a “key-switch” variable called “RUN_ENVIR” to decide whether this job is to be run in a “development” environment (EMC) or a production environment (NCO). Depending on the input for this environmental variable, the job then reads in a parameter file called “verf_gridtobs_config” which actually defines the working directory, the jlogfile, the locations of scripts, parameter files and executables, the input/output directories…and so on. You will need to review this parm file and make necessary modifications.
EX-Scripts: exverf_gridtobs_fits.sh.sms
This is a generic driver script. It reads in the parameter
file called “verf_gridtobs_modelinfo”, which
specifies the “regions” of the verification and the verification length for a
certain model. For example, for the
USH-Scripts:
These are called the utility scripts, they are the scripts called by the driver scripts and do the real work.
1)
verf_gridtobs_fits.sh
This is the main script called by exverf_gridtobs_fits.sh.sms to process the EDITBUFR, PREPFITS and GRIDTOBS steps.
2)
verf_gridtobs_getbufr.sh
and verf_gridtobs_getbufr_aqm.sh
They are called by verf_gridtobs_ fits.sh to obtain the appropriate prepbufr file (the OBS file) for the verification.
3)
verf_gridtobs_getmodinput.sh
This script determines and retrieves the necessary model output files to be verified against the observations.
4)
verf_gridtobs_ens_sref.sh
This is the driver script to calculate the VSDB records for the ensemble verification (GRIDTOBS_ENS) for the SREF model only. It decides the number of pieces for the job to run on POE(currently I divide it into 33 pieces for the 00/12Z cycles, and 27 pieces for all other cycles , which means this job will need 33 (00/12Z) and 27 (other cycles) tasks to run. It is called by the J-Job script JVERF_GRIDTOBS_FITS.sms.dev. This script is specific to the “SREFENS” and “SREFENSX” runs only.
5)
verf_gridtobs_srefens.sh
This script is a sub-script to verf_gridtobs_ens_sref.sh. It is the main script that calculates the VSDB records for each “piece” of the ensemble run. This script is specific to the “SREFENS” and “SREFENSX” runs only.
6)
verf_gridtobs_aqmmax.sh
This script generates the VSDB records for the “max” and “ave” verifications for the OZONE and PM runs. It is called by the J-Job script JVERF_GRIDTOBS_FITS.sms.dev. This script is specific to the “OZONEMAX”, “PMMAX” and “PMAVE” verifications only.
PARM Files:
1)
verf_gridtobs_config (common to all models)
This file is used in the J-Job script. It defines the locations of the scripts, parameter files and executables, the working directory, the input/output directories, the jlogfile and the values for the several variables such as SENDCOM, SENDDBN, SENDSMS. The usage of this parameter file will make it possible for the developers and the production to share the same J-Job script with only a change in a key-switch variable called “RUN_ENVIR”. For the development run, we should use “RUN_ENVIR=dev”.
2)
verf_gridtobs_modelinfo
(common to all models)
This file is used in the EX-script. It defines the “regions” or “grids” to be verified for a specific model and the verification length. To add a new model (e.g. namx, namy) to be verified, just add it to the end of this file with the same format as the existing models shown below:
MODEL|REGIONS|VERIFICATION LENGTH|FORECAST RANGE|
aqm|ozone pm|48|48|
gfs|gfs gfs40 gfs40ak|84|84|
gdas|gdas|09|09|
ngm|ngm|48|48|
nam|nam nam40 namak nam40ak|84|84|
ndas|ndas|03|03|
ruc|ruc ruc40|12|12|
rtma|rtma|00|00|
dgex|dgexco dgexak|192|192|
gfse|gfseco gfseak|192|192|
hiresw|eastnmm eastarw westnmm westarw aknmm akarw hinmm hiarw prnmm prarw|48|48|
rsm|rsmh|48|48|
namhw|nameasthw namwesthw namprhw namhihw namakhw|48|48|
sref|ebmjc ebmjp ebmjn ekfcc ekfcp ekfcn rsasc rsasp rsasn rrasp rrasn emc emp1 emn1 emp2 emn2 nmmc nmmp1 nmmn1 \
nmmp2 nmmn2 srmean ebmjcak ebmjpak ebmjnak ekfccak ekfcpak ekfcnak rsascak rsaspak rsasnak rraspak \
rrasnak emcak emp1ak emn1ak emp2ak emn2ak nmmcak nmmp1ak nmmn1ak nmmp2ak nmmn2ak srmeanak|87|87|
srefx|ebmjcx ebmjpx ebmjnx ekfccx ekfcpx ekfcnx rsascx rsaspx rsasnx rraspx rrasnx emcx emp1x emn1x emp2x emn2x \
nmmcx nmmp1x nmmn1x nmmp2x nmmn2x srmeanx|87|87|
3)
verf_gridtobs_cycles
(common to all models)
This parameter file defines the cycles in which the model output data are available. The cycles are used for the “verf_gridtobs_getmodinput.sh” script to determine the model input files to be added to the verification.
4)
verf_gridtobs_domains
(common to all models)
This parameter file defines several parameters for the verification of a specific model “region”, where:
1st column: the model region, e.g. for
2nd column: the NET variable. Regions/Models are grouped into several “NET” which share the same input parm cards to the PREPFITS and GRIDTOBS steps.
3rd column: the RUN variable. This variable is used to determine the input model file name, e.g. for the GDAS model, the file name will be “gdas1.tCCz.pgrbfHH”, and the model file name should be “gdas1”.
4th column: the model name in capital case to be used in the GRIDTOBS input parm card.
5th column: the model name in capital case to be used in the PREPFITS input parm card.
6th column: the starting forecast hour of the model output.
7th column: the end forecast hour of the model output.
8th column: the model output increment.
9th column: the first segment of model input file name
10th column: the second segment of the model input file name
11th column: the third segment of the model input file name
12th column: the extension (.tm00) of the file name
13th column: whether COPYGB is needed to convert the input file into different grib file.
14th column: the grid number for the input files to be converted to using COPYGB
15th column: the verification grid
Here is a portion of the file(for the
#REGION||NET|RUN|MODEL|PLL3|SFHR|EFHR|INC|FILNAM1|FILNAM2|FILNAM3|TMKK|DOCYGB|CGRID|GRDNAM|
### GFS ###
gfs|global|gfs|GFS|GFS|00|84|06|pgrbf| | | |NO| |3|
gfs40|global40|gfs|GFS|GFS|00|84|06|master.grbf| | | |YES|212|212|
gfsak|globalak|gfs|GFS|GFS|00|84|06|master.grbf| | | |YES|242|242|
gfs40ak|globalak|gfs|GFS|GFS|00|84|06|master.grbf| | | |YES|216|216|
###
nam|meso|nam|NAM|NAM|00|84|06|awphys|awip12|awip32|.tm00|NO| |218|
nam40|meso|nam|NAM|NAM40|00|84|06|awip3d|awip3d| |.tm00|NO| |212|
namak|mesoak|nam|NAM|NAM|00|84|06|awak3d|awak3d| |.tm00|NO| |242|
nam40ak|mesoak|nam|NAM|NAM40|00|84|06|awipak|awipak| |.tm00|NO| |216|
5)
verf_gridtobs_grid104
(common to all models, in GRIDTOBS step)
There are two files “regions” and
“grid104” that are used to divide grid 104 into different sub-regions so that
statistics may also be performed within each of these sub-regions. This file
“grid104” assigns each grid point in the NCEP grid 104 with a unique sub-region
number that is consistent with the definition in the file regions. Currently,
29 sub-regions are defined within NCEP grid 104. This is a fix file that you DO NOT need to modify when adding a new
model.
6)
verf_gridtobs_regions
(common to all models, in GRIDTOBS step)
This file defines the two-digit
number and three-letter abbreviation associated with each sub-region. This is also a fix file you DO NOT need to
change when adding a new model.
7)
verf_gridtobs_ keeplist (common to all models, in EDITBUFR step)
This is an input control file to be used in the EDITBUFR step, which allows specification of time window, areal extent and observation type to be included in the output file in order to filter out unnecessary observations. If you want to add a new model to the verification, please first check and see if your “NET” (in the 2nd column of the “verf_gridtobs_domains” file) has been defined in this file. If not, you can add it to the end of this file using the same format as in the existing NET.
8)
verf_gridtobs_levcat
(common to all models, in the PREPFITS step)
This file specifies the number of
levels to be read from the GRIB files and the logical flag of the data
categories. There are 10 data categories. If you need to add a new model to the
verification, please add the “regions” to the end of the file using the same
format as in the existing file.
9)
verf_gridtobs_ prepfits.tab (common to all models, in the PREPFITS step)
This is a bufr table defining BUFR mnemonics. For the OZONE and PM runs, they use their own “prepfits.tab” files, which are “prepfits.tab_ozone” and “prepfits.tab_pm” respectively.
10) verf_gridtobs.${NETWORK}
(each NETWORK has one specific GRIDTOBS control file)
The control file to be used in the
GRIDTOBS step, it contains essentially the bounding parameters over which the
partial sums are to be accumulated. This
bounding parameter info also facilitates generation of the 9 headers of the
VSDB record. Please refer to Perry’s document http://www.emc.ncep.noaa.gov/mmb/papers/shafran/gridtob.pdf
for a detailed description of each element in this control file. The control
files for all the verification models are grouped into several NETWORK according to the verification
V01 10
1 MODNAM/GRDNAM
1 19
1 ADPUPA
4 G236
G245
G246
G247
2 SL1L2
FHO
11 Z
T
Q
RH
PBL 6 500 1000 1500 2000 2500 3000
CIN
LI 5 -10 -8 -5 -2 0
TROP
PWO
VWND
11 P1000
P850
P700
P500
P400
P300
P250
P200
P150
P100
P50
V01 10
1 MODNAM/GRDNAM
1 19
3 ANYAIR
PROFLR
VADWND
4 G236
G245
G246
G247
1 SL1L2
2 T
VWND
9 P1000-850
P850-700
P700-550
P550-400
P400-300
P300-250
P250-200
P200-150
P150-50
V01 10
1 MODNAM/GRDNAM
1 19
3 ANYSFC
ONLYSF
GPSIPW
14 G104/NWC
G104/SWC
G104/NMT
G104/SMT
G104/GRB
G104/SWD
G104/NPL
G104/SPL
G104/MDW
G104/LMV
G104/GMC
G104/NEC
G104/SEC
G104/APL
2 SL1L2
FHO
14 SLP
T
TMAX
TMIN
RH 12 10 20 30 40 50 60 70 80 85 90 95 100
DPT 7 0 5 10 15 20 25 30
TCLD 12 10 20 30 40 50 60 70 80 85 90 95 100
GUST 4 10 20 30 40
HEAT 4 80 90 100 110
CHILL 5 -20 -10 0 10 20
PWO
CLDBT
VWND
1 SFC
V01 10
1 MODNAM/GRDNAM
1 19
3 ANYSFC
ONLYSF
GPSIPW
13 G104/HWI
G104/NPO
G104/SPO
G104/NAO
G104/SAO
G104/GLF
G104/PRI
G104/CAR
G104/CAM
G104/NSA
G104/WCA
G104/ECA
G104/MEX
2 SL1L2
FHO
14 SLP
T
TMAX
TMIN
RH 12 10 20 30 40 50 60 70 80 85 90 95 100
DPT 7 0 5 10 15 20 25 30
TCLD 12 10 20 30 40 50 60 70 80 85 90 95 100
GUST 4 10 20 30 40
HEAT 4 80 90 100 110
CHILL 5 -20 -10 0 10 20
PWO
CLDBT
VWND
1 SFC
V01 10
1 MODNAM/GRDNAM
1 19
3 ANYSFC
ONLYSF
GPSIPW
1 G236
2 SL1L2
FHO
14 SLP
T
TMAX
TMIN
RH 12 10 20 30 40 50 60 70 80 85 90 95 100
DPT 7 0 5 10 15 20 25 30
TCLD 12 10 20 30 40 50 60 70 80 85 90 95 100
GUST 4 10 20 30 40
HEAT 4 80 90 100 110
CHILL 5 -20 -10 0 10 20
PWO
CLDBT
VWND
1 SFC
You can check out the GRIDTOBS verification package from EMC’s Subversion Server on
https://svn.ncep.noaa.gov/emc/verif/grid2obs/gridtobs_new
2.
What you
need to change to make your own verification run
1)
The SMS scripts
Currently what we have in running are only the operational models,
including:
AQM (including OZONE and PM)
AQMMAX (including OZONE and PM)
DGEX
GDAS
GFS
HIRESW
NAMHW
NDAS
NGM
RSM
RTMA
RUC
SREF
SREFX
SREFENS
SREFENSX
Except for the AQM and DGEX (runs 4 times per day), all the other models
run every 3-hourly (00Z, 03Z, 06Z, ….21Z). Each model
for each cycle will need to have a separate SMS script. So if you want to run
your verification 3-hourly, you will need to create 8 SMS scripts for it.
2)
The PARM files
The Parm files you will need to edit to add
your new model verification are:
verf_gridtobs_config (need to define or modify the INDIR variable)
verf_gridtobs.modelinfo
verf_gridtobs.cycles
verf_gridtobs.keeplist
(grouped by NETWORK)
verf_gridtobs.levcat
verf_gridtobs.domains
And you will also need to create your own the GRIDTOBS control file (to
generate the VSDB records) if it does not exist. Currently the control files
(grouped by NETWORK) we have are:
DGEX: verf_gridtobs.dgex
verf_gridtobs.dgexak
GFS/GDAS: verf_gridtobs.global verf_gridtobs.globalak
verf_gridtobs.global40
NAM/NDAS: verf_gridtobs.meso verf_gridtobs.mesoak
NGM: verf_gridtobs.ngm
RUC: verf_gridtobs.ruc
RTMA: verf_gridtobs.rtma
HIRESW/NAMHW: verf_gridtobs.hrwwest verf_gridtobs.hrweast verf_gridtobs.hrwhi verf_gridtobs.hrwpr verf_gridtobs.hrwak
SREF/SREFX: verf_gridtobs.sref verf_gridtobs.srefak
OZONE/PM: verf_gridtobs.ozone verf_gridtobs.pm
SREFENS/SREFENSX: verf_gridtobs.srefens
verf_gridtobs.srefensx
Specifically, if the model you want to add to the verification is not an
operational model (running at NCO), you will need to define or modify the INDIR
variable to point to the model input file directory in the verf_gridtobs_config
file. If it is not defined previously, it will take the default value set up in
the J-Job script as:
Export INDIR=${INDIR:-/com/${NET}/prod/${RUN}}
3)
The USH script
Usually you might not need to modify any of the scripts to add a new
model into the verification system. But if the naming structure of the input
files is specific(like the NAM, NDAS, RUC, RTMA, SREF, SREFX), you will need to
check the verf_gridtobs_getmodinput.sh script
to see if you need to add a block there or not (in the “case” statement).
3.
How to run
the verification
To run the GRIDTOBS verification,
simply submit the jobs to the loadlevelor using “llsubmit”, e.g. to submit the
llsubmit verf_gridtobs_nam_00.sms (for the 00Z cycle)