DIAGNOSTIC SYSTEM DOCUMENTATION
GrADS-based Diagnostic System

Table of Contents

Overview

The purpose of this diagnostic system is to allow access to global analyses and forecasts so that time-mean (systematic) errors and other forecast characteristics can be explored. The analysis and forecast data come from standard pressure grib (pgb) files available at /global/noscrub/emc.glopara/global/gfs. Both GFS and ECMWF analyses and forecasts can be accessed.

The following fields may be accessed and displayed in the diagostic scripts (asterisk indicates for GFS only):
Temperature ([Kelvin], 26 levels)
Geopotential height (26 levels)
Relative humidity (26 levels)
Zonal and meridional wind (26 levels) and wind speed
Temperature ([Celsius], derived from Kelvin temperature)
Temperature ([Fahrenheit], derived from Kelvin temperature)
Absolute vorticity*
– 2 m temperature
– 2 m dewpoint
– 2 m specific humidity*
– 2 m relative humidity
– 10 m zonal and meridional wind and wind speed
Mean sea-level pressure
Surface pressure
Cloud water (21 levels)*
Total column cloud water*
Total column precipitable water*
Total column relative humidityTemperature (Celsius, derived from T [Kelvin])
Temperature (Fahrenheit, derived from T [Kelvin])

In principle, any variable in the pgb files can be displayed and analyzed, but there are some limitations as noted below.

This diagnostic system uses GrADS-based scripting to produce images of biases and forecast errors for individual verifying times. It employs standard "control" (c\ .ctl) files for access to both analysis and forecast files for the GFS and ECMWF. These files become available in units of months (in default mode), although one can easily configure averages for any contiguous time period with a month. For example, one can access analyses for the month of September 2015 and all GFS and ECMWF forecasts that are valid for that month. There is an overhead corresponding to the time necessary to generate all of the gribmap files for that month. Trivially, one can extend the access to multiple months but, of course, the overhead for making gribmap files is correspondingly longer.

The system uses templates and UNIX scripting to generate appropriate control files and their gribmap files. A setup grads script prompts responses to gather the choice of diagnosed variable, level, forecast hour and other parameters defining the diagnosis. A batch file is available to generate these choices in a batch execution and there is a default configuration for a rapid run through of the setup for simple debugging purposes. Currently the system runs diagnostics on a single variable at a single level and forecast time but a batch run of any number of variables can be executed with a simple driver script and grads run in batch mode.

Scripts operate on the dimension environment (lon, lat, height, time) invoked in the setup file. Scripts return to the grads prompt when finished. Another appropriate script can then be invoked or the user can either re-run the setup script, move into batch mode or rummage through the available environment within the opened forecast files.

The system has some limitations. They are:
– At the present time, only forecasts verifying at 00 UTC and analyses can be fully analyzed. In case study mode (see Control File Structure below), forecasts initiated at 00 or 12 UTC can be accessed.
– Variables are accessed through an ascii table to simplify the rather complex names given in the standard control files. For example, the control file variable PRMSLmsl is abbreviated to pmsl. This table contains other information such as units, contouring themes and scaling for each variable.
– The system can process one variable at one level and one set of verification times for each run. Each run requires a separate setup using bias_setup.gs. However, the system can be executed in batch mode, and can be driven by a command script. An example is given (batch_sample.gs).

Control file structure

This diagnostic system keys off the forecast valid date (an exception is noted below). One control file enables access to every analysis within the month. Control files for GFS forecasts give separate access to f06, f12, ...f120 and f240 hours. For example, for December 2015, there are 31 analysis times. The first time in the f06 forecast control file corresponds to a 6-hour forecast initiated from 18 UTC 30 November. The first time in the f12 forecast control file corresponds to a 12-hour forecast initiated from 12 UTC 30 November, etc.

The system can also be executed in case study mode. In case mode, a control file enables access to all desired forecasts (either GFS, ECMWF or both) from a given initial date. Another control file enables access to the verifying analysis for each forecast time. Both GFS and ECM forecasts can be accessed and either analysis can be used for verification. GFS analyses are available every 6 hours, which corresponds to the forecast interval. ECMWF analyses are available every 12 hours so ECMWF verification is possible for the GFS at 00 and 12 UTC times only. If the ECMWF analysis is chosen for verification, then the GFS forecasts are winnowed accordingly.

System Structure

The top diagnostic system directory contains six sub-directories as follows.

The top-level directory also contains the shell script to generate the system "bias_file_setup.sh" which is copied there after it has completed execution. This shell script should be executed from outside the top-level directory. The top-level directory also contains a copy of this documentation.

The library directories are not changed except to load new grads scripts and shell scripts, either to implement new versions or to add new capabilities. Each new execution of the generating script "bias_file_setup.sh" removes the exec and working directories but some lines in this script can be commented out to preserve the libraries if desired.

Library contents are copied from e is oStephen.J.Lord directories, so that some changes may occur inadvertently unless the user maintains an independent source of scripts, templates, etc. A Subversion (svn) interface is planned.

Ownership

The system prototype is owned by "Stephen.J.Lord" and has a default top-level directory of /ptmpp1/Stephen.J.Lord/tempsjl. In bias_file_setup.sh, both Stephen.J.Lord and tempsjl can (and should) be changed to the user's directories to adopt the diagnostic system and to ensure its independence from other users or other applications by the same user. Also, in bias_file_setup.sh, files are copied from "Stephen.J.Lord" directories; this will be changed when a SubVersion-based system is implemented in the near future.

Required local files (exec_input)

Required local files are read by the diagnostic system in the course of generating graphics. They are stored in the exec_input directory.

Templates

Template files are:

Shell Scripts (shell_scripts)

UNIX shell scripts in this directory set up the diagnostic system and generate .ctl files. System prototype is owned by "Stephen.J.Lord"; see remarks above concerning ownership. The following files contain ownership information.

GrADS Scripts (grads_scripts)

Graphics Scripts

==> anal_compare.gs <==

Execute "anal.compare.gs" to compare GFS and ECMWF analyses at individual analysis times (00 UTC only). Script displays analysis difference (shaded) and each analysis (contour). Analysis files are opened in bias_setup.gs.

==> batch_sample.gs <==

Executing "batch_sample.gs" generates multiple runs of the diagnostic system using successive applications of the bias_setup.bat file. For each configuration of the .bat file, bias_setup.gs is executed and an appropriate grads graphics script is then called. The UNIX stream editor "sed" is used to modify strings in the .bat file to obtain the desired setup.

==> bias_gen.gs <==

Execute "bias_gen.gs" to define time mean error (bias) over defined domain in x, y, time. Script displays horizontal fields of Analysis, Verifying Forecast and Bias (F-A). Prerequisite is running bias_setup.gs. Analysis and Forecast files are opened in bias_setup.gs. Script will run under "fcst" mode to display a single forecast time, but will also operate under "tser" mode to display an average of forecast times, such as 8-14 days. Statistics, including maximum, minimum, average (bias) and standard deviation values are listed for the selected lat-lon domain. This may not be exactly the same domain as displayed on the graphic if projections other than lat-lon are used.

==> bias_setup.gs <==

Execute "bias_setup.gs" to define all of the system choices. There are 11 sections, each of which determines these choices. Some of these choices are: the verifying and forecast models, the verification mode, the forecast variable, plot labels, time and horizontal domain and level, and options controlling graphical output, plotting storm tracks, and color scheme. In interactive mode, the script names the default and requests confirmation or a new choice. In default mode, all choices are made within the script. In batch mode, each choice is defined by a separate section, one by one. After completing each section, the choices are reviewed (together with the variables carrying these choices), and they can be approved (as a group) or revised individually. The varparm file contains details on variable abbreviations and names, labels, levels, model availability and color schemes and is also used to check on whether the chosen variable is entered correctly. File intbatdef determines whether interactive (i), batch (b) or default (d) mode is used. Required .ctl files are opened in bias_setup.gs and they should be put locally in the exec directory. A utility script, listfiles.gs, gives data file description information from the .ctl file. Section 12 is a prototype for setting up observations. Some cosmetic changes were made to ensure the Section banners are lines up with the same line lengths. A new domain covering the northern and southern East Pacific Ocean was added. See in-script documentation for details of recent upgrades.

==> case_compare.gs <==

Executing this script is in case mode only. It generates a 4 panel page displays the verifying analysis (upper left), the forecast model error (upper right), the forecast from the verifying model (lower left), if different from the forecast model) and the difference between the two forecasts. If the verifying model and the forecast model are the same, only the 2 upper panels are displayed. Panels are displayed for each forecast time. See in-script documentation for details of recent upgrades.

==> error_fhour.gs <==

Execute "error_fhour.gs" to diagnose forecast error at specific fcst hour for requested variable over defined domain. Script displays horizontal map of Analysis, Forecast and Error (F-A). A statistics string was added (see bias_gen.gs).

==> error_growth.gs <==

Execute "error_growth.gs" to examine error growth. Errors are shown every 6-12 hours (depending on output file time resolution) and forecasts are superimposed in 24 hour groups (0,6,12,18 in day 1, etc.). bias_setup.gs opens all forecast files and this script is run from the tser option.

==> error_sect.gs <==

Execute "error_sect.gs" for profile variables. Script displays vertical section of Analysis, Forecast and Error (F-A bias). Prerequisite is running bias_setup.gs and opened Analysis and Forecast files and x-y dimension from prior plot. This script should only be executed for fcst option in bias_setup.gs.

==> error_survey.gs <==

Execute error_survey.gs to depict time-mean error distributions geographically. The time mean squared error is plotted to pull out the largest errors, whether positive or negative. A zonal mean error is also shown on the same graphic. The domain is divided into lat/lon boxes and the spatially averaged (time mean) error magnitude is calculated for each box. These values are sorted and errors exceeding given percentiles (e.g., the largest 25% and 10%) are colorized and plotted as a solid color for each box. The geography is overlaid as are the box average values.

==> plot_storm_track.gs <==

Execute "plot_storm_track.gs" to plot storm tracks, including extratropical storms (MLxxxx) and tropical storms from various basins. Tracks are extracted from the database of Guang-Ping Lou. Storm file is hardwired to storm.track.anal.window for verification/analysis positions. Scripts collect_anal_storm_tracks.gs and collect_fcst_storm_tracks.gs (incomplete) are used to add tracks to the storm.track.anal.window file. Forecast positions are read from file specific to each storm and verification date. Track plotting options are read from external file (plttrkinfo).

Utility Scripts

==> bias_file_setup.gs <==

"bias_file_setup.gs" is a major upgrade from DS 1.3.1 and is not upward compatible with its predecessor. It now reads eight input parameters to generate access to model output fields; these parameters are determined by user input in "bias_file_setup.sh." They are:

==> case_setup.gs <==

Script is called with input parameters defining the year, month, day, hour and maximum forecast hour for the case. Unix commands (e.g., sed) fill in the templates and generate the .ctl files and execute the gribmap. Script re-written to accommodate upgraded procedure for generating .ctl files and executing gribmap.

==> catlg_list.gs <==

Script lists a catalog file contents. There are currently two catalog files:

==> cbarn.gs <==

Standard grads color bar generator script.

==> collect_anal_storm_tracks.gs <==

Execute "collect.anal.storm.tracks.gs" to extract all analysis/verification storm track positions for a specified date window determined by the current GrADS environment from the storm catalog. Output file is in local directory where Grads is being run.

==> collect_fcst_storm_tracks.gs <==

Execute "collect.fcst.storm.tracks.gs" to extract all storm track positions for a specified date and domain window determined by the current GrADS environment.

==> contour_set.gs <==

The script "contour_set.gs" does the set up for the sjl contouring scheme for each plotted variable. Contour min, max and interval are read from an external survey (consurvey) file. Other files are internal (generated by bias_setup.gs and varparm.gs). Survey file has entries for vertical levels and forecast hours. Not all forecast hours are tabulated(too many), but contour assignments at 24 h and beyond are assigned to that particular forecast day (e.g., fhour=30, 36, 42 are assigned to 24 h.)

==>create_storm_catalog.gs <==

Generates .

==> create_storm_catalog.gs <==

Script to create a storm catalog. Entries are storm name (including month), beginning analysis time and final analysis time. Catalog is accessed by the storm track collection scripts for efficient access when the diagnostic system is running. Processed storm information files from Guang-Ping Lou are used. Currently, storms from November-December 2015 have been cataloged; an expanded set of dates is anticipated.

==> decstr.gs <==

Utility script to decode a string with delimiters into separate words.

==> digs.gs <==

Trims grads digits to a requested size, both a total size and a number of digits after the decimal point.

==> fillnum.gs <==

Utility function to fill numbers for labeling purposes. E.g., a 6 hour forecast would be displaced as "f06" rather than "f6."

==> fillstr.gs <==

Utility functions to calculate length of input string (lengstr) and substitute blanks to fill out a shorter string into one of requested length.

==> g_off.gs <==

Turns off the grads graphics and date for each graphic. Currently not implemented.

==> initial.gs <==

Sets up display characteristics, colors and the font. White background is used.

==> int.gs <==

Utility function to truncate a floating point number into an integer.

==> listfiles.gs <==

Generates a list of opened files and provides summary data information from the .ctl file.

==> obs_setup.gs <==

Prototype script to generate observations information for display in the DS. Under development for this DS release.

==> rgbset2.gs <==

Defines colors by assigning levels of red, green and blue. See the grads manual.

==> sort.gs <==

Utility to sort a list of numbers. Used in error_survey.gs.

==> stats.gs <==

Utility to capture statistics by parsing the "set gxout stat" command. Uses digs.gs.

==> survey.vars.daily.gs <==

Utility script to generate a table of contour intervals and maximum/minimum values for all variables in the varparm file for selected forecast times. Includes full fields as well as forecast-analysis differences. Analysis differences are estimated from operational GFS and ECM analyses. Forecast times are sampled as: 00 (analysis), 06, 12, 24, 48, 72, 96, 120, 240 h. Table is used in sjl contour scheme.

==> survey.vars.monthly.gs <==

Same as for survey.vars.daily, except for monthly mean fields and a smaller sample of model levels (925 500 300 200 50 10 hPa).

==> varparm.gs <==

Manages the variable parameters in the varparm file. Called by bias_setup.gs. Input is basic information on variables, levels, etc generated by bias_setup.gs. Can be called with mode (0) to check whether the analysis variable abbreviation has been spelled correctly. If not, changing it at the appropriate review opportunity in bias_setup.gs is essential. When called in mode (1), it adds additional information from the varparm file to the plotvar file. Upgraded to add prototype observational information for selected variables.

Version / Date Description
1.2 10/3/2016 bias_setup.gs is re-written. It contains 11 sections, each of which defines required input variables. Input of all variables also contains a review step in which revisions to correct or change input is allowed. As a consequence, the code structure has improved clarity and modularity. As before, the setup can also be done in default mode (d) or batch mode (b) as well as interactive mode (i).
— a "case" option was added. User specifies the ymdh (yyyymmddhh) and the system generates .ctl files for the forecast and verifying analyses and runs gribmap. Either GFS or ECMWF or both can be run. If the verifying analysis and forecast model are different, both forecasts are verified against the requested analysis and the forecast difference is also displayed.
— application scripts are modified to improve flexibility of adding new variables such as temperature (celsius, tc, equal to t (kelvin) minus 273.1) and tf (fahrenheit).
survey.vars.monthly.gs script for monthly sjl contours is expanded to include 200 hPa as a sampled level. This is to improve the contouring of wind speed in jet regions.
— Canned domains. User has choice of 9 pre-defined domains. Each domain has a data domain, defined map projection and defined mpvals. The domains are:
* — > Glo: Global
* — > NH: Northern Hemisphere
* — > SH: Southern Hemisphere
* — > NA: North America
* — > CON: CONUS
* — > Ala: Alaska
* — > WEur: Western Europe
* — > ARC: Arctic
* — > AntARC: Antarctic
* — > NAtltr: North Atlantic tropics
* — > EPactr: Northeast Pacific tropics
* — > WPactr: West Pacific tropics
* — > Cus: Custom (user sets domain)
— In response to user request, the color background for difference maps around the zero contour is changed from gray to white and the geography is changed to black.
— Temporal means over the chosen time period were added.
— Zonal means, e.g., for height-latitude/longitude cross sections, were added.
— Access to T, Z, U, V and RH at the highest levels (1, 2, 3, 5, 7 hPa) was added for the GFS.
1.3 11/2/2016 bias_setup interactive questions are revised to remove approval of individual input. Approval and revisions are exclusively part of the review step only.
— Hours 144, 168, 192 and 216 are added to forecasts and verification.
— Utility script to list open files was added.
1.3.1 11/17/2016 — Bugs in bias_setup.gs questions and dialog were fixed.
— A new graphics scripts, error_survey.gs, was added. Documentation added.
2.0.0 1/18/2017 — The system has been upgraded to incorporate EMC parallel runs and runs designated by any user. A catalog file (model_runs_catlg) has been introduced to describe any model run and bias_file_setup.sh and bias_file_setup.gs have been upgraded to read the catalog, choose any number of operational (GFS and ECM) and parallel runs, generate the appropriate .ctl files and execute gribmap on all chosed model runs. The templates and .ctl file generator files have been generalized to accommodate the parallel runs, which have a slightly different file structure than the operational runs, and to pave the way for accessing observations and other kinds of data. In brief, the .ctl file generator has been broken into pieces to accomodate different .ctl variables (between the GFS and parallels on one hand) and the ECM on the other. For increased efficiency in setting up the system, bias_file_setup.sh enables choosing a maximum forecast length (e.g., 120 h) and a forecast interval (e.g., 6 h) so that the .ctl files and gribmap generator execute only the chosen range. Default forecast intervals are now created. They are: 3 h for forecast length (fclen) <= 72 h, 6 h for fclen <= 120 h, 12 h for fclen <= 240 h and 24 h for <= 384 h.
Bias_setup.gs has been restructured into 11 sections and completely overhauled for both clarity and for adding new capabilities.
— New utility scripts have been added.
— Some documentation has been added for the most important scripts.
— Some infrastructure has been added as a prototype for accessing observations. Access to conventional observations, upper air (mandatory Raobs), land and ocean surface, and aircraft (AMDAR and "aircft") is under development.

For diagnostic system results see page Figures.