Documentation of The Eta Model Product Generator Geoff Manikin NCEP/EMC/MMB geoffrey.manikin@noaa.gov 1. Introduction In the original version of National Centers for Environmental Prediction's Eta model, interpolation of the output to the various grids produced operationally was performed in the post processor. With, however, the increasing number of output grids, it became less computationally efficient to produce each of these grids in the post processor. In 1997, the product generator was constructed to make the interpolations a series of spinoff processes. The post processor now only produces Eta output on the native grid. All desired fields on the various output grids must be available on the native grid, as no diagnostic computations are performed in the product generator. 2. Input Datacards The critical files for user choices with regards to the interpolations are the master files, labeled eta_masterX.ctl where X=1,2,3,4,5. The following is an example of an entry in one of these files: 41 100 1 244 0 2 1 ABS VORT ON P SFCS 84/221/X/2.7/eta.AWIP32/60/60 84/217/X/2.7/eta.AWP217/0/0 84/211/X/2.7/eta.AWP211/2/4 84/207/X/2.7/eta.AWP207/2/4 There are 7 parameters set on the initial line. (Note: the final 2 were added in the spring of 2003.) The first indicates the parameter as defined in the parameter table in Office Note 388, the version of which is denoted by the sixth entry in the above line. In the above example, the requested parameter is number 41 from Table 2, which is absolute vorticity. The second entry is the level type indicator from Table 3 of Office Note 388. The 100 value designates an isobaric level. The 3rd and 4th entries are used for the actual level value(s). For a single level, the value is determined by 256*entry3 + entry4. In this example, 1*256 +244 shows that absolute vorticity is requested at 500 mb. If the desired field is denoted by 2 levels such as temperature in the 120 to 150 mb boundary layer, the 2 levels occupy spots 3 and 4 as follows 11 116 150 120 0 2 1 TEMP IN BNDRY LYR The fifth entry is the time range indicator from Table 5. In this example, the 0 indicates an instantaneous value, while a 4 would indicate an accumulation, while a 3 denotes an average. The seventh entry tells the model what type of interpolation to perform. A '2' tells the product generator to perform a nearest neighbor interpolation, while any other value tells it to perform a standard bilinear interpolation. It is recommended that a '1' be used when not requesting a nearest neighbor interpolation, but it is not mandatory. The code will give a warning if a value other than 1 or 2 is used but will assume that the standard interpolation is desired. The tables referenced in this section can all be found at http://www.nco.ncep.noaa.gov/pmb/docs/on388 The lines below the one with the field name provide information on the selected output grids. The '84' is the Generating Table or Model in Office Note 388, Table A entry (http://www.nco.ncep.noaa.gov/pmb/docs/on388/tablea.html) for the Eta model. The next entry is for the number of the output grid as defined in ON 388, Table B (http://www.nco.ncep.noaa.gov/pmb/docs/on388/tableb.html). The next entry is for the type of WMO header information to add. The typical entry is 'X' for 'no header', as WMO headers are added to Eta fields sent to AWIPS in a separate code. The header could, however, be added in the product generator with an 'A' designating 'AWIPS' and an 'H' designating 'intertional' message type. The next entry relates to the precision packing. If the value is positive, that number of significant digits is preserved. If the value is negative, binary precision is used. A value of -1 tells the packer to store data to the nearest 1/2. A value of -2 tells the packer to store data to the nearest 1/4; any value of -n changes the packing to the nearest 1/(2**n). More documentation is available in the FNDBIT subroutine in the product generator code. The character string indicates the name of the output file for the particular grid. The output file will have the name NAMEHH.tm00 where NAME is the given string and HH is the forecast hour. In the above example, the grid 221 output file would be labelled eta.AWIP3224.tm00 for a 24-hr forecast. The final 2 entries are the number of pre- (1st number) and post-(2nd number) interpolation smoothing passes that will be performed using a 25-point Bleck filter. In the above examples, 2 pre- and 4 post-interpolation smoothing passes will be made for 500 mb vorticity on the 211 and 207 grids, while no smoothing will be performed for this field on the 217 grid. These two numbers are also used for requesting tiled output such as that at: http://www.emc.ncep.noaa.gov/mmb/research/tiles.221.html and an example is shown in the above entry for grid 221. The first number refers to number of tiles in the x-direction as well as number of pre-interpolation passes, and the second number designates number of post-interpolation passes in addition to number of post-interpolation passes. If the numbers are greater than 10, the number of tiles is obtained by dividing the entry by 10, while the number of passes is then set equal to the remainder. In the above example, the entries are 60 and 60. This indicates that the output grid should be divided into a 6x6 set of smaller output grids with no smoothing. If the numbers were 62 and 94, the generator would tile the output into a 6x9 layput with 2 pre- and 4 post-interpolation smoothing passes. For the 6x6 tiling requested above for grid 221, files named eta.AWIP32HH.T would be generated where HH is the forecast hour and T is the tile number (1,2,3....36) where tile 1 is the lower left corner of the grid. The file with data on the full grid would also be generated and would have the name eta.AWIP32HH.tm00. 3. Scripts The product generator is currently run with 5 scripts. The various output grids have been divided among the different scripts. To run the product generator, a native grib file is required such as eta.tHHz.egdawpFF.tm00 in /com/eta/prod/eta.YYYYMMDD where HH is the model cycle, FF is the forecast hour, and YYYYMMDD is the date stamp with 4-digit year. The name of the grib file should be placed into a file called 'inputFF.prd' for which FF is the forecast hour. A sample execution of the product generator is: export XLFUNIT_10="master3.ctl" export XLFUNIT_21="$FIXeta/eta_wgt_12km_207" export XLFUNIT_22="$FIXeta/eta_wgt_12km_211" export XLFUNIT_23="$FIXeta/eta_wgt_12km_217" export XLFUNIT_24="$FIXeta/eta_wgt_12km_221" export XLFUNIT_25="$FIXeta/eta_wgt_12km_237" export XLFUNIT_41="$PARMprod/eta_kwbx.tbl" export XLFUNIT_42="$PARMprod/eta_time.tbl" export XLFUNIT_43="$PARMprod/eta_parm.tbl" export XLFUNIT_44="$PARMprod/eta_grid.tbl" export XLFUNIT_45="$PARMprod/eta_levl.tbl" /nwprod/exec/eta_prdgen < input${fhr}.prd > prdgen.out${fhr} Unit 10 is the master file as described in section 2. Units 21 and higher are the weights files for grid points on the selected output grids. All are available in /nwprod/fix, and the grid is identified by the 3 digits at the end of the name. Different weights are used for different resolutions of the native grid - the 12 km native grid is specified above. More or fewer than the 5 output grids listed above can be requested. Three rules, however, must be followed: First, unit numbers for the weights files must begin at 21 and increase by 1 as needed. Second, the weights files should be ordered sequentially by output grid number. (In the above example, the order is 207, 211,217,221,237). Finally, at least 1 field in the master file must be requested on each grid for which a weights file is assigned a unit number number. The .tbl files are available in /nwprod/parm 4. A More Detailed Look at the Code This section is provided as an in-depth documentation of the bowels of the product generator code and not critical to running it. It serves a guide for anyone who wishes to modify the code or understand the flow through it. The source code can be found in the /nwprod/sorc/eta_prdgen directory for those with access to the NCEP system. The subroutine PRDGEN is the main program of the product generator. The parameters IMAXIN and JJMAXIN specify the maximum number of points in the x and y directions for the input native grid. While these values can be set to extremely large values to ensure that they are sufficiently large, it is advised to set them to exactly the dimensions of the native grid so avoid having large blocks of usused array space make the executable require too much memory. Similarly, IMAXIN and IMAXOT are the x and y dimensions of the output grids. MAXTILE is maximum number of tiles requested for any output grids. It is currently set to 54, as grid 218 is tiled 9 x 6. MAXF is the maximum number of fields that can be interpolated to various grids, and MAXG is the maximum number of output grids for a single execution of the product generator; again, these can be set higher than actual maximum values but should not be set too much higher to avoid using excess memory. Finally, IBUFSIZE needs to be larger than the typical number of bytes of a standard native grib file. It must typically be increased when the resolution of the native grid is increased. The first called subroutine is READ_SORT_CTL which reads the master control file. A subroutine called CTL_RDR is the actual reading of the file, and it returns the needed PDS elements (9-12, 21, and 4) the nearest neighbor/bi-linear interpolation flag, model number, grid number, WMO header type. precision factor, output grid name, number of output grids, numbers of smoothing passes, and numbers of tiles in the x and y directions for each unique field requested. It also returns the total number of unique output grids that will have to be prepared during the execution of the prdgen code. For example, if 38 fields are requested on grid 212, 4 fields are requested on grid 104, and 1 field is requested on grid 215, this variable (IG1) would have the value of 3. It also returns a parameter NFLDS which is the total number of unique fields to be processed. This count is increased by 1 for any field listed in the master file regardless of the number of output grids on which it is requested. Also, if a field is requested by bi-linear interpolation and again by nearest neighbor, this counts as 2 separate fields, and each requires a distinct entry in the master file. After reading the master file, the smoothing pass and tile numbers are sorted, as are the PDS elements, and the nearest neighbor flag. A grid definition section (GDS) is then created for each output grid, and each unique file name is assigned a unit number and opened. The unit for the grib data file is opened, and the output grids are sorted by number to complete the subroutine. The next subroutine is READ_SORT_GRIB which reads in the grib file. Each grib record is read, and PDS elements for parameter, level type, level indicator, time range indicator, and parameter table are extracted for the read field. For 1 to NFLDS (number of requested fields in the master file), these PDS elements are matched against those for each requested field. If there is a perfect match, all is well. If there is no match, the code notes that the grib record was read but that the field was not requested in the master file. After each record has been read, the code notes any field requested in the master file for which there is no grib record in the input file. Loop 100 is performed over all of the desired output grids. For each grid, the appropriate weights file is read in. Loop 30 is then performed over all of the requested output fields which can be generated. The field is first upacked from the native grid grib file, and an accompanying v-component is unpacked if the field is a u-wind component or the u-component of the storm motion vector. A recently added loop has been added for the situation in which the nearest neighbor option was selected for the current field. At every grid point, the 4 weights are compared, and the largest is selected as the nearest neighbor. The clockwise vector rotation sine and cosine which are needed for vector interpolation are taken for the point with the highest weight. By the end of the nearest neighbor loop, the weights are either the ones read in (if bi-linear interpolation has been selected) or the one determined in the nearest neighbor loop (if the nearest neighbor option has been selected). The next if test determines whether the desired field is a vector, a precipitation field, or a scalar. The first check is to see if the field is a u-wind component or u-vector. If it is, the u and v components are both passed into FILTER_UV with the number of pre-interpolation smoothing passes to be performed. The routine INTERP_UV is then called to perform the interpolation. If the field is not a u-component or vector, the FILTER_SC routine is called to perform the pre-interpolation smoothing. Code was added to the FILTER_SC routine to prevent the generation of negative relative humidity values and values greater than 100 per cent from smoothing. Continuing through the if block, if the requested field is a precipitation field (including snowfall), and the nearest neighbor option is not requested, the field is interpolated by the routine INTERP_PPT which performs a budget interpolation. Otherwise, INTERP_SC is called to interpolate the scalar field. It should be noted that the INTERP_SC routine is used to interpolate a precipitation field if the nearest neighbor option is selected. There is one more loop that distinguishes between a vector and scalar field. If the field is a u-wind component or vector, FILTER_UV is again called twice (a second time for the v) with the number of post-interpolation smoothing passes passed in as an argument. The maximum values for the u and v-components are then computed to later determine the number of bits. EXTEND is then called if the bitmap flag for the field is 1 or if the field is requested on grid 6 or grid 207; the bitmaps for these two grids changed when the new domain was specified for the 32-km version of the model. This routine fills in missing regions with boundary values to avoid using bitmaps. Finally, both the u- and v-components are sent to GRIB_OUT which packs them into grib and writes them out. If tiled output was requested, tiles are generated with calls to TILE_OUT. If the field is a scalar, the same progession is followed with post- interpolation smoothing in FILTER_SC, a call to EXTEND if needed, a call to GRIB_OUT to pack the field into grib, and a call to TILE_OUT if tiles are requested. It should be noted that the GRIB_OUT subroutine calls FNDBIT to deal with precision issues followed by PUT_GB to actually pack the field.