Skip Navigation Links www.nws.noaa.gov
NOAA logo - Click to go to the NOAA homepage National Weather Service NWS logo - Click to go to the NWS homepage
EMC Logo
Navigation Bar Left Cap Home News Organization
Navigation Bar End Cap

         MISSION / VISION    |    About EMC

EMC > NEMS > RUNNING NEMS
NOAA Environmental Modeling System

Running NEMS



This page is meant to guide you in setting up and running experiments for the different models within NEMS. If you are interested in starting out with regression tests, please refer to 'Getting Started' . Bear in mind that, currently, there is no simplified way of running the NMM-B or GFS individually. It may require changes to the regression test script (RT.sh), the ability to recognize other files involved, creation of your own run scripts, and the creation of your own compatible input files.

Libraries & Dependencies


Currently, NEMS requires the following external libraries to be installed on the system in order to run properly.
  • ESMF versions 3.1.0rp2 and 5.2.0rp1 are supported. Available from the ESMF web site.
  • netcdf - available from the UNIDATA website.
  • The following libraries are from NCEP and can be downloaded from the NEMS trunk (located in EMC's Subversion repository) in /trunk/util/syslib.tar:
    • bacio
    • w3
    • sp
    • nemsio
  • The following will be used only when write_dopost is set to .true. in the configure file:
    • g2 - /nwprod/lib
    • g2tmpl - /nwprod/util/sorc/g2tmpl.fd
    • sigio, sfcio - /nwprod/lib
    • unipost - /climate/save/wx20wa/esmf/nems/unipost_r14561
    • png, jasper - /usrx/local/64bit/lib; jasper is the library to make jpeg2000 packing
The last two libraries listed above may already be installed on your system (Linux), but may not be by default. PNG, in turn, requires zlib, but that is almost always installed, at least on Linux. They are required for compilation of the g2 (GRIB2) library.



Check Out and Build NEMS:


If you are interested in just running regression tests, please refer to 'Regression Tests'. Otherwise, continue on below.

Process to checkout and build NEMS:
ActionActive DirectoryCommand
Check out NEMS/gpfs/td1/emc/**/save/Firstname.Lastname (Tide)
/gpfs/gd1/emc/**/save/Firstname.Lastname (Gyre)
/scratch*/portfolios/NCEPDEV/**/save/Firstname.Lastname (Zeus)
* - 1 or 2
** - global, meso, etc.
svn checkout https://svnemc.ncep.noaa.gov/projects/nems/trunk
Set the version of the ESMF library that will be used (if running RT.sh, this will be done automatically). Make note of machine specific options:.../trunk/srcESMF 3.1.0rp2:
./esmf_version 3_wcoss
./esmf_version 3_zeus
./esmf_version 3_yellowstone

ESMF 5.2r:
./esmf_version 5.2_wcoss (for 5.2.0rp2)
./esmf_version 5.2.0rAPI_zeus

ESMF 6.3.0r:
./esmf_version 6.3r_wcoss

ESMF with reference NUOPC layer (ESMF 6.2.0):
./esmf_version nuopc_zeus
Clean any previous compilations..../trunk/srcgmake clean
Compile and build the core(s) needed (/job/regression_tests/RT.sh will build the default settings - NMM and GFS with GOCART).../trunk/srcSee below...


Commands used to compile various NEMS cores (outside of /job/regression_tests/RT.sh):
  • gmake nmm - NMM only
  • gmake gsm - GSM only
  • gmake gen - GEN only
  • gmake fim - FIM only
  • gmake nmm_gsm - NMM and GSM
  • gmake gsm GOCART_MODE=full - GSM (with GOCART)
  • gmake nmm_gsm GOCART_MODE=full - NMM and GSM (with GOCART)
  • gmake nmm_gsm_gen_post GOCART_MODE=full - GEN, NMM and GSM (with GOCART and post)
  • gmake nmm_post - NMM (with post)
  • gmake gsm_post - GSM (with post)
In order to build NEMS with ocean components, all of the above targets support the optional OCN variable. Supported values are: "dummy", "hycom", and "mom5". Specifying multiple values, e.g. "gmake nmm_gsm OCN=mom5,dummy", is supported.

The executable, NEMS.x, will appear in /exe.



How To Run NMM-B


Using regression test script on WCOSS and Zeus:


It is possible to run the NEMS NMM-B by creating your own run script based on RT.sh.
  1. In /job/regression_tests, copy RT.sh_IN to a new filename (e.g. 'cp RT.sh_IN test.sh').
    • In your new script, comment only 'SKIP_NMM=YES' and uncomment 'CLEAN_SORC=NO' and 'CLEAN_RUNDIR=NO' to keep your source code and test results.
  2. In RT.sh, change all 'RT.sh_IN' to your new filename created in step 1.
  3. Submit your test script (e.g. 'sh test.sh').
  4. Output will be found in your /ptmp directory.

Using the NMM-B Launcher (coming soon):


The state of the Launcher on WCOSS is in a state of flux. For reference, please refer to the evolving draft of Launcher documentation.


How To Run NMM-B Nests


The NMM-B has telescoping static and moving nest capability. All domains, whether the uppermost parent or any nest, are functionally equivalent and thus each needs its own configure file. Both 1-way and 2-way interaction between parent and child domains are available. In addition to running nests using the regression test input and configuration (similar to the method mentioned in 'How to Run NMM-B'), one can also follow as specified below.

The following instructions are also available in the README.NMM file.

1-way nesting: [+/-]


2-way nesting: [+/-]


NMM-B output files are written in a gridded format readable by GrADS (.ctl). To understand NMM-B output, please refer to the 'Output' tab.



How to Run GFS


On WCOSS:


Currently, the best way to run the NEMS GFS on WCOSS is to create your own run script based on RT.sh.
  1. In /job, move gfs_fcst_run.IN_IBM to gfs_fcst_run.IN.
  2. In /job/regression_tests, copy RT.sh_IN to a new filename (e.g. 'cp RT.sh_IN test.sh').
    • In your new script, comment only 'SKIP_GFS=YES' and uncomment 'CLEAN_SORC=NO' and 'CLEAN_RUNDIR=NO' to keep your source code and test results.
  3. In RT.sh, change all 'RT.sh_IN' to your new filename created in step 1.
  4. Submit your test script (e.g. 'sh test.sh').
  5. Output will be found in your /ptmp directory.

On Zeus:


In addition to isolating specific GFS regression tests in RT.sh_IN, as mentioned above, one can also use the following method to run GFS on Zeus.

As part of the 2013 NEMS/GFS Modeling Summer School, a suite of scripts and sample initial conditions was created to allow students to run the NEMS GFS. The code is located in /scratch2/portfolios/NCEPDEV/global/save/glopara/Summer_School_2013. Below is information on how to use the code:
  1. Move to your working directory, then use the following command to copy necessary files: 'sh /scratch2/portfolios/NCEPDEV/global/save/glopara/Summer_School_2013'.
  2. Compile NEMS/GFS code:
    • cd Summer_School_2013/Tutorials
    • sh makefile_nems.sh
      • Compile log can be viewed in ../sorc/global_fcst.fd/Compile.log
  3. Make sure your global_fcst executable is located in Summer_School_2013/exec.
  4. Submit an experiment by using 'sh submit_nems.sh' in your /Tutorials directory. By default, an experiment will run at current operational T574 resolution.
    • Make changes to the following corresponding to your account:
      • 'export acc='
      • 'export NEMSDIR='
      • 'export RUNDIR='
  5. Output, by default, can be found in the directory defined by OUTDIR in submit_nems.sh.
Sample input files are provided in /scratch2/portfolios/NCEPDEV/global/noscrub/glopara/Summer_School_2013_ICs.


How to Run FIM (coming soon)






Porting Information


Following is a procedure for defining functional equivalence on different environments:
  1. Select key fields from the import state Fi (may include a selection of initial cases).
  2. Run control run C of application on control environment.
  3. Perturb each Fi in the last significant digit.
    1. Nominally after the 6th decimal digit for 32-bit reals.
    2. For each value Fi(x), construct a distribution.
    3. Gaussian distribution with width being | Fi(x)/1.e6|.
    4. Randomly select perturbation from that distribution.
  4. Run perturbed run P from perturbed import state on control environment.
  5. Select key fields from the export states Cj (control run) and Pj (perturbed run) - may include a selection of output times.
  6. Determine area weight w(x)
  7. Compute the butterfly vector .
  8. Butterfly vector will have units of original export fields.
  9. Now run control run but on different environment D.
  10. Compute difference vector like butterfly vector but using D instead of P.
  11. Compute dimensionless difference butterfly number vector .
  12. For a bit-identical requirement, vj must be zero.
  13. For a functionally equivalent requirement, vj must be largely less than 10.
The presentations listed below show preliminary results from porting parts of NEMS to other computers:
Contributors: Eugene Mirvis, Tom Black, Ed Colon, Jun Wang