How to Implement BUFR Table Changes for Satellite Ingest Types
bufrtab.005, bufrtab.008, bufrtab.012 and bufrtab.021)
(Last Revised 07/11/2014)
Please take a moment to read the Disclaimer for this non-operational web page.
NOTE: Links open in a new window
PLEASE NOTE: When changing BUFR tables for satellite ingest data types, complete the following procedure in its entirety on each machine (i.e., tide and gyre).
BUFR table changes add an extra degree of complexity to an implementation, because the BUFR tables are used as a definitive layout of the contents of a particular BUFR type/subtype. Specifically, the same BUFR table that is used by a tide/gyre satellite ingest job to encode a particular type of observation into BUFR is also used by the subsequent tranjb process to append those same observations to the BUFR files for that type within the BUFR database /dcom/us007003. So whenever the BUFR table is changed for a particular type/subtype, a copy of the new table needs to be written into all of the satellite ingest BUFR files for that type/subtype within the database, at the point directly before which observations encoded according to the new table will begin to be appended to each file. Applications which read these satellite ingest database files will then respond to the new table and adjust accordingly to the new layout definition as they continue reading through the files.
Note that, while one might assume that it would only be technically necessary to write a copy of the new table to files of the actual type/subtype(s) which are changing, one cannot guarantee 100% success in subsequent tranjb processing unless a copy of the new table is appended to all satellite ingest files in the database which use the same table. In other words, if, e.g., a change is being made to type/subtype b021/xx201 within bufrtab.021, one should update all satellite ingest files of type b021 within the database, not just the xx201 files. The reason for this is two-fold. First, it is possible that a change to the BUFR table for a particular BUFR type could cause tranjb to fail when processing compressed BUFR messages for this BUFR type, even for subtypes other than those modified in the table (if the new table is not copied into these files). Second, even if tranjb processing is successful in this case, it is possible that downstream dumpjb processing could fail, again even for subtypes other than those modified in the table (if the new table is not copied into these files).
Here is the procedure to follow for updating satellite ingest files in the database when a BUFR table change is being implemented:
In each case, the last command may take a minute or so to complete depending on the number of tankfiles being updated within the database, so be patient!!
In each case, the last command may take several minutes to complete depending on the number of tankfiles being updated within the database as well as their size (some of the b021 files are very large), so be patient!