README DOCUMENT Centre de Meteorologie Spatiale (CMS) First ISCCP Regional Experiment (FIRE) Atlantic Stratocumulus Transition Experiment (ASTEX) June 1992 This file contains information about the FIRE ASTEX CMS sample read software and data files. Read software has been furnished, since data files are in binary format and cannot be readily interpreted. Software setup and operating instructions are described in the following sections. This readme file includes the following sections: Section 1.0 - introduction, brief mission description. Section 2.0 - describes the read program (source files). Section 3.0 - discusses how to create executables from the C program source. Section 4.0 - demonstrates how to invoke the run-time executable. Section 5.0 - provides general information on the data sets. Section 6.0 - implementation notes. Section 7.0 - examples of output files. Section 8.0 - errata, assumptions, etc. If you have questions while using the FIRE ASTEX CMS sample read software, please contact the Langley DAAC Science, User and Data Support office. They may be reached at: Langley DAAC Science, User and Data Support Office NASA Langley Research Center Mail Stop 157D Hampton, Virginia 23681-0001 Voice: (757) 864-8656 FAX: (757) 864-8807 e-mail: larc@eos.nasa.gov http://eosdis.larc.nasa.gov 1.0 INTRODUCTION The Atlantic Stratocumulus Transition Experiment (ASTEX) was an inter- disciplinary, international field experiment performed near the Azores Islands during June 1992 as part of the First ISCCP Regional Experiment (FIRE) project. The 6 major components of ASTEX are aircraft (UW C131), MAGE (Marine Aerosol and Gas Exchange), model (ECMWF analysis), satellite (images and data derived from satellite observations), SOFIA (Surface of the Ocean, Fluxes, and Interactions with the Atmosphere), and surface (ships, buoys, and radiosonde data). The Centre de Meteorologie Spatiale (CMS) has produced a satellite-based series of images as part of ASTEX. This product is based on observations by the geostationary METEOSAT and the Advanced Very High-Resolution Radiometer (AVHRR) aboard the NOAA 11 and 12 polar orbiters. Meteosat is a series of European geostationary satellites of which two (METEOSAT 3 and METEOSAT 4) observed the ASTEX region. Observations from these satellites were submitted to the archive in binary format. Satellite data from the Centre de Meteorologie Spatiale (CMS) contains images of the ASTEX region, employing both METEOSAT and NOAA satellites. There are four main results: sea surface temperature, surface solar irradiance, cloud classification, and downward longwave flux. 2.0 The Read Program A general purpose read utility, fire_ax_cms_read, was created to provide you with a simple easy to use program capable of extracting useful information from binary CMS data files. Based on the file read software authored by Mssr. C.M.S Lannion, it is capable of extracting raw raster channel image data and providing header/summary reports. While the software recognizes both I1 and I2 pixel data formats, the read software must be configured for a specific mission at compilation. This is done by setting switches in the Makefile using a text editor before creating the program executable. Besides the README file, read software consists of five source files. A UNIX makefile (Makefile) which compiles the C program source and creates the executable file. The main program is self contained and includes its own set of utility library routines. The executable has been successfully created and tested on SunOS 4, Solaris, HP-UX and SGI (IRIX) platforms. Makefile.cms - UNIX make utility which creates executables. fire_ax_cms_read.c - the main program. When activated, the read software can provide several plain ASCII text summary report files from a given data file. The first option generates a detailed list of header parameters. As data files are processed, information is appended to the summary file. It should be noted that headers for a significant number of data sets contain little more than basic raster format parameters. A geolocational summary can be generated which lists the latitude and longitude for each pixel in the raw raster data image. The pixel data are listed sequentially and are grouped by latitude. Data are expressed in decimal integers notation with corresponding longitude values incremented according to the resolution of the survey instrument. One or more channel raster images may be extracted from the binary data file. Channel data raster image data varies in size according to the native format and the pixel/line resolution. Channel raster images are in raw data format. 3.0 How to Create Executables After creating a suitable working directory on your account, copy in the three read program source files. Begin by editing and setting configuration parameters in the Makefile provided. While it is not necessary that the data files be on the same directory as the read, output report summaries are. 3.1 The UNIX make utility The UNIX Makefile script is a convenient utility which, when set up properly, greatly eases the burden of software management. It is provided to create the executable. However, unless you are using this on a SunOS system, it may not run properly without making modifications. 3.2 Starting out After deciding on a working directory, copy all read software files and note the system type. If you are unsure, entering "uname -a" on the command line will identify the system name and OS version. Then edit the Makefile system dependent options, and set the PROJECT macro to match the mission for your data sets. First thing you should do is to change the name of the makefile from Makefile.cms to Makefile. Otherwise, you will need to enter "make -f Makefile.cms" wherever "make" is referenced in this document. HINT: It's a VERY good idea here to save a read-only backup copy of the original at this point. 3.2.1 Edit Makefile source and set compiler options according to your platform, HP, Sun, etc. 3.2.2 Edit makefile source to set the PROJECT macro to the appropriate mission specific name. A helpful feature was added to makefile to display a list of valid mission names. Choose one name from the list, and assign it to PROJECT. Alternately, PROJECT can be defined from the command line when make is invoked (see 3.1.4). at the command line, enter: make project -or- make names The following list is output to the screen: CMS General Read Makefile (Centre de Meteorologie Spatiale) >>valid project names are: PROJECT = LWFLUX #%% FIRE AX CMS LONG WAVE FLUX PROJECT = SOLAR_DAY #%% FIRE AX CMS SOLAR DAY PROJECT = SOLAR_HR #%% FIRE AX CMS SOLAR HR PROJECT = SOLAR_MN #%% FIRE AX CMS SOLAR MN PROJECT = SOLAR_WK #%% FIRE AX CMS SOLAR WK PROJECT = SST_DAY #%% FIRE AX CMS SST DAY PROJECT = SST_WEEK #%% FIRE AX CMS SST WEEK PROJECT = SST_FNTS #%% FIRE AX CMS SST FNTS PROJECT = SST_CLOUD #%% NOT SUPPORTED >>inspection syntax note: make CSC=INSPECTION_ PROJECT=* 3.2.3 Locate and identify your CMS data files. Double check file names to ensure they agree with mission naming conventions. Note the path if different from the current working directory. Be sure to note whether the "PROJECT" switch is appropriate for your data set. 3.2.4 Clean up the current working directory by removing previous or renaming current report and object files. At the command line, enter: % make clean 3.2.5 Ensure makefile is configured with the correct dependencies. Since dependencies determine relationships which control makefiles behavior, this is a critical step. If this fails, you have to check the system paths to include files hard coded in the program source and set them according to your system. At the command line, enter: % make depend 3.2.6 Create the mission specific data read program executable. At the command line, enter: % make all -or- % make all PROJECT=mission_specific_name (to override the internal PROJECT macro assignment) At this point, you should be ready to run the cms read software. example: make all PROJECT=LWFLUX 3.3 Makefile Errors If you have properly edited the Makefile and set the system and project dependent parameters, you should not be in for any rude surprises. However, errors do occur. Occasionally, the system loader complains loudly when it can not find libraries or perhaps the compiler spews forth apparently meaningless messages. Read software was compiled across several platforms without problem. Should you get errors, edit the Makefile, and go to the comment line marked: # -SYSTEM DEPENDENT CHANGES----------------------------- Check STD_DEFINES and STD_CPP_DEFINES to make sure they are set properly for your system. Also, make sure LD_LIBRARY_PATH has the path to your systems' libraries. If you are unsure, get your system administrator involved. The following is a typical make error response when the PROJECT macro has not been set: (HP-UX Hewlett Packard system) % make cms % rm -f cms % cc -g -DHPUX -I. -DNO_HAVE_PROTOTYPES -DNO_STDARG -D \ % -o cms fire_ax_cms_read.c % cpp: "", line 0: warning 2003: Illegal argument to '-D' option. % cc: "fire_ax_cms_read.c", line 458: error 1588: "title_string" undefined. % cc: "fire_ax_cms_read.c", line 631: error 1588: "chan_name_set" undefined. % cc: "fire_ax_cms_read.c", line 631: error 1528: Subscript expression must combine pointer and integer. % cc: "fire_ax_cms_read.c", line 631: error 1528: Subscript expression must combine pointer and integer. % cc: "fire_ax_cms_read.c", line 672: error 1588: "chan_name_set" undefined. % cc: "fire_ax_cms_read.c", line 672: error 1528: Subscript expression must combine pointer and integer. % cc: "fire_ax_cms_read.c", line 672: error 1528: Subscript expression must combine pointer and integer. % cc: "fire_ax_cms_read.c", line 1121: error 1588: "LAT_DEB" undefined. % cc: "fire_ax_cms_read.c", line 1121: error 1588: "RESOLUTION" undefined. % cc: "fire_ax_cms_read.c", line 1130: error 1588: "LON_DEB" undefined. % *** Error code 1 % % Stop. 4.0 Running the CMS Read Software The read software takes input file names as command line arguments, and at least one data file name is required. >>at the command line prompt, enter: % cms file [ file file .. etc. ] Where: file = your data set file name(s) >>the cms read software will accept and process multiple file names entered on the call line. ----------------------NOTE---------------------- A description of data set file names for each CMS mission can be found in section 5. ----------------------NOTE---------------------- 4.1 Read Software Option Menus The read software requires one command line argument before it will execute. If you enter "cms" from the prompt, the read software displays the following information screen and exits. ******************************************************************* * CMS General Read * * (Centre de Meteorologie Spatiale) * * * * CMS READ PROGRAM: * * designed to interpret CMS binary raster data files. * * requires: * * a) compilation for correct mission (PROJECT in Makefile) * * and platform specific options. (ARFLAGS, STD_DEFINES..) * * b) one or more command line arguments required * * arg1..n = name of input data file(s) * * * ******************************************************************* 4.1.1 The Select Options Menu The following menu is written to the screen. You may elect to dump the header info, header and channel data or dump channel data only. NOTE - data files which contain multiple channel raster images will cause the read software to display another screen, prompting you to select and enter a channel number. Refer to section 4.1.3, Channel Selection Menu. ******************************************************************* * CMS General Read * * (Centre de Meteorologie Spatiale) * * * * Please make a selection from the menu. * * * * 1. DUMP HEADERS ONLY * * 2. DUMP HEADER AND CHANNEL DATA * * (there are 1 channel(s) of raster image data on this file) * * 3. DUMP CHANNEL DATA ONLY * * * * Enter 'x' to exit or terminate processing. * ******************************************************************* 4.1.3 Channel Selection Menu The read software will display the following screen and prompt for additional information only when files contain more than one channel raster data image. ****************************************************** * There are %2d channels found on this file. * * Select a channel number to dump... (1-%2d) * * * * Enter 'x' to exit or terminate processing. * ****************************************************** 5.0 Data Set General Information The following subsections contain a brief description of the various metrics and nomenclature specific to each mission. 5.1 CMS SST DAY These are Sea Surface Temperature (SST) and front intensity images based on NOAA 11 and 12 AVHRR observations aboard the polar obiting satellites. The file naming convention is: tqDDMMYYx.fis where DD = day of the week MM = month of the year YY = year x = 1 NOAA-11 nighttime mosaic 2 NOAA-12 morning mosaic 3 NOAA-11 daytime mosaic 4 NOAA-12 evening mosaic These files are: I1 pixels, 752 pixels/row, 652 rows. Each pixel has a spatial resolution of 0.02 degrees. The units are: temperature [degree C] (offset by ?? deg.) 5.2 CMS SST WEEK These files are weekly temperature fields from observations made by the NOAA 11 and 12 polar orbiting satellites. The file naming convention is: thseXXyyZZ.fis where XX = week number in 1992 yy = jo (jour) day nu (nuit) night ma (matin) morning ZZ = 11 NOAA-11 12 NOAA-12 These files are: I1 pixels, 752 pixels/row, 652 rows. Each pixel has a spatial resolution of 0.02 degrees. The units are: temperature [degree C] (offset by 10 deg.) 5.3 CMS SST FRONTS These files are weekly front intensity fields from observations made by the NOAA 11 and 12 polar orbiting satellites. The file naming convention is: fasteseXX.fis where XX = week number in 1992 These files are: I1 pixels, 752 pixels/row, 652 rows. Each pixel has a spatial resolution of 0.02 degrees. The units are: front intensity [degree C/5km] 5.4 CMS CLOUD It should be noted that unique properties of the satellite CMS Cloud Classification data lead to the creation of separate read software. The software was not designed to process the FIRE ASTEX CMS Cloud Classification data. 5.5 CMS LWFLUX These files are the calculated downward longwave flux at the surface derived from METEOSAT observations. The file naming convention is: raDDMMYYsxx.fis_tmp where DDMMYY is the date and xx = "slot number" Mean time (UT) is obtained from the slot number over the ASTEX region by the formula: UT = (xx/2) - 0.17 These files are: I2 pixels, 188 pixels/row, 163 rows. Each pixel has a spatial resolution of 0.08 degrees. The units of flux are Wm^-2. 5.6 CMS SOLAR DAY These files are calculations of the daily solar irradiance at the surface, based on observations by the METEOSAT. The file naming convention is: esqDDMMYYx.fis where DDMMYY is the date These files are: I2 pixels, 376 pixels/row, 326 rows. Each pixel has a spatial resolution of 0.04 degrees. The header of each file claims there are two channels, although the provided documentation states that there is only one channel per file. The units are: flux [tenths of Joule/cm^2] 5.7 CMS SOLAR HOUR These files are calculations of the daily solar irradiance at the surface, based on observations by the METEOSAT. The file naming convention is: esqDDMMYYx.fis where DDMMYY is the date These files are: I2 pixels, 376 pixels/row, 326 rows. Each pixel has a spatial resolution of 0.04 degrees. The header of each file claims there are two channels, although the provided documentation states that there is only one channel per file. The units are: flux [tenths of Joule/cm^2] 5.8 CMS SOLAR MONTH These files are calculations of the monthly solar irradiance at the surface, based on observations by the METEOSAT. The file naming convention is: esmxx.fis where xx is the month number of 1992. These files are: I2 pixels, 376 pixels/row, 326 rows. Each pixel has a spatial resolution of 0.04 degrees. The header of each file claims there are two channels, although the provided documentation states that there is only one channel per file. The units are: flux [tenths of Joule/cm^2] 5.9 CMS SOLAR WEEK These files are calculations of the weekly solar irradiance at the surface, based on observations by the METEOSAT. The file naming convention is: eshsexx.fis where xx is the week number of 1992. These files are: I2 pixels, 376 pixels/row, 326 rows. Each pixel has a spatial resolution of 0.04 degrees. The header of each file claims there are two channels, although the provided documentation states that there is only one channel per file. The units are: flux [tenths of Joule/cm^2] 6.0 Implementation Notes The read software was compiled and tested successfully on HP and SunOS operating systems and must be configured for your platform. Read section 3.0 carefully for instructions on creating the executable. Please read this entire document carefully. 7.0 Output Files The read software produces two output report files in plain ASCII text format. Since the files are overwritten each time the read software is executed, you will need to rename the *.RPT file after each pass. As an example, the read software has been compiled with the project set to "LWFLUX". So, in this case the following file names will begin with "FIRE_AX_CMS_LWFLUX". The cms read software was run using a representative CMS LWFLUX data set, I2 pixel format as follows: % cms ra220692s11.fis These files were produced by the read software: FIRE_AX_CMS_LWFLUX.LOG - contains running log of program/system status FIRE_AX_CMS_LWFLUX.RPT - summary; varies according on menu option ra220692s11.fis.C1 - binary raster image data extracted from file 7.1 File Header Information This option details key information extracted from the header structures of each data file. The "fis->" keyword denotes the the name of header structure and element being displayed. ***************************************************************************** * * * Langley DAAC * * * * PROJECT_LEVEL/DATA_SET_LEVEL READ SOFTWARE * * FIRE_AX_CMS_LWFLUX * * * * VERSION NUMBER: $Revision: 1.3 $ * * Language: C * * Platforms supported: SUN * * HP * * SGI * * Dec Alpha * * Contact: Langley DAAC Science, User and Data Support Office * * Mail Stop 157D * * Hampton, VA 23681-0001 * * PHONE: (757)864-8656 FAX: (757)864-8807 * * * ***************************************************************************** date: Fri Aug 16 08:26:49 EDT 1996 : ra220692s11.fis Header contents: : fis->Mod mode=RO [0] : fis->de.fil="ra220692s11.fis " originating file name : fis->de.org="PLC " raster data sequence : fis->de.typ="I2 " data type, pixel : fis->de.mxp=188 maximum number of pixels : fis->de.mxl=163 maximum number of lines : fis->de.mxc=1 maximum number of channels : fis->de.auc="orstom " author of program : fis->de.djc=15513 creation date, julian : fis->de.ser="antinoe " originating facility : fis->de.tit="essai " title : fis->de.aum="orstom " author of program : fis->de.djm=15513 current date, julian : fis->de.mis=0 mission code number : fis->de.nim=0 mission sequence number : fis->de.ins=0 numerical code : fis->de.oss=0 orbit number : fis->de.ijr=0.000000 instant date, julian : fis->de.llp=0.000000 physical station number : fis->de.csc=" " instrument orientation : fis->de.anw=0.000000 NW latitude : fis->de.onw=0.000000 NW longitude : fis->de.ane=0.000000 NE latitude : fis->de.one=0.000000 NE longitude : fis->de.ase=0.000000 SE latitude : fis->de.ose=0.000000 SE longitude : fis->de.asw=0.000000 SW latitude : fis->de.osw=0.000000 SW longitude : fis->de.npp=0 number of primary points : fis->de.npl=0 number of primary lines : fis->de.ndp=0 number of fine points : fis->de.ndl=0 number of fine lines : fis->de.ijd=0.000000 start recording time, julian : fis->de.ijf=0.000000 end recording time, julian : fis->de.nlm=0 number of lines per maniquin : fis->de.nor=376 number bytes per record (theoretical) : fis->de.nri=163 number of bytes per raster line : fis->de.nve="f7 sps9 2.1" file processing utility version number : fis->de.nmi=0 number of data images this mission : fis->de.nbr=167 total number of records this file : fis->de.pad=0 number of padding bytes : fis->dd[0].cod="UNUSED " qualification code : fis->dd[0].npr=0 number of primary records : fis->dd[0].ndr=0 number of fine points 7.2 Channel Data Report This summary consecutively lists each pixel value with its corresponding geopositional location. ***************************************************************************** * * * Langley DAAC * * * * PROJECT_LEVEL/DATA_SET_LEVEL READ SOFTWARE * * FIRE_AX_CMS_LWFLUX * * * * VERSION NUMBER: $Revision: 1.3 $ * * Language: C * * Platforms supported: SUN * * HP * * SGI * * Dec Alpha * * Contact: Langley DAAC Science, User and Data Support Office * * Mail Stop 157D * * Hampton, VA 23681-0001 * * PHONE: (757)864-8656 FAX: (757)864-8807 * * * ***************************************************************************** date: Wed Aug 14 17:41:04 EDT 1996 ***************************************************************************** * * * PROJECT_LEVEL/DATA_SET_LEVEL READ SOFTWARE * * explicit listing of longwave flux raster image data by geolocation * * * ***************************************************************************** These data values are extracted from: file: ra220692s11.fis channel number: 1 bytes per pixel: 2 Now do the processing for row 1 Latitude : 39.99 pixel count LONGWAVE FLUX Longitude in the data for row 1 Wm^-2 ------------------------------------------------------------------------------ 1 3837 -29.97 2 3837 -29.89 3 3832 -29.81 .. ..... . . ...... ........ 187 3768 -15.09 188 3677 -15.01 These data values are extracted from: file: ra220692s11.fis channel number: 1 bytes per pixel: 2 Now do the processing for row 2 Latitude : 39.91 pixel count LONGWAVE FLUX Longitude in the data for row 2 Wm^-2 ------------------------------------------------------------------------------ 1 3749 -29.97 2 3837 -29.89 3 3832 -29.81 4 3832 -29.73 .. ..... . . ...... ........ .. ..... . . ...... ........ ....etc.... 7.3 Status Information During the normal course of execution, the read software interacts with the system to open files, and performs sundry housekeeping while executing operator requests. The *.LOG file provides an ongoing record of read software activity as a debugging aid. file name: FIRE_AX_CMS_LWFLUX.LOG type: ASCII text contents: record program error status during operation sample contents: data read in OK: fp_rdi1 on ra220692s11.fis code=1 8.0 ASSUMPTIONS The calculations done to determine the beginning date, ending date, and the processing date were computed from the file naming conventions for the following missions: CMS SOLAR WEEK CMS SST WEEK CMS SST FRONTS example: fasteseXX.fis where XX = week number in 1992 For the year of 1992 the only part of the date that was given in the file name was the week number that the data was recorded in. The dates to use for creating the metadata were determined by calculating which week in 1992 the week number taken from the file name was. This allowed for the creation of a date like the following example: 01/06/1992. Since the first week in January 1992 began on Wednesday, it was determined that for the purposes of creating the date based upon the week number that the week number would be assigned to the Monday of each week in the month. Because the first Monday in January began on the sixth the first week in January would began on this date. By using the date of each Monday in the month a total of fifty two Mondays would be assigned for the year of 1992.