FORMAT FOR THE DMSP DATABASE DMSPDBASE1.FOR reads and analyzes a data tape and loads the data into two databases while producing a plot file for each polar crossing. The potential analysis and graphing routine are based on the DMSPPOTMOD3.FOR program. The short database consists of the magnitude and locations of the electrostatic potential maximum and minimum, the zero crossing, current values of Kp and IMF, etc. during one polar pass. The longer database consists of the values of the potential, horizontal, and vertical flows averaged into four-second bins (roughly corresponding to 0.25 degree of latitude or 30 kilometers) along with the corresponding magnetic latitude, invariant latitude, magnetic local time. geographic latitude and longitude. Each pass in both databases has with a unique index number that also appears on the plot so that they can be easily correlated. Short database format (DMSPARAM.DAT) The short database containing the most significant calculated parameters from each pass is a formatted file so that the data can be easily scanned by the user from a terminal. It is a indexed database keyed on the sfindex variable to allow for quick searches. SFINDEX is the keyed index number that uniquely identifies each polar crossing. It is an eleven character string comprised of satellite identification number (digits 1,2), the year number (digits 3,4), the day of year number (digits 5 through 7), the hour number (digits 8,9) and the minute number (digits 10,11). The time chosen is the time the satellite crosses the equator beginning a hemisphere crossing. Thus the file for the F8 crossing starting at 14:08 UT on September 1, 1987 is 08872441408 PSIMAX is the electrostatic potential maximum (in kV) during the polar crossing. The location of the maximum is given by the magnetic local time (SCMLTMAX), invariant latitude (INVLATMAX), and the magnetic latitude (IMLATMAX). The potential minimum and its location are likewise described by PSIMIN, SCMLTMIN, INVLATMIN, and IMLATMIN. IQUALFLAG is a three digit quality flag for the pass. The first digit tells how close the satellite got to the magnetic pole during the crossing. If it got within five degrees the first digit is 3, between five and ten degrees the first digit is 2, and between ten and fifteen degrees the first digit is 1. If the satellite did not get any closer than fifteen degrees, the pass is not usable and the first digit is set to 0. The second digit is a measure of the scatter of the data in the polar region: 2 = good, 1 * fair, and 0 = unusable. The third digit tells whether the total potential drop is greater or less than 40 kV, and, if the potential difference is greater than 40 kV, it also tells which of the three Heppner-Maynard patterns it most closely matches. (NOTE: this matching to the H-M models is the same algorithm as used in DMSPPOTMOD2.FOR and DMSPPOTMOD3.FOR and ONLY WORKS FOR SATELLITES IN A DAWN-DUSK ORBIT. For F9 and subsequent "noon-midnight" satellites another criteria will be chosen.) For the third digit, 3 means the DE model, 2 means the A model, 1 means the BC model, and 0 means the potential difference is less 40 kV. The location of the zero potential crossing between the potential maximum and minimum is given by ZEROMLT and IZEROMLAT. (This assumes that the convection pattern is the typical two cell pattern seen during southward IMF and there is only one zero between the maximum and minimum. For cases during northward IMF there may be more than one zero between the extrema. In such cases the program is recording only the first zero seen by the spacecraft after the first extreme.) PLMIDPOTSF is the electrostatic potential (in kV) at the midpoint along the track between the extrema, and its location is given by PLMIDMLT and IPLMIDMLAT. POTMEANSF is the electrostatic potential (in kV) of the mean of the extrema, and its location is given by POTMEANMLT and IPOTMEANMLAT. KPSHORT is the Kp index at the time of the first extreme. The program saves the Kp value at the time of the beginning and at the time of the ending of the entire hemisphere pass. Since Kp changes every three hours and each hemisphere pass is 54 minutes in length, most of the passes will have the same value for both Kp. But for the passes that straddle two different Kp values, the program determines which Kp value was in effect at the time of the first extreme. The values for Kp are converted from standard form to a numerical value by the following code: "0" = 0, "0+" = 1. "1-" = 2, "1" = 3, "1+" = 4,...., "9+" = 28. IAEINDEX and BXSHORT, BYSHORT, and BZSHORT are (respectively) the hourly AE index and the hourly IMF (GSE) components in effect at the time of the first extreme. The program saves the values of these variables at the times ofthe beginning and the ending of the hemisphere pass. Since the AE index and the IMF data available to us is in hourly averages and each hemisphere pass is 54 minutes in length, then most of the passes wi:l have different values for the beginning and ending variables. The program determines in which hour the first extreme fell and then assigns the appropriate values for AE index and IMF to the database. (NOTE ON AVAILABILITY OF AE AND IMF DATA: We are in the process of obtaining the AE hourly averages, but do not currently have them and we are not sure when they will be available. So that the building of the database is not unduly delayed, the program current saves a fill value of 990 for all AE index values. The 990 fill value indicates that no data are currently available, but there may be some in the future, as opposed to the standard fill value of 999 which indicates the data are permanently missing. The IMF data are taken from the NSSDC OMNI file which currently extends through mid October of 1988. This means that we now have data covering the first sixteen months of DMSP-F8's lifetime. This data set is only about 40-50% complete due to the time when the IMP-J satellite is inside the Earth's magnetosphere. Periods when there is no IMF data are indicated by zero values for all three IMF components. The IMF database will be updated as soon as further data become available from the NSSDC.) IPOTOFS is the offset (rounded down to the nearest kilovolt) from the electrostatic potential calculation. CHANGES FROM THE DBASE2 SHORTFILE FORMAT TO THE DBASE4 SHORTFILE FORMAT: IQUALFLAG is now a 2-digit integer. The tens digit is the IQFLAG (Quality Flag) which is unchanged from the original shortfile. The 1's digit is the model number (IMODNUM) classification and ranges from 0 to 7 as follows: 0 northward IMF (delta PSI < 40 kV, highest mag.lat > 75 deg.) 1 HM model BC 2 HM model A 3 HM model DE 4 unusuable (zero occurred in far nightside or else pattern too distored to classify unambigously) 5 skimmer (unusable - deltaPSI < 10 kV) 6 unusable (either observed maximum or minimum was zero) 7 unusable (one of the corrections > 10 or negative, pattern likely too distorted to classify unambigously.) CORRECTMAX and CORRECTMIN are scaling factor to be multiplied by the observed maximum and minimum potentials to get the estimated global maximum and minumum potentials in the model given by the IMODNUM index. MLATHIGH is the highest magnetic latitude reached by the spacecraft on this pass. The format and size of the short database (DMSPSHORT.DAT) is as follows: OLD SHORTFILE FORMAT CHANGED NEW SHORTFILE FORMAT item format item format SFINDEX All SFINDEX All PSIMAXSF F5.1 * PSIMAXSF F5.1 SCMLTMAX F4.1 SCMLTMAX F4.1 INVLATMAX I4 ** INVLATMAX I4 IMLATMAX I4 ** IMLATMAX I4 PSIMINSF F6.1 * PSIMINSF F6.1 SCMLTMIN F4.1 SCMLTMIN F4.1 INVLATMIN I4 ** INVLATMIN I4 IMLATMIN I4 ** IMLATMIN I4 IQUALFLAG I3 X IQUALFLAG I3 ZEROMLT F4.1 ZEROMLT F4.1 IPLMIDMLAT I4 ** IPLMIDMLAT I4 POTMEANSF F7.1 * X CORRECTMAX F6.3 POTMEANMLT F5.1 X CORRECTMIN F6.3 IPOTMEANMLAT I5 * X MLATHIGH I4 KPSHORT I2 KPSHORT I2 IAEINDEX I3 IAEINDEX I3 BXSHORT F5.1 BXSHORT F5.1 BYSHORT F5.1 BYSHORT F5.1 BZSHORT F5.1 BZSHORT F5.1 IPOTOFF I4 IPOTOFF I4 total: 24 variablespass. * electrostatic potentials are given in kilovolts ** all magnetic latitudes and invariant latitudes are multiplied by 10 and recorded as integers in order to save the space occupied by the decimal points, this is automatically corrected when the data is read by the program READDMSPPARAM.FOR Long database format (DMSPPOTBIN.DAT) The long database is an unformatted database which saves all the flow data and potential data into bins of four-second long averages along with the standard deviations of both the horizontal and vertical data and the number of points averaged in each four-second bin. The corresponding magnetic latitude, magnetic local time, invariant latitude, geographic latitude, geographic, and time (in seconds) for each four-second block are also saved. These values correspond to the midpoint in time of each four-second block. The program saves the number of data points used in each four-second bin to serves as a check against missing data and/or possibly bad data samples. Normally each bin contains the average of 24 raw data points. The program saves the values of Kp, AE index, and the IMF at the beginning time of the hemisphere crossing and at the ending time of the hemisphere crossing. It further determines the IMF values for the hour prior to the beginning time of the crossing. This can be used later in searching for periods of steady IMF conditions. The program also saves the number of four-second bins in the file as a check against missing data. A nominal hemisphere crossing consists of 810 bins corresponding to 54 minutes of data. The long database is a sequential file so that it can be manipulated using standard DEC database management programs. Format form of the long database (LONG.DAT) item description SFINDEX identification index number of the pass BXPREV \ BYPREV I IMF components for hour Prior to starting time BZPREV / BXSTRT \ BYSTRT I IMF components for hour of starting time BZSTRT / BXSTOP \ BYSTOP I IMF components for hour of ending time BZSTOP / IAEINDEXSTRT AE hourly average at starting time IAEINDEXSTOP AE hourly average at ending time KPSTRT Kp value at starting time KPSTOP Kp value at ending time IMAX number of four-second bins in this pass CHF correction factor used to zero the horiz flow baseline CVF correction factor used to zero the vert flow baseline ________________________________________________ I XuTIME (sec) time (UT) at the midpoint of this bin I FLwH3 (km/s) horizontal flow averaged for the four-second bin (note 1) I FLwv3 (Am/s) vertical flow averaged for the four-second bin (note 1) I STDEVH (km/s) standard deviation of the horizontal flow data in this bin I STDEVV (km/s) standard deviation of the vertical flow data in this bin I NPTS number of telemetry points averaged in this bin I POTLNG (kV) average electrostatic potential for each bin (note 2) I SCCHMLAT magnetic latitude at midpoint of this bin I sCCHMLTM magnetic local time at midpoint of this bin I sCCHINVLAT invariant latitude at midpoint of this bin I SCCHLAT geographic latitude at midpoint of this bin I SCCHLONG geographic longitude at midpoint of this bin ------------------------------------------------- This section repeats 810 times (once for each of the four-second bins in the rss-) Note 1: The flow values here are the values after corotation has been removed BUT NOT after performing the constant offset to all points done in the program in order to force the ends to go to zero. Thus these numbers may not match the flow values on the plot, but differ by a dc offset. They will, however, match the curves on the "quick-look" plots. In order to save space in the database. each element of the flow data has 3.0 km/s added to it to remove the negative sign. This extra amount of 3.0 is automatically removed whenever the data is read using the READDMSPPOTBIN.FOR template. Note 2: The potential is calculated from 50 degrees magnetic latitude poleward (except for cases where the auroral circle has expanded equatorward). Thus the potential at all points equatorward of that is defined (and saved) as zero. Disk space and CPU usage. One day (28 hemisphere passes) requires 48 minutes of CPU time to process the data files and generate the plot files. Taking the option to skip the plotting and merely generating the data files requires about 13.5 minutes of cpu time. The 28 plot files require 5128 blocks of disk storage. Thus an entire tape of ten days would require just over 50,000 blocks. These plots will be printed and filed in notebooks, so none of them will be kept permanently on the disk. The short database requires 18 blocks per day, while the long database requires approximately 2200 blocks per day. Adding those two together and rounding up gives us about 2300 blocks of data per day. Thus each satellite would generate something on the order of 840,000 blocks of data per year, One optical disk can hold 1 gigabyte of data or roughly 1.9 million blocks of data. This means we could place about two and quarter years worth of data on each optical disk. Marc Hairston--University of Texas at Dallas--Center for Space Sciences REVISED: 18 FEB 93