1.0 0 Introduction

The UFILES data file system was developed at the Princeton Plasma Physics Laboratory (PPPL) in the early 1980s as a basis for standardized communication and processing of numeric data. Any single precision floating point numeric data function of one dimension f(x), two dimensions f(x,y), or three dimension f(x,y,z) may be stored in a UFILES data file. In addition, scalar data may be stored in keyword-list based UFILES structure. Each file is internally documented, and space for labels and physical units of all quantities is provided. The identification of the origin of the data -- 5 or 6 digit "shot" number, 4 character device (tokamak) id, and shot date-- was originally adapted for data that originates from a PPPL tokamak diagnostic. Traditional UFILES files are written in ascii in fixed FORTRAN formats in fixed order, with one function per file, assuring the simplicity, transportability, and speed of UFILES read/write subroutines. In recent years, portable compressed binary, NETCDF and HDF UFILE format variants have been invented. UFILE read routines support all formats automatically; the format on write defaults to ascii.

The UFILES system was originally created in 1982 for the PDX tokamak, to facilitate cross-computer communications for the relatively small fraction of experimental data which was selected for detailed off-line analysis. Standardization of data through the UFILES system greatly simplifies the problems of plotting, averaging, smoothing, and inputting data into the analysis codes, because standardized routines for these functions could be provided, instead of requiring special routines for each data type. The relative simplicity of the UFILES system not only facilitates inter-computer communications but provides a means of substantially reducing the amount of computer code needed in support of data acquisition for the off-line analysis codes.

The purpose of this document is to describe UFILES and related software. The document is divided into two sections. The first section provides a more detailed introduction to the UFILES system and how it aids the flow of data from experiment to analysis. The second section deals with the UFILES system itself. The UFILES data structures are described, and all information (FORTRAN calling sequences) needed to create and/or access a UFILES file are provided.

System standardization through installation of UFILES has eased software maintenance problems by allowing the "retirement" of thousands of lines of coding. Interchangeability of measurements of the same quantity by different diagnostics (for example, far infrared laser and microwave interferometer line integral electron density measurements) has been greatly simplified through data format standardization under UFILES. UFILES has made possible the sharing of considerable quantities of data handling software between the PLT, PDX, and TFTR experiments. UFILES standardization has enabled a substantial increase of data through-put in the TRANSP analysis system: nearly three times as many complete analyzable datasets in 1982 when compared with 1981, before UFILES was introduced.

1.1 1 Overview

UFILES unites data under a standard format. Therefore problems associated with the data, such as access or display, are united, and need only be solved once instead of once for each type of non-standard data.

Of course, this important result can be obtained through any standardization technique. For example, it may be obtained by a data archiving system-- a system which takes the digitized output of diagnostics on an experiment, standardizes this data, produces fast displays (within seconds of the running of the experiment) and automatically stores the data in a standard accessible form on a designated storage device for later access. However, in order to handle at high speeds the huge quantities of data automatically produced by each experimental cycle, an archiving system must often be highly specialized to its local hardware environment. The result is that while archived data may be conveniently accessible on the computer system controlling data acquisition for the experiment, it is likely to require translation before it can be introduced into any other computer system. Furthermore, if two similar experiments are running with different data acquisition systems, it may be advantageous to use the same programs to analyze data from both experiments-- but the format of the archived data from each experiment is incompatible. Thus, a second tier of data standardization is desirable; a standard to which data from all experiments on all computers may be converted. The system supporting this standard need not have the ultra-high volume capacity of an archiver, since only selected data need be translated to it. the UFILES system is intended to be such a "second tier" system, and to provide features which would be incompatible with the high-volume requirements of an archiver, while discarding complicating features where possible.

The following table provides a summary comparison of UFILES to a hypothetical archiver system:

UFILES Archiver Single computer system optimization: no yes Multi-computer system support-ability: yes no Efficient binary data structure: yes yes Transportable ASCII data structure: yes yes Multi-channel data structure capability: no yes "One function per file" simplicity: yes no Annotated self-documenting files: yes no Under 2000 lines code: yes no 100% FORTRAN source code: yes no

The second tier UFILES system affords great opportunities of simplification when compared to the archiver. This is because it may be more completely divorced from the complications involved in the details of maintaining individual diagnostics on an experiment. Thus the archiver may be required to store all data associated with a given diagnostic during a particular shot, including calibration, offsets, "intermediate" traces, etc., so that the diagnostic physicist can trouble-shoot his device when problems arise, or, extract "by hand" results which are inaccessible to automated real-time software. The UFILES system, on the other hand, needs only save the "result", i.e. the physical quantity that was successfully measured.

Because of the relative simplicity of "results" data, the UFILES system supports just four numeric data file architectures:

0d: Scalar data file. A list of scalars (floating point numbers) identified by a keyword, label, and physical units. 1d: A list of scalars (optional) followed by a single numeric data function of the form f(x), and labels and physical units for the dependent and independent coordinates. 2d: A list of scalars (optional) followed by a single numeric data function of the form f(x,y), and labels and physical units for the dependent and 2 independent coordinates. 3d: A list of scalars (optional) followed by a single numeric data function of the form f(x,y,z), and labels and physical units for the dependent and 3 independent coordinates.

These file architectures will be described in detail in section 2. Note that only one scalar list and/or one numeric function result is allowed per file. This restriction allows an enormous simplification in UFILES software (mapping out the data contents of each file is very simple) compared to archiver systems. As a result the entire system consists of only a few hundred lines of 100% FORTRAN code, and may be brought up on a new computer in a matter of hours.

Because UFILES deals only in selected results data, the size of individual data files might be unimportant. If so, an all-ASCII format may be employed. This, of course, is a considerable aid in terms of making data transportable from computer to computer without requiring additional translation software. Lines in the file may be tagged to aid readability of the file, and comments may be carried along to document the origin and history of the data. UFILES supports a comment-spooling capability (subsection 2.5) which is utilized by all standard UFILES-based utilities, so that comments from all prior processing of the data are retained, not just comments from the most recent transformation. Thus a UFILES file becomes a document containing considerable information which may be accessed simply by printing the file, without requiring any special access software at all.

For the situation where file size is a problem a more sophisticated version of UFILES has been developed which makes available an integerized compressed binary UFILES format maintained in parallel with the standard ascii format. NETCDF and HDF UFILES formats also exist.

1.2 2.0 Description of the UFILES System

This section will describe in detail the file structures supported by UFILES and the subroutine calls to read or create these structures. Accessing (or creating) UFILES data is a five step process

1. The association of a FORTRAN logical unit number (LUN) with a filename form, device (usually a disk - VMS only), and a directory path. 2. A shot (experiment) number is supplied (this completes the filename whose form was specified in step 1), and the UFILES file is opened for read or write access. Some computer operating systems dependence is to be expected in the implementation of steps 1 and 2, since different operating systems have different ways of identifying devices and files. Steps 1 and 2 are described in detail in subsection 2.1. 3. For all UFILES data structures a single subroutine call is sufficient to transfer the entire data contents to/from the file from/to program memory. The data structures and read/write calls are described in subsections 2.2, 2.3, and 2.4 below. 4. This step is optional, but may be important: if a file is being written, comments may be appended to the end of the file after the standard UFILES data write is complete. From the point of view of the UFILES system, any non-standard data is a "comment". Subroutine USPOOL (subsection 2.5) may be used to transfer ascii comments from an input file to an output file, including binary formatted UFILES. If a file is being read it will be possible to access the comments at the end of the file, after the standard UFILES format data has been read. 5. Close the file.

When writing a UFILES file there is an additional optional step which allows the user to specify that a non-ascii form of UFILES be written.

UFILES error messages are discussed in subsection 2.7.

UFILES runs differently on VMS and UNIX. On UNIX the "disk name" is ignored, the syntax is slightly different, and filenames are case sensitive.

1.3 2.1 File Identification, OPEN and CLOSE

UFILES file identification is a two step process

1. Subroutine UFSETR is utilized to associate a FORTRAN logical unit number (LUN) with a physical device (usually a disk), directory path, and filename form which lacks only a shot (experiment) number to complete the filename. The UFILES system allows up to 32 distinct filenames- form/LUN associations to be defined simultaneously. 2. The missing shot number is supplied and the file is opened with subroutines UFOPRD (read access)or UFOPWR (write access). This two-step procedure is convenient for accessing a series of files with the same name form (i.e. usually data originating from the same diagnostic for a series of shots): a single call to UFSETR may be followed by multiple calls to UFOPRD or UFOPWR on the same LUN (as long as only one file is to be open at a time on that LUN).

Subroutine UFSETR establishes the connection between a LUN and a filename form. Because different computer operating systems accept different filename forms, UFSETR calls tend to have some operating system independence.

On VMS systems, UFSETR may be used as follows:

CALL UFSETR(ILUN,'Q','XTN','DSK1:','TRANSP.PDX')

This associates Fortran logical unit member ILUN with files DSK1:[TRANSP, PDX]Q*.XTN. Note that UFSETR has five arguments: (integer) logical unit number, and (character) file name prefix, suffix, disk, and directory path.

On UNIX systems, filename case sensibility is an issue. For historical reasons having to do with VMS portability, the default is for the filename prefix and suffix to be folded to upper case. To reset the default, use the UFNCAP routine prior to the first UFSETR call, as follows:

CALL UFNCAP(0)

force lowercase filename prefix and suffix

CALL UFNCAP(1)

force uppercase filename prefix and suffix (default)

CALL UFNCAP(2)

use case as passed to UFSETR

Note that UFNCAP effects the case of the filename prefix and suffix components only - not the directory path.

For example:

CALL UFNCAP(1) CALL UFSETR(ILUN,'q','XTN',' ','/usr/users/dmccune/data')

associate files /usr/users/dmccune/data/Q*.XTN to the FORTRAN unit number ILUN. Note that the 4th argument (disk name) should be left blank.

On both VMS and UNIX systems, the current working directory is assumed if the directory argument is left blank.

UFILES imposes no restrictions on the filename beyond the form defined by UFSETR, but use of some convention is recommended. For example, PDX UFILES data may be identified as to diagnostic data-type by the 3-character extension in the filename. Files ending in "T1T" are TVTS electron temperature profiles; files ending in "FIR" are far-infrared laser line-averaged density traces. The prefix character is used to indicate the processing state of the data, e.g. "T" for translated (unmodified) results data, "A" for data constructed by averaging several shots; "S" for data that has been smoothed. However, such organization of filenames is imposed by the user of UFILES, not the UFILES system itself.

If an interactive applications program requires that the user identify a UFILES file, interactively via the command line, this may be facilitated by the following call:

CALL UFNTER(LUN,LABEL)

where LUN is the logical unit number and LABEL is an up-to 20 character description of the file's functionality (e.g. "INPUT DATA FILE" or "OUTPUT: SMOOTH DATA").

UFNTER uses the UREAD i/o controller (see http://w3.pppl.gov/~pshare/help/uread_hlp.html) to talk to a user at a terminal. UFNTER displays the file label and prompts the user for the directory, filename prefix and extension. When the necessary data has been supplied and verified a call to UFSETR is made, associating LUN with the specified file name form.

Once the association between FORTRAN LUN and filename form has been established, only the 5 digit shot number need be supplied to complete identification of a specific file. This is done only when the file is actually to be opened. Two UFILES subroutines are supplied for this purpose:UFOPRD and UFOPWR. Since the operating system dependent features of the filename have been taken care of by the UFSETR call, these subroutines look the same on all computers:

CALL UFOPRD(LUN,ISHOT,IER)

opens a file for read-only (sharable) access on FORTRAN unit LUN. ISHOT contains the 6 digit shot number between 1000 and 999999. The balance of the filename determined by the last call to UFSETR with the same LUN argument. IER is a completion code: 0 denotes success, 1 denotes that an error occurred trying to name or open the file.

CALL UFOPWR(LUN,ISHOT,IER)

opens a file for write access (the file is created). The arguments have the same significance as in UFOPRD.

CALL UFCLOS(LUN)

Closes the file currently open on FORTRAN unit LUN after reading or writing.

Note that the routines UFSETR, UFOPRD/UFOPWR, and UFCLOS must be used in the correct order. If UFCLOS is called for unit LUN before a file has been opened with UFOPRD or UFOPWR on that LUN, an error message results; if UFOPRD or UFOPWR is called before UFSETR an error message results because the filename cannot be formed. If UFSETR is called for unit LUN when a UFILES file is open on that unit an error will result. However, if no file is open the UFSETR call re-assigns LUN to the specified new filename form. The UFCLOS call should be used rather than a FORTRAN CLOSE(UNIT=LUN) statement so that the UFILES system knows which LUNs are free to be re-assigned.

Of course, in normal operation a call to a UFILES read or write subroutine will be made between the call to UFOPRD or UFOPWR and the call to UFCLOS. These read/write subroutines are described in the next three subsections.

1.4 2.2 Scalar data file structure and access

One type of UFILES data structure is a list of keyword-identified scalar numbers. This type of data may constitute the entire file or be the first part of a UFILES one-dimensional,two-dimensional, or three-dimensional data file. The latter file types will be considered in subsections 2.3,2.4, and 2.5. An example of an ascii UFILE containing only scalars is shown in figure 2.1.

Each scalar consists of a single number, stored in the file in FORTRAN 1PE13.6 format. Associated with each scalar is a 30 character label which is divided into a 10 character colon-terminated keyword, a 10 character descriptive label, and a 10 character physical units label. The following is a record-by-record description of the contents of UFILES scalar data files.

RECORD 1: DATA ID [1X,I5,1X,A4,1X,I1]: 5 digit shot number, 4 character source tokamak id, and maximum dimensionality of data in the file (0, 1 or 2. 0 for a scalar data file, 1 for a one-dimensional data file, and 2 for a two-dimensional data file. RECORD 2: SHOT DATE [1X,A10]: 10 character shot date, i.e. date when the experiment was run which produced the data in the file. UFILES does not require that the date be in any particular format. RECORD 3: NUMBER OF SCALARS IN FILE [1X,I3]: number of scalars, NSC, in the data file. In 1d and 2d UFILES files NSC=0 is allowed; in 0d (scalar) files NSC must be greater than or equal to 1. RECORDS 4,6, ... 3+2*NSC-1: SCALAR QUANTITIES: [1X,1PE13.6]: scalar data in floating point format with 5 digit precision. RECORDS 5,7, ... , 3+2*NSC: SCALAR LABELS: [1X,3A10]: 30 character labels for each scalar quantity. the first 10 characters are reserved for the keyword which must end in a colon leading blanks, and the character preceding the colon must not be a blank. The second 10 characters are reserved for a descriptive label (arbitrary format), and the last 10 characters are reserved for a physical units label (arbitrary format).

All subsequent records in the file are comment records which are not accessed by UFILES read/write subroutines and are therefore in free format (cf. subsection 2.5).

An applications program which is to access a UFILES scalar data file must provide array space for the scalar data and labels. In other words declarations of the form:

PARAMETER (NSCMAX= maximum number of scalars; array dimension) REAL SCVAL (NSCMAX) (scalar data array) CHARACTER*4 TDEV (tokamak id) CHARACTER*10 SHDATE (shot date) CHARACTER*10 SCLAB(3,nscmax) (scalar labels array)

must be made.

The UFILES subroutines to read and write scalar data files are UF0DRD AND UF0DWR respectively. To write a scalar UFILES file first the file is named and opened (as described in subsection 2.1 above), and then:

CALL UF0DWR(LUN,TDEV,SHDATE,SCVAL,SCLAB,NSC,IER)

Note that the shot number is not passed. This was defined by the prior call to UFOPWR.

FORTRAN supports a variety of ways of filling the SCLAB labels array. The DATA statement probably supplies the conventional technique, but if the label array is too large EQUIVALENCEing smaller arrays may be required to avoid exceeding the maximum number of continuation lines in the DATA statement. Elements of the label array may also be defined or redefined with assignments ("=") statements. Contiguous blocks of the label array may be filled with WRITE statements, but care must be taken that characters are coded for each element, so that descriptive or physical units labels do not intrude into keyword fields.

To read a scalar UFILES file the file is first named and opened with calls to UFSETR and UFOPRD, and then:

CALL UF0DRD (LUN, TDEV,SHDATE,SCVAL,SCLAB,NSCMAX,NSC,IER)

UF0DRD may also be used to read only the scalar data in a one- or two-dimensional UFILES data file. A warning that the one- or two- dimensional file is being read as if it were a scalar file will be generated.

After a UF0DRD call has been made, individual scalars may be extracted by calls to the INTEGER function IUFKEY, which returns the index location of a scalar identified by a particular keyword. for example, if the scalar file Z37065.NTN illustrated in figure 2-1 has been read in, the following code:

ILOC=IUFKEY('T_AVGLIM1',SCLAB,NSC) IF(ILOC.EQ.0) deal with error: no such key "T_AVGLIM1:" TAVG1 = SCVAL(ILOC)

would extract the number of beams as an integer;

ILOC=IUFKEY('T_AVGLIM2',SCLAB,NSC) IF(ILOC.EQ.0) deal with error: no such key "T_AVGLIM2:" TAVG2 = SCVAL(ILOC)

would extract the power injected by the second beam.

The keyword-delimiting colon may be omitted or included in the keyword string argument in the call to IUFKEY.

figure 2.1 - a scalar UFILE

37065RUN 0 0 6 ;-SHOT #- F(X) DATA -UF1DWR- 10-Jul-98 RPLOT DATA ;-SHOT DATE- UFILES ASCII FILE SYSTEM 12 ;-NUMBER OF ASSOCIATED SCALAR QUANTITIES- 3.7065E+04 ;-SCALAR, LABEL FOLLOWS: NSHOT: SHOT NO. 0.0000E+00 ;-SCALAR, LABEL FOLLOWS: RUNLABEL: TFTR.88 37065Z15 0.0000E+00 ;-SCALAR, LABEL FOLLOWS: TRANSFORM:NONE 4.0000E+00 ;-SCALAR, LABEL FOLLOWS: T_AVGLIM1: 1ST TIME SECONDS 4.5000E+00 ;-SCALAR, LABEL FOLLOWS: T_AVGLIM2: 2ND TIME SECONDS 9.4245E+15 ;-SCALAR, LABEL FOLLOWS: MNEUT: DATA (+) N/SEC 3.8480E+13 ;-SCALAR, LABEL FOLLOWS: FBNTS: DATA (+) N/SEC 7.6664E+13 ;-SCALAR, LABEL FOLLOWS: FTNTS: DATA (+) N/SEC 6.8639E+15 ;-SCALAR, LABEL FOLLOWS: BTNTS: DATA (+) N/SEC 4.2099E+15 ;-SCALAR, LABEL FOLLOWS: BBNTS: DATA (+) N/SEC 9.7911E+14 ;-SCALAR, LABEL FOLLOWS: NEUTX: DATA (+) N/SEC 1.2168E+16 ;-SCALAR, LABEL FOLLOWS: NEUTT: DATA (+) N/SEC ;----END-OF-DATA-----------------COMMENTS:----------- WRITTEN BY PLOTR FROM RPLOT RUN DATA ON 10-Jul-98 AFTER MULTIPLOT PT DISPLAY MODE; PLOT LABEL: NEUTRON EMISSION (XNEUT) VS TIME ... RUNID = 37065Z15 ; RUNLB2 = TFTR.88 37065Z15 USER COMMENTS: this is an example of a scalar Ufile containing neutron data written out from the TRANSP RPLOT program.

1.5 2.3 One-dimensional data file structure and access

Perhaps the most commonly used UFILES data structure is the one-dimensional data file. This consists of an optional set of keyword scalar data in the format described in the previous subsection, followed by a numeric representation of a function of a single independent coordinate: f(x). Both the function value and the independent coordinate are written out explicitly and labels are supplied for both f and x. UFILES therefore expects x as well as f to be supplied as an array of numbers. UFILES does not require that the values of x be monotonically increasing as conventional, but many UFILES- based utility programs do make this assumption.

A sample UFILES one-dimensional data file, S84011.FM1, is shown in figure 2-2. The start of the file contains information on data origin and keyword scalar data in the format described in the previous subsection. This is followed by labels for the independent coordinate x and the dependent coordinate f(x). Then there is a "processing code" used by UFILES utilities to determine if given data has been already averaged and/or smoothed. Then the number of point of one-dimensional data is given, followed by the data itself.

figure 2.2

84011TFTR 1 0 6 ;-SHOT #- F(X) DATA -UF1DWR- 10-Jul-98 1995 ;-SHOT DATE- UFILES ASCII FILE SYSTEM 1 ;-NUMBER OF ASSOCIATED SCALAR QUANTITIES- 3.0647E+00 ;-SCALAR, LABEL FOLLOWS: T0: TIME (SECONDS) RADIAL POSITION CM ;-INDEPENDENT VARIABLE LABEL- FM INV. DENSITY (CM**-3) ;-DEPENDENT VARIABLE LABEL- 2 ;-PROC CODE- 0:RAW 1:AVG 2:SM. 3:AVG+SM 30 ;-# OF PTS- X, F(X) DATA FOLLOW: 1.650000E+02 1.716667E+02 1.783333E+02 1.850000E+02 1.916667E+02 1.983333E+02 2.050000E+02 2.116666E+02 2.183333E+02 2.250000E+02 2.316666E+02 2.383333E+02 2.450000E+02 2.516666E+02 2.583333E+02 2.650000E+02 2.716666E+02 2.783333E+02 2.849999E+02 2.916666E+02 2.983333E+02 3.049999E+02 3.116666E+02 3.183333E+02 3.249999E+02 3.316666E+02 3.383333E+02 3.449999E+02 3.516666E+02 3.583333E+02 0.000000E+00 0.000000E+00 1.859729E+13 1.992839E+13 2.092568E+13 2.158290E+13 2.245249E+13 2.387198E+13 2.480670E+13 2.441662E+13 2.470550E+13 2.602110E+13 3.451269E+13 4.712587E+13 6.613856E+13 8.414732E+13 9.995340E+13 1.097628E+14 1.154937E+14 1.159537E+14 1.055719E+14 8.211321E+13 5.177535E+13 2.673841E+13 2.145849E+13 2.290933E+13 2.160770E+13 1.683392E+13 1.440870E+13 4.005486E+12 ;----END-OF-DATA-----------------COMMENTS:----------- USER COMMENTS: this is the last time point extracted from the profile evolution Ufile provided by Mike Zarnstorff THIS FILE CONTAINS DATA EXTRACTED FROM A 2D UFILES FILE WITH THE SAME SHOT NUMBER, BY UTILITY PROGRAM EXTRAC 2D UFILES INPUT FILE: S######.FMA Ufile copy written by TRDAT. "F" data compression, n = 7710 range = 0.0000E+00 to 1.1595E+14 >error classification maximum value average value absolute 1.7029E+09 1.1244E+08 relative 7.1279E-04 9.1799E-06 rel (|data| > 0.1*|max|) 1.2722E-04 3.0465E-05 ... number of words after compression: 2154 PRODUCED BY PROGRAM GSMOO2 [TRANSP] >> DATA FROM INPUT FILE: "MY_UFILES:A84011.FMA " DATA "X" AND "Y" AXES INTERCHANGED FROM INPUT DATA SMOOTHING ALGORITHM NAME: FILTR6.FOR USER COMMENTS: interchange axes, 20ms smooth

The record by record description of the 1d UFILE follows:

RECORDS 1, ... , 2*NSC+3: shot number, tokamak id, dimensionality (=1), shot date, number of scalars (NSC), and scalar data. Format descriptions are given in subsection 2.2 above - these records are identical to corresponding records in UFILES scalar data files. However, in one-dimensional data file NSC=0 is allowed. RECORD 2*NSC+4: INDEPENDENT COORDINATE (X) LABEL:[1X, 3A10]: 30 character label for the one-dimensional data x coordinate. The first 20 characters are reserved for the coordinate name; the last 10 characters for physical units. If the data is to "concatenated" with other data of a similar form to create a function of 2 independent coordinates (cf. subsection 2.4 below), it may be desirable to leave the 11th thru 20th character positions in the name blank, because these will be inaccessible to graphics routines utilized by UFILES-based utilities for two-dimensional data. RECORD 2*NSC+5: DEPENDENT COORDINATE (F) LABEL: [1X, 3A10]: 30 character label for the one dimensional data function (f) coordinate. The first 20 characters are reserved for the name; the last 10 characters for physical units. RECORD 2*NSC+6: PROCESS CODE [I1]: Integer code used by UFILES-based utility programs. =0 to indicate unprocessed data, =1 to indicate averaged data (formed from more than one shot), =2 to indicate smoothed data, =3 to indicate averaged and smoothed data. Thus if the smoothing utility GSMOO1 is used on data with the processing code set to 2, a warning message to the effect that the data has already been smoothed, will be generated. But the user will not be prevented from applying further smoothing to the data. RECORD 2*NSC+7: NUMBER OF POINTS (NPTS): [1X,I10]: The number of X and F(X) data points in the file. RECORD 2*NSC+8: X ARRAY: [6(1X,1PE13.6) repeating]: The X data array which is always written out explicitly. Thus an unevenly spaced or even non-monotonic sequence of X values may be supplied. As many lines are written as needed to specify NPTS points 6 per line. RECORD 2*NSC+9: F(X) ARRAY: [6(1X,1PE13.6) repeating]: The F(X) data array. As many lines are written as needed to specify NPTS points 6 per line.

All subsequent records in the file are comment records which are not accessed by UFILES read/write subroutines and are therefore in free format (cf. subsection 2.5 below). An applications program which is to access all the data and labels in a UFILES one-dimensional data file must provide array space for the data and labels. In other words declarations such as the following are needed.

PARAMETER NMAX= size of arrays to hold x,f(x) [declare NSCMAX,SCVAL, TDEV, SHDATE, and SCLAB as shown in subsection 2.2 for scalar data, tokamak id code and shot date] REAL X(NMAX) F(NMAX) (arrays to hold x, f(x)) CHARACTER*10 XLAB(3), FLAB(3) on the VAX ( label arrays for X and F data) (note character*30 XLAB, FLAB will also work)

The declarations or the equivalent must appear near the top of the applications program. The name of the UFILES subroutine to write a one-dimensional file is UF1DWR; to read a one-dimensional file UF1DRD. There is also a simplified read routine, UF1DRE, which returns the data, only without labels. If UF1DRE is used only NMAX, X and F (or equivalent) need be declared. UF1DRE is simpler to use e.g. in the input phase of an analysis code, if there is to be no automatic check of the physical units, etc. of the input data.

To write a one-dimensional UFILES file, first name and create the file (subsection 2.1)

CALL UF1DWR(LUN,TDEV,SHDATE,X,F,N,IPROC,XLAB,FLAB,NSC,SCVAL,SCLAB,IER) WHERE: all values must be supplied by the caller except IER. LUN,TDEV,SHDATE, NSC,SCVAL, and SCLAB are as in UF0DWR - see subsection 2.2 X: array containing independent variable values to write F: array containing dependent variable values to write. N: number of values in X, F to write IPROC: process code-- historical, use a value of zero. XLAB: 30 character string containing: characters 1-10: the name of "x" (e.g. "POSITION"); characters 11-20: blank; characters 21-30: physical units of "x" (e.g. "METERS "). FLAB: 30 character string containing: characters 1-20: the name of "f" (e.g. "ELECTRON DENSITY "); characters 21-30: physical units of "f" (e.g. "N/CM**3 "). IER: completion code. IER=0 denotes success.

To read a UFILES one-dimensional data file completely (i.e. with scalar numbers and all labels as well as the one-dimensional data itself), the file is first named and opened with calls to UFSETR and UFOPRD as described in subsection 2.1:

CALL UF1DRD(LUN,TDEV,SHDATE,X,F,NMAX,N,IPROC,XLAB,FLAB,NSCMAX,NSC,SCVAL, SCLKAB,IER) WHERE: only the values of LUN, NMAX and NSCMAX, are supplied by the caller.

If only the one-dimensional data function itself, and no labels or scalar values are desired, a simpler call is available. After calling UFSETR and UFOPRD,

CALL UF1DRE(LUN,X,F,NMAX,N,IER,LUNXT) WHERE: The values of LUN, NMAX, and LUNXT are supplied by the caller. LUNXT is the LUN of an open i/o channel on which the discarded labels and scalar values in the file are to be printed. If LUNXT=0 no attempt is made to print out the labels and scalar values.

If UF1DRE is used, the scalar data that might also have been in the UFILES one-dimensional data file is unavailable. It is possible to read only the scalar data from the one-dimensional data file with a call to UF0DRD, as described in subsection 2.2. However, if UF1DRE AND UF0DRD are both used to read the same file, the file must be closed and reopened (call UFCLOS then UFOPRD between the read calls.

1.6 2.4 Two-dimensional data file structure and access

UFILES supports a two-dimensional data file. This consists of an optional set of keyword scalar data in the format described in the sub-section 2.2, followed by a numeric representation of a function of two independent coordinates, f(x,y). Both the function values and the independent coordinates are written out explicitly, and labels are supplied for f,x, and y. UFILES therefore expects the x and y as well as f to be supplied as an array of numbers. UFILES does not require that the values of x or y be monotonically increasing as is conventional, but many UFILES- based utility programs do make the assumption.

figure 2.3 - 2-d file

84011TFTR 2 0 6 ;-SHOT #- F(X) DATA -UF1DWR- 10-Jul-98 1995 ;-SHOT DATE- UFILES ASCII FILE SYSTEM 0 ;-NUMBER OF ASSOCIATED SCALAR QUANTITIES- TIME (SECONDS) ;-INDEPENDENT VARIABLE LABEL: X- RADIAL POSITION CM ;-INDEPENDENT VARIABLE LABEL: Y- FM INV. DENSITY (CM**-3) ;-DEPENDENT VARIABLE LABEL- 2 ;-PROC CODE- 0:RAW 1:AVG 2:SM 3:AVG+SM 4 ;-# OF X PTS- 30 ;-# OF Y PTS- X,Y,F(X,Y) DATA FOLLOW: 3.004748E+00 3.024748E+00 3.044748E+00 3.064748E+00 1.650000E+02 1.716667E+02 1.783333E+02 1.850000E+02 1.916667E+02 1.983333E+02 2.050000E+02 2.116666E+02 2.183333E+02 2.250000E+02 2.316666E+02 2.383333E+02 2.450000E+02 2.516666E+02 2.583333E+02 2.650000E+02 2.716666E+02 2.783333E+02 2.849999E+02 2.916666E+02 2.983333E+02 3.049999E+02 3.116666E+02 3.183333E+02 3.249999E+02 3.316666E+02 3.383333E+02 3.449999E+02 3.516666E+02 3.583333E+02 0.000000E+00 0.000000E+00 0.000000E+00 0.000000E+00 0.000000E+00 0.000000E+00 0.000000E+00 0.000000E+00 1.853483E+13 1.857508E+13 1.865975E+13 1.859729E+13 2.006095E+13 1.998877E+13 1.997697E+13 1.992839E+13 2.113666E+13 2.100896E+13 2.097495E+13 2.092568E+13 2.181956E+13 2.168353E+13 2.167174E+13 2.158290E+13 2.268845E+13 2.256423E+13 2.257047E+13 2.245249E+13 2.391246E+13 2.385174E+13 2.393454E+13 2.387198E+13 2.475702E+13 2.471470E+13 2.483614E+13 2.480670E+13 2.489318E+13 2.475702E+13 2.461902E+13 2.441662E+13 2.564574E+13 2.542126E+13 2.506798E+13 2.470550E+13 2.702941E+13 2.674237E+13 2.640013E+13 2.602110E+13 3.475557E+13 3.479973E+13 3.468564E+13 3.451269E+13 4.516627E+13 4.577899E+13 4.653339E+13 4.712587E+13 6.152201E+13 6.309705E+13 6.472912E+13 6.613856E+13 7.654582E+13 7.906716E+13 8.178952E+13 8.414732E+13 9.015765E+13 9.353420E+13 9.700956E+13 9.995340E+13 9.815098E+13 1.021681E+14 1.063521E+14 1.097628E+14 1.026928E+14 1.071426E+14 1.118207E+14 1.154937E+14 1.031800E+14 1.078002E+14 1.124306E+14 1.159537E+14 9.507085E+13 9.917996E+13 1.028291E+14 1.055719E+14 7.374849E+13 7.679795E+13 7.982016E+13 8.211321E+13 4.796608E+13 4.950615E+13 5.074637E+13 5.177535E+13 2.680765E+13 2.701127E+13 2.676644E+13 2.673841E+13 2.117079E+13 2.101417E+13 2.119387E+13 2.145849E+13 2.301897E+13 2.286977E+13 2.284339E+13 2.290933E+13 2.154010E+13 2.150712E+13 2.154917E+13 2.160770E+13 1.703424E+13 1.711255E+13 1.696169E+13 1.683392E+13 1.376407E+13 1.393800E+13 1.423971E+13 1.440870E+13 3.787859E+12 3.916457E+12 3.962620E+12 4.005486E+12 ;----END-OF-DATA-----------------COMMENTS:----------- X AND/OR Y AXIS MODIFIED DATA FOR SHOT #####= 84011 FROM FILE: 2D ORIGINAL DATA : S######.FMA

A sample UFILES two-dimensional data file, X84011.FMA, is shown in figure 2-3. The start of the file contains information on data origin and keyword scalar data in the format described in the previous subsection. This is followed by labels for the independent coordinates x and y, and dependent coordinate f(x,y). then there is a "processing code" used by UFILES utilities to determine if given data has been already averaged and/or smoothed. Then the number of points along each dimension of the data is given, followed by the data itself. The following is a detailed record-by record description of the UFILES two-dimensional data file structure:

RECORDS 1, ... , 2*NSC+3: shot number, tokamak id, dimensionality (=2), shot date, number of scalars (NSC), and scalar data. Format descriptions are given in subsection 2.2 above- these records are identical to corresponding records in UFILES scalar data files. HOWEVER, in two- dimensional data files NSC=0 is allowed. RECORD 2*NSC+4: INDEPENDENT COORDINATE (X) LABEL: [1X,3A10]: 30 character label for the two-dimensional data x coordinate. The first 20 characters are reserved for the coordinate name; the last 10 characters for physical units. However, graphics subroutines utilized by UFILE-based utilities often only allow 10 characters for the coordinate name, therefore it is preferable to leave character positions 11-20 blank if possible. RECORD 2*NSC+5: INDEPENDENT COORDINATE (Y) LABEL: [1X,3A10]: 30 characters for the second independent coordinate of the two dimensional data function. Format the same as the label. RECORD 2*NSC+6: DEPENDENT COORDINATE (F) LABEL: [1X,3A10]: 30 character label for the two dimensional data function (f) coordinate. The first 20 characters are reserved for the name; the last 10 characters for physical units. RECORD 2*NSC+7: PROCESS CODE [I1]: Integer code used by UFILES-based utility programs . =0 to indicate unprocessed data, =1 to indicate averaged data (formed from more than one shot), =2 to indicate smoothed data, =3 to indicate averaged and smoothed data. Thus if the smoothing utility GSMOO2 is used on data with processing code set to 2, a warning message to the effect that the data has already been smoothed, will be generated. But the user will not be prevented from applying further smoothing to the data. RECORD 2*NSC+8: NUMBER OF X POINTS (NX): [1X,I10]: The number of X points i.e. the length of the second dimension of the data function. RECORD 2*NSC+10: X ARRAY: [6(1X,1PE13.6) repeating]: The data array which is always written out explicitly. Thus an unevenly spaced or even non-monotonic sequence of Y values may be supplied. As many lines are written as needed to specify NY points 6 per line. RECORD 2*NSC+11: YARRAY: [6((1X,1PE13.6) repeating]: The data array which is always written out explicitly. Thus an unevenly space or non-monotonic sequence of Y values may be supplied. As many lines are written as needed to specify NY points per line. RECORD 2*NSC+12: F(X,Y) ARRAY: [6(1X,1PE13.6) repeating]: The F(X,Y) data array. As many lines are written as needed to specify NX*NY points 6 per line. NOTE that the X-variation of the data function is stored contiguously, i.e. the order of the data is all of f vs. x at the first y, then all of f vs. x at the second y, etc. This is consistent with the FORTRAN convention for the storage of data in multiply-subscripted arrays.

All subsequent records in the file are comment records which are not accessed by UFILES read/write subroutines and are therefore in free format (cf. subsection 2.5).

An applications program which is to access all the data and labels in a UFILES two-dimensional data file must provide array space for the data and labels. In other words declarations such as the following are needed:

PARAMETER NXMAX= size of array to hold x PARAMETER NYMAX= size of array to hold y PARAMETER NFMAX= size of array to hold f (optional) [declare NSCMAX,SCVAL,TDEV,SHDATE, and SCLAB as shown in subsection 2.2 for scalar data, tokamak id code and shot date] REAL X(NXMAX), Y(NYMAX) arrays to hold x,y) REAL F(NXMAX,NYMAX) or REAL F(NFMAX) (array to hold f. The dual dimensionality of the data to be held in F need not be made explicit in the declaration. IF F is to be a "buffer' of size NFMAX instead, NFMAX should be big enough to handle the maximum expected size NX*NY of data to be held in F). CHARACTER*10 XLAB(3), YLAB(3), FLAB(3) on the VAX (label arrays for X, Y, and F) (NOTE CHARACTER*30 XLAB,FLAB will also work

These declarations or the equivalent must appear near the top of the application program.

The name of the UFILES subroutine to write a two-dimensional file is UF2DWR; to read a two-dimensional file, UF2DRD. There is also a simplified read routine, UF2DRE, which returns the data, only without the labels. If UF2DRE is simpler to use e.g. in the input phase of the analysis code, if there is to be no automatic check of the physical units, etc. of the input data.

CALL UF2DWR(LUN, TDEV,SHDATE,F,NF1,X,NX,Y,NY,IPROC,FLAB,XLAB,YLAB,NSC, SCVAL,SCLAB,IER) WHERE: all values must be supplied by the caller except IER. LUN, TDEV,SHDATE,NSC,SCVAL, and SCLAB are as in UF0DWR - see subsection 2.2 F: array containing dependent variable data. NF1: size of the first dimension of F, as it will be declared inside the subroutine UF2DWR. If the data in F is stored contiguously the NF1=NX should be set. REAL F(NF1,NY) and carries out the write with a statement of the form WRITE(LUN,format) ((F(I,J),I=1,NX),J=1,NY). X: array containing 1st independent variable data NX: number of X values to write Y: array containing 2nd independent variable data. NY: number of Y values to write IPROC: process code-- see description, UFILES 2-d file RECORD 2*NSC+7. FLAB: 30 character string containing: characters 1--20: the name of "f" (e.g. "ELECTRON DENSITY "); characters 21--30: physical units of "f" (e.g. "N/CM**3 "). YLAB: 30 character string containing the name of "y" characters 1-10) and the physical units of "y" (characters 21-30) For example: "TIME SECONDS ". IER: completion code. IER=0 denotes success.

To read a UFILES two-dimensional data file completely (i.e. with scalar numbers and all labels as well as the two-dimensional data itself), the file is first named and opened with calls to UFSETR and UFOPRD as described in subsection 2.1., then:

CALL UF2DRD(LUN, TDEV, SHDATE, F, NF1, X, NXMAX, Y, NYMAX, IPROC, NX, NY, IFLAG, FLAB, XLAB, YLAB, NSCMAX, NSC,SCVAL,SCLAB,IER) WHERE: only the values of LUN, NXMAX,NYMAX,NF1,NSCMAX, and IFLAG are supplied by the caller. UFODRD-- see subsection 2.2 F: array to contain the 2-d data function in the file NF1: (input) size (as determined by the caller) of the first dimension of the doubly subscripted array F. In UF2DRD F is declared REAL F(NF1,NY). The calling program does not necessarily have to declare F as a doubly subscripted array - it may instead be declared as a singly dimensioned array large enough to contain the data in the file. In either case read carefully the description of the flag IFLAG below. X: array to contain the 1st independent variable data. NXMAX: (input) size of the X array as determined by the calling program. An error will occur if the file to be read contains more than NXMAX X values. Y: array to contain the 2nd independent variable data. NYMAX: (input) size of the Y array as determined by the calling program. An error will occur if the file to be read contains more than NYMAX Y values. IPROC: process code- see description, UFILES l-d file RECORD 2*NSC+6. NX: (returned from file) the actual number of X array data points in the file. NY: (returned from the file) the actual number of Y array data points in the file. IFLAG: (input) if IFLAG=1 the data is read consecutively into the F array, so that the first NX values in the storage space F contains the variation of F vs.X at Y(1), the next NX values in F contain the variation at Y(2), etc. If IFLAG=1 is used, the caller should set NF1=NFMAX/NYMAX, where NFMAX is the total size of the F array. If IFLAG=1 an error condition occurs if the product NX*NY is greater than the product NF1*NYMAX, i.e. there are not enough words in F to contain the data in the file. if IFLAG not = 1, the data is read by the FORTRAN statement READ(LUN,format) ((F(IX,IY),IX=1,NX),IY=1,NY), with the result that if NF1 .gt. NX, the data is not read contiguously into the Array, i.e. the values F(NX+1,IY),...,f(NF1,IY)are not filled by the READ. If IFLAG is not = 1, an error state occurs if NX .gt. NF1. XLAB: 30 character string containing label for X. The last 10 characters contain the physical units. FLAB: 30 character string containing label for F. The last 10 characters contain the physical units. IER: completion code. IER=0 denotes success.

If only the two-dimensional data function itself, and no labels or scalar values are desired, a simpler call is available . After calling UFSETR and UFOPRD, use:

CALL UF2DRE(LUN,F,NF1,X,NXMAX,Y,NYMAX,NX,NY,IFLAG,IER,LUNXT)

WHERE: The values of LUN, NF1, NXMAX, NYMAX, IFLAG, and LUNXT are supplied by the caller.

All arguments except LUNXT are as described for UF2DRE (in fact UF2DRE simply calls UF2DRD to fill X, Y and F). LUNXT is the LUN of an open i/o channel on which the discarded labels and scalar values in the file are to be printed. If LUNXT=0, no attempt is made to print the labels and scalars.

If UF2DRE is used the scalar data that might also have been in the UFILES two-dimensional data file is unavailable. It is possible to read only the scalar data from the two-dimensional data file with a call to UF0DRD. If both UFODRD and UF2DRE are used to read the same file, the file must be closed and reopened (call UFCLOS then UFOPRD) between the read calls.

1.7 2.5 Three-dimensional data file structure and access

The name of the UFILES subroutine to write a three-dimensional file is UF3DWR; to read a three-dimensional file, UF3DRD. There is also a simplified read routine, UF3DRE, which returns the data, only without the labels.

To write a three-dimensional UFILES file, first name and create the file (cf. subsection 2.1 above), then:

SUBROUTINE UF3DWR(ILUN,IDEV,ZSDATE,F,NF1,NF2,X,NX,Y,NY,Z,NZ,IPROC, FLAB,XLAB,YLAB,ZLAB,NSC,SCVAL,SCLAB,IER) ILUN-- FILE LOGICAL UNIT NUMBER IDEV-- 4 LETTER TOKAMAK DEV. CODE ("PLT ", ETC.) ZSDATE-- SHOT DATE (10 CHARS, E.G. "10-AUG-81 ") X(NX),F(NF1,NF2,NZ),Y(NY),Z(NZ)-- REAL NUMERIC DATA TO BE WRITTEN ARRAYS AND DIMENSIONS IPROC- PROCESS CODE 0= "RAW", 1= AVERAGED 2= SMOOTHED, 3=AVERAGED AND SMOOTHED XLAB-- 30 CHARACTER LABEL FOR X FIRST 20 CHARACTERS-- NAME OF X LAST 10 CHARACTERS-- PHYSICAL UNITS OF X E.G. XLAB = 'MAJOR RADIUS CM ' FLAB-- 30 CHARACTER LABEL FOR F - SAME FORMAT AS XLAB YLAB-- 30 CHARACTER LABEL FOR Y - SAME FORMAT AS XLAB ZLAB-- 30 CHARACTER LABEL FOR Z - SAME FORMAT AS XLAB NSC-- NUMBER OF ASSOCIATED SCALARS TO WRITE ** NOTE NSC=0 IS ALLOWED ** SCVAL- VALUE(S) OF ASSOCIATED SCALARS SCLAB- ASSOCIATED SCALARS KEYWORDS, LABELS, PHYS. UNITS. IER=0 ON RETURN IF WRITE WAS SUCCESSFUL

To read a UFILES three-dimensional data file completely (i.e. with scalar numbers and all labels as well as the three-dimensional data itself), the file is first named and opened with calls to UFSETR and UFOPRD as described in subsection 2.1., then:

SUBROUTINE UF3DRD(ILUN,IDEV,ZSDATE,F,NF1,NF2,X,NX,Y,NY,Z,NZ,IPROC, NXACT,NYACT,NZACT,IFLAG,FLAB,XLAB,YLAB,ZLAB, NSCD,NSCA,SCVAL,SCLAB,IER) ILUN-- FILE LOGICAL UNIT NUMBER IDEV-- TOKAMAK DEVICE CODE (4 LETTERS E.G. "PLT ") ZSDATE-- SHOT DATE (10 CHARS, E.G. "10-AUG-81 ") X(NX),F(NF1,NF2,NZ),Y(NY),Z(NZ)-- REAL NUMERIC DATA TO BE READ ARRAYS AND DIMENSIONS IPROC-- PROCESS CODE 0= RAW, 1 = AVERAGED 2= SMOOTHED, 3= AVERAGED AND SMOOTHED NX,NF1,NF2,NY,NZ ARE INPUT NXACT,NYACT,NZACT ARE THE ACTUAL DIMENSIONS OF THE X AND Y AND Z ARRAYS IN THE FILE *** IFLAG: ---> IF IFLAG=1 THE DATA IS READ CONSECUTIVELY INTO THE F ARRAY; IF IFLAG .NE. 1, THE DATA IS READ BY READ (((F(IJ,IK,IL),IJ=1,NXACT),IK=1,NYACT),IL=1,NZACT) XLAB-- 30 CHARACTER LABEL FOR X FIRST 20 CHARACTERS-- NAME OF X LAST 10 CHARACTERS-- PHYSICAL UNITS OF X E.G. XLAB = 'MAJOR RADIUS CM ' FLAB-- 30 CHARACTER LABEL FOR F - SAME FORMAT AS XLAB YLAB-- 30 CHARACTER LABEL FOR Y - SAME FORMAT AS XLAB ZLAB-- 30 CHARACTER LABEL FOR Z - SAME FORMAT AS XLAB NSCD-- SPACE ALLOCATED FOR ASSOCIATED SCALARS NSCA-- RETURNED ACTUAL NUMBER OF SCALARS IN FILE ** NOTE ** NSCD MAY = 0 IN WHICH CASE THESE ARE DUMMIES: SCVAL-- LIST OF ASSOCIATED SCALAR VALUES READ SCLAB-- LIST OF ASSOCIATED SCALAR KEYWORD/LABELS READ FORMAT OF SCLAB: 30 CHARS, 10 CHAR KEYWORD BLOCK, 10 CHAR LABEL, 10 CHAR PHYS. UNITS-- SEE SUBROUTINE UF0WRT, FILE UFILS0.FOR IER=0 ON RETURN IF READ WAS SUCCESSFUL

If only the three-dimensional data function itself, and no labels or scalar values are desired, a simpler call available . After calling UFSETR and UFOPRD, use:

SUBROUTINE UF3DRE(ILUN,F,NF1,NF2,X,NX,Y,NY,Z,NZ, NXACT,NYACT,NZACT,IFLAG,IER,LUNO) ILUN (INPUT) - FILE LOGICAL UNIT NUMBER FILENAME WILL HAVE BEEN CONSTRUCTED (SRS. UFSETR,UFOPEN) AND FILE OPENED (SR. UFOPRD) BEFORE THIS ROUTINE IS CALLED X ARRAY FOR 1ST INDEPENDENT VARIABLE (RETURNED FROM FILE) Y ARRAY FOR 2ND INDEPENDENT VARIABLE (RETURNED FROM FILE) Z ARRAY FOR 3RD INDEPENDENT VARIABLE (RETURNED FROM FILE) F ARRAY FOR DEPENDENT VARIABLE (RETURNED FROM FILE) NF1,NF2,NX,NY,NZ (INPUT) SIZES OF X,Y,F ARRAYS - SEE DECLARATIONS NF1 FOR 1ST DIM. OF F ARRAY (DOUBLY SUBSCRIPTED) ERROR WILL OCCUR IF ARRAY SIZES ARE INSUFFICIENT TO ACCOMODATE DATA IN FILE BEING READ. NXACT,NYACT - THE ACTUAL SIZES OF THE X AND Y DATA ARRAYS IN THE FILE. THE F ARRAY WILL THEN CONTAIN NXACT*NYACT PTS. IFLAG (INPUT) - IF IFLAG=1 THE DATA IS READ CONSECUTIVELY INTO THE F ARRAY; IF IFLAG=0, THE DATA IS READ VIA "READ (..) ((F(IJ,IK),IJ=1,NXACT),IK=NYACT) WHICH MAY RESULT IN THE DATA NOT BEING CONTIGUOUSLY STORED. IER ERROR RETURN CODE 0= NORMAL SUCCESSFUL COMPLETION LUNO (INPUT) OPEN I/O CHANNEL FOR OUTPUT OF FILE DATA LABELS (OUTPUT SUPPRESSED IF LUNO.LE.0)

1.8 2.6 Comment Spooling

As will have been noticed by examining the sample UFILES file is figures 2-1, 2-2, and 2-3, all UFILES files are divided into two parts: a data section at the top and a comments section at the bottom, separated by the line: ;---END-OF-DATA---------------------COMMENTS:--------- This line is the last line written by any UFILES write subroutine (UF0DWR, UF1DWR, or UF2DWR). The line is also read by the read routines (UF0DRD etc.) so that on exit from these routines the file being read is left open and the next line is the first line of comments.

Comment lines are never accessed by UFILES subroutines (except for subroutine USPOOL which is described below), so the format of these lines is free (but still formatted ascii data with a limit of 160 characters per line if USPOOL is to be used). It would be possible to insert non-standard data in the comments lines of a UFILES file, but special purpose FORTRAN coding would have to be written to access this data.

Subroutine USPOOL can be used to copy the comments of a UFILES input file to a UFILES output file (or terminal):

CALL USPOOL(LUNI,LUNO)

where LUNI is the LUN of the input file and LUNO is LUN of the output file. USPOOL simply transfers the comments from the input file to the end of the output file, line by line, using A-format FORTRAN READs and WRITEs (binary READs and WRITEs in the case of binary compressed UFILES). The UFILES standard data portion of the files (the part above the line "--END-OF-DATA--") should already have been read in unit LUNI and written out on unit LUNO, when USPOOL is called. The following outline of a UFILES-based application program illustrates the use of USPOOL and its place relative to other UFILES subroutines:

PROGRAM PROG DATA LUNI,LUNO,LUNT /23,24,25/ C SET UP UFILES I/O CHANNELS. INPUT Annnnn.DAT; OUPUT Bnnnnn.DAT CALL UFSETR(LUNI,'A','DAT',' ',' ') CALL UFSETR(LUNO,'B','DAT',' ',' ') C ASK USER FOR SHOT NUMBER 10 CALL UREADI('PROG> ENTER SHOT NUMBER:$',ISHOT) C OPEN FILES, READ INPUT DATA CALL UFOPRD(LUNI,ISHOT,IER) IF(IER.NE.0) GO TO 10 C (VAX ONLY) REQUEST COMPRESSED BINARY OUTPUT FILE: CALL UFCMPR(LUNO,1) C OPEN OUTPUT FILE CALL UFOPWR(LUNO,ISHOT,IER) C ALSO CLOSE INPUT FILE ON ERROR IF(IER.NE.0) GO TO 90 C CALL UF1DRD(LUNI, ... , IER) ...[process the input data] C WRITE THE OUTPUT FILE CALL UF1DWR(LUNO, ... ,IER) C COMMENTS C WHEN WRITING COMMENTS TO THE END OF BINARY UFILES ON THE VAX, C USE A TEMPORARY *ASCII* FILE OPENED BY UFOPCF SUBROUTINE CALL UFOPCF (LUNT, IER) WRITE(LUNT,8001)[arguments] 8001 FORMAT ([describe processing done by PROG]/ >'COMMENTS SPOOLED FROM INPUT FILE:') C SPOOL CONTENTS OF TEMPORARY FILE CALL UCSEND(LUNT, LUNO) C COPY COMMENTS FROM INPUT FILE TO OUTPUT FILE CALL USPOOL(LUNI,LUNO) C ALL DONE CALL UFCLOS(LUNO) 90 CONTINUE CALL UFCLOS(LUNI) GO TO 10 END

Thus by using USPOOL any comments that were in the input file are copied into the output file. USPOOL reads from the input until end-of-file is encountered.

If a standard ascii UFILES file is written, comments can be appended directly to the end of the file with formatted WRITE statements. If other UFILES formats are written it is necessary to use a temporary intermediate file because formatted WRITEs cannot be made on a unit connected for binary output. The subroutine call UFOPCF(LUNT,IER) opens such a temporary file (an unused LUN must be supplied). After all formatted comments have been written into the temporary file on unit LUNT and spool the comments to the (binary or ascii) output file on unit LUNO. UFOPCF and UCSEND are available only in the VAX version of UFILES and are needed only for the binary UFILE option, although they will also work with ascii UFILES on the VAX. Their use is illustrated above in the sample routine. The comment spooling routine USPOOL is available on all machines.

By using USPOOL in all utility programs for processing "intermediate" UFILES data, it is possible to assure that each UFILES file contains a comment history of the data in the file from the time the data was first translated into UFILES format. Comment spooling is used in the generalized averaging, smoothing, and concatenation utilities. An example of the resulting comment history may be seen in the sample file in figure 2-2.

1.9 2.7 Error Messages

UFILE subroutines may produce a variety of error messages. These are written on a job's controlling terminal (interactive jobs) or in the job's log file (batch jobs).

Common error that are detected by UFILES subroutines ( and do not abort the controlling program) are:

(1) Failure to open a file. Usually the file does not exist or the name is incorrectly specified. The full filename used is always displayed on the terminal and should be checked carefully.

(2) (on a UFILES read) The controlling program did not provide sufficient array space to hold the data in a UFILES file. partial reads are generally not supported. The controlling routine must be recompiled with expanded arrays. The UFILES error message should be sufficient to identify the insufficiently dimensioned array and what dimensions are needed to hold the data in the specified UFILES file.

(3) Incorrect ordering of UFILES subroutine calls. The ordering of calls outline in subsection 2.0 above must be followed. Often an error of this sort will be caused by an error in the LUN passed to a UFILES subroutine. For example, an error corresponding to an attempt to open a file without a prior UFSETR call may only mean that the UFOPRD or UFOPWR call received an incorrect LUN (e.g. a zero due to a typo in the main program).

There are some common coding errors that cannot be trapped by UFILES and may abort the controlling program or cause odd behavior on some systems:

(1) Adjustable array dimension error. Usually a zero is passed as a subroutine argument which is supposed to specify the dimension of an array. On the VAX an error message to this effect will be produced and a traceback pointing to the line in the source code which generated the bad call. On other systems one usually gets an abort (but no traceback), or improper functioning of the program. On some systems the program may survive long enough to produce the UFILES error message that insufficient array space (zero) was provided e.g. to read in a data array.

(2) Argument type error or scalar passed as an array. The calling program has passed non-CHARACTER data variable as a subroutine argument where CHARACTER data is expected (or vice versa), or, a scalar variable address is passed as the start address of a large array. This sort of error is most pernicious because on most systems it is not detected but can cause "unrelated sectors of memory to be over-written, leading to a detectable run-time error later in program execution- and far removed from the actual cause of the error. On the VAX an "ACCESS VIOLATION" trap at runtime usually indicates an error of this sort. In the case of a CHARACTER/non-CHARACTER subroutine argument error, the traceback will point to the source line in the subroutine where the incorrectly typed variable is first referenced. In the case of the scalar/array subroutine argument error the "ACCESS VIOLATION" detection and traceback are less reliable. In either case the traceback may point back to UFILES source- contact D. Mc Cune (x2731) for help. On most non-VAX systems the subroutine argument mismatch errors are usually NOT DETECTED but the program runs incorrectly or generates runtime errors which are useless in diagnosing the root problem. On all systems GREAT CARE is recommended in setting up subroutine argument calling sequences because errors here are often the most difficult to diagnose and correct.

Most UFILES-generated error messages include the name of the subroutine where the error was detected, though this may not be the same subroutine that the user's application routine originally called but a lower level routine instead. Therefore the following lists of UFILES error messages is categorized by when it is likely to occur. Four categories are provided: (A) message may occur at any time; (B) message occurs when trying to identify and open a UFILES file; (C) message occurs when trying to read or write a UFILES file; (D) message occurs when closing a UFILES file.

(A) Messages which may occur as a result of any UFILES call:

?[name] - ATTEMPT TO USE UNASSIGNED LOGICAL UNIT NUMBER: [NUMBER] CALL UFSETR(LUN,...) FIRST

meaning: The value [number] of the LUN argument in the call to to UFILES subroutine [name] is incorrect, or, UFSETR was never called to associate that LUN with a filename form. The name [name] will be the name of the subroutine called by the user's application program.

corrective action: supply the correct LUN in the indicated subroutine call, or, insert a call to UFSETR.

(B)Messages which may occur during file identification or open.

? UFSETR- TABLE OVERFLOW OF L.U.N. [n1] UFILES L.U.N. ASSIGNMENT FAILED MAXIMUM NUMBER OF L.U.N. ASSIGNMENTS [n2]

meaning: too many LUNs have been assigned by distinct CALLs to UFSETR. (The maximum [n2] is 32 on all systems). [n1] gives the value of the LUN in the call to UFSETR where the error occurred.

corrective action: re-use an LUN to stay under the limit, or get the author to rebuild the UFILES library with an increased upper limit.

? UFSETR - ATTEMPT TO RESET OPEN L.U.N.: [n1] CALL UFCLOS TO CLOSE UNIT FIRST BEFORE REASSIGNMENT

meaning: UFSETR was called for LUN [n1] and a file was opened on that LUN, but then UFSETR was called again to re-assign the LUN before the file was closed (or, UFCLOS was not used to close the file so UFILES does not know that the file is closed).

corrective action: close the file with CALL UFCLOS(LUN) before re-assigning the LUN.

?UFSETR- IMPROPER LENGTH DIRECTORY NAME: [n1]

meaning: the directory/sub-directory name argument in a UFSETR call is of improper length [n1]. The argument must contain between 1 and 13 characters.

corrective action: use a different sub-directory name, or get the author to reconfigure UFILES on the VAX to accept longer directory/sub-directory names. If [n1] is zero, do not pass a null string UFSETR.

? UFSNAM: CANNOT NAME FILE: ILLEGAL SHOT NUMBER: [n1]

meaning: a shot number not between 1000 and 999999 was supplied in a call to UFOPRD or UFOPWR. Therefore the attempt to construct a filename failed; no attempt to open any file is made.

corrective action: fix the shot number or get the author to expand the range of shot numbers accepted by the UFILES system.

LOGICAL UNIT STATUS: OPEN FOR [access type] ? [subroutine name] - STATUS INCORRECT; L.U.N. = [n1]

meaning: a call to [subroutine name], either UFOPRD or UFOPWR, tried to open a file on a LUN on which a file is already open, with the specified [access type] is DATA WRITE, a file was opened on the LUN by UFOPWR; if [access type] is DATA READ, a file was already opened on the LUN by UFOPRD; if [access type] is READ/WRITE COMMENTS, the file has been opened and the UFILES standard data has been read or written, but the file has not yet been closed.

corrective action: re-arrange your calls so that you do not try to have two files open on the same LUN at the same time; or, insert UFCLOS calls where needed.

? UFOPRD: FAILED TO OPEN FILE: [name] [VAX only:] ERRSNS CODE [n1] - HIT CARRIAGE RETURN [DAS only:] ERRSNS CODE [n1] [n2] - HIT CARRIAGE RETURN [sel ONLY:] IOSTAT = [N1] - HIT CARRIAGE RETURN

meaning: Attempt to open a UFILES file for READ access failed. If the error is "file not found" on the VAX the message "--FILE NOT FOUND --" will also be displayed. On the DAS the "file not found" ERRSNS code is 29 250 for [n1][n2] respectively. On the DAS if an improperly formatted or non-octal ppn is in the filename [name] the additional message "% ERROR IN PPN" will be displayed.

corrective action: Carefully examine the filename [name] and ascertain that the specified device, ppn/directory, and file exist and are specified with correct syntax. If this does not reveal the source of the error consult your FORTRAN manual for the meaning of the ERRSNS or IOSTAT code displayed.

(C) Messages which may occur during file read or write.

LOGICAL UNIT STATUS: [status] ? [routine name] - STATUS INCORRECT; L.U.N. =[n1]

meaning: UFILES read/write subroutine [routine name] was called using LUN [n1] whose status [status] is invalid. A LUN which has been assigned to UFILES with a UFSETR call may have any one of five status conditions: (1)CLOSED (2)OPEN FOR DATA WRITE (3)OPEN FOR READ/WRITE COMMENTS (4)OPEN FOR DATA READ (5)OPEN: TEMPORARY COMMENTS FILE When a call to a UFILES WRITE routine (UF0DWR,UF1DWR, or UF2DWR) is made , only status (2) is correct. Status (1) would indicate that the file had never been opened; status (4) would indicate that the file has been opened for read access only; status (3) indicates that the data has already been written. Status (5) indicates a temporary comments file created by a call to UF0PCF (VAX only). When a call to a UFILES READ routine (UF0DRD, UF1DRD, UF1DRE, UF2DRD, or UF2DRE) is made, only status (4) is correct; Status (1) would indicate that the file was never opened; status (2) would indicate that the file was opened for write, not read access; and status (3) indicates that the file has already been read.

corrective action: (a) be sure that correct file LUN arguments are supplied to all UFILES subroutine calls; (b) maintain correct order of calls to UFIES subroutines. This may involve branching on error conditions -e.g. if a file open failed with an error message, proceeding to try to read the same file will cause additional error messages.

?UF0WRT - SCALAR QUANTITY KEYWORD LACKS TERMINATING ":" OR IS BLANK. KEYWORD CAUSING ERROR: [keyword value]

meaning: A UFILES write call has been made with incorrectly labeled scalar data. See subsection 2.2 for formatting rules or scalar data labels.

? UF0WRT - DUPLICATE SCALAR KEYWORD: [keyword value]

meaning: the same keyword is used twice in a scalar data keyword list. This is not allowed. The error follows a UFILES WRITE call.

? DIMENSION OF X ARRAY EXCEEDS FIRST DIMENSION OF F ARRAY --UF2DWR-- NX= [n1] NF1=[n2]

meaning: this non-fatal warning may occur after any call to a UFILES read subroutine. It indicates that the file being read ha been renamed by a non-UFILES program (e.g. a system COPY or RENAME utility).

UF0DRD WARNING: DIMENSIONALITY OF INPUT FILE = [n1]

meaning: warning that UF0DRD is being used to read only the scalars out of a file which contains [n1]-dimensional data as well.

? UF0DRD-- NO SCALARS IN FILE-- NO DATA

meaning: UF0DRD tried to read a file which contained no scalar data (i.e. NSC, the number of scalars, is zero).

% UF0RDR WARNING-- ARRAY SPACE FOR SCALAR NOT PROVIDED: VALUE=[data value] LABEL="[data label]", DATA LOST.

meaning: insufficient array space was provided to hold the scalar data in the file. This error can occur as the result of a read of a file of any dimensionality.

corrective action: Allocate more array space in the applications program; i.e. increase the array dimension NSCMAX as described in subsection 2.2.

?DIMENSIONALITY OF INPUT FILE = [n1] IS INCORRECT

meaning: An attempts was made to read a UFILES two-dimensional data file with a one dimensional read routine, or a one-dimensional file with a two-dimensional read routine

corrective action: Use the correct read call, or, identify the correct file to be read.

? UF1DRD-- DATA TO BE READ WILL NOT FIT IN PASSED ARRAYS: NACT= [n1] .GT. NDIM= [n2]

meaning: The applications program did not allocate sufficient array space to hold the data on the one-dimensional data file. NACT is the number of points in the data file; NDIM is the space allocated.

corrective action: increase relevant array dimensions in the applications program, e.g. NMAX as described in subsection 2.3 above.

? UF2DRD - [X or Y] ARRAY TO SMALL: SIZE=[n1] NEED=[n2]

meaning: One of the one-dimensional arrays for holding the independent variables of a two-dimensional data function is too small, by the amount indicated, to hold the data in the data file being read.

corrective action: increase the appropriate array dimension(e.g. NXMAX or NYMAX as described in subsection 2.3).

? UF2DRD - F ARRAY TOO SMALL TOTAL AVAILABLE WORDS = [n1] NEEDED = [n2]

meaning: Total size of F array is too small on UF2DRD call where IFLAG=1. See subsection 2.4

corrective action: Increase the size of the F array.

? UF2DRD - 1ST DIMENSION OF F ARRAY TOO SMALL SIZE = [n1] NEED = [n2]

meaning: F is too small in the first dimension. Can occur on a UF2DRD call with IFLAG=0. See subsection 2.4

corrective action: increase the first dimension of F.

(D) Error that may occur when closing a file

UFCLOS: ATTEMPT TO CLOSE IMPROPERLY OPENED FILE: UNIT=[n1]

meaning:UFCLOS was called to close a file on LUN [n1] but no file was open on that LUN (or the file was not opened with a UFILES UFOPRD or UFOPWR call).

UFCLOS: CLOSING FILE BEFORE DATA WRITE: UNIT[n1]

meaning: a file was created by an UFOPWR call on unit [n1] but no UFILES data was written into that file (no UFILES WRITE routine was called).

corrective action: use the UFILES subroutines in their correct order.

Redirecting Error Message Output

In order to redirect error messages to a different file, create a new file e.g. "UFILES.LOG" on unit LUNERR. Then, CALL UFEMSG(-LUNERR) (note "-" sign) will cause all UFILES messages to go to the file instead of SYS$OUTPUT. Exception: many system errors at file OPEN time must go to SYS$OUTPUT. The routing of these messages is not under UFILES control.

If you have a JUNK area (read $ HELP TRANSP JUNK) you might choose to put your UFILES.LOG file there. The full filename would then be "UFILES_JUNK:UFILES.LOG". This file will then be available for examination upon occurrence of "unanticipated" error situations, without having UFILES.LOG files scattered about your working disk directories.

SECONDARY ALTERNATIVE: CALL UFIMSG(0) CALL UFEMSG(0) will suppress SOME BUT NOT ALL messages from the UFILES system. The user will NOT be given a chance to acknowledge errors.

These alternatives should be implemented in the initialization phase of your applications program.

1.10 3.0 Enhancements

1.11 3.1 Format Variants

16-bit binary -- maximum compression, but the resulting file is not portable to remote machines, and, data loss can occur. compression ratios of 10:1 (relative to ascii) are typical. 32-bit portable binary -- loss free compression, resulting file is portable between systems. compression ratios of 5:1 (relative to ascii) are typical.

Compression offers two advantages: (1) major savings of disk space, and (2) faster i/o performance, since the time to read a larger file is typically greater than the cpu time to decompress the data.

16-bit compression is appropriate for linearly digitized data intended for local use. If the data originates from a 12-bit diagnostic linear digitizer, compression/decompression will result in no data loss.

However, if the data is the result of model computation, or if there is the possibility of wanting to ship the data off site, the 32-bit portable binary compression method is much to be preferred.

Read access to compressed binary Ufile data is automatic.

However, to write compressed UFILES two changes in your source code are likely to be needed: for setup and for comments.

1.12 3.1.1 Setup

*After* the UFSETR call, *prior* to the UFOPWR call, add

CALL UFCMPR(ILUN,2) ! 32-bit loss-free portable binary

or

CALL UFCMPR(ILUN,1) ! 16-bit binary - caution! - data loss

or

CALL UFCMPR(ILUN,3) ! HDF UFILE

or

CALL UFCMPR(ILUN,4) ! NETCDF UFILE

to set the compressed data output flag for unit ILUN.

CALL UFCMPR(ILUN,0)

clears the flag, allowing an old style ascii UFILES data write. Note that UFCMPR must be called prior to EACH UFOPWR call. UFCMPR does not establish a permanent default.

1.13 3.1.2 Comments

CALL UFOPCF(ILUNC,IER)

to open a scratch formatted comments file on unit ILUNC, using formatted FORTRAN writes. Write the comments on ILUNC. Then,

CALL UCSEND(ILUNC,ILUN)

to send the comments in binary form to the binary compressed output file on unit ILUN. UCSEND rewinds ILUNC and uses USPOOL to copy the comments to the binary output file. USPOOL has been enhanced to support comment output to binary files.

Compressed data write calls generate statistics describing the (generally negligible) error associated with compression. These statistics are appended as comments in the compressed binary

1.14 3.1.3 UFHDCK - read UFILES header

UFHDCK can only be called AFTER calling UFOPRD to open the file, but BEFORE calling one of the read routines e.g. UF1DRD or UF2DRD.

The syntax of the call is:

C declaration: INTEGER INFO(8) .... C read Ufile header: CALL UFHDCK(ILUN,INFO,IER) This calls the routine UFHDCK: C-------------------------------------------------------------- C UFHDCK C READ HEADER OF UFILE AND RETURN STATISTICAL INFORMATION C SUBROUTINE UFHDCK(ILUN,INFO,IER) C C INPUT: C ILUN -- L.U.N. OF A JUST - OPENED UFILE C C OUTPUT: C INFO -- INFORMATION BLOCK C INFO(1)=FILE DIMENSIONALITY C INFO(2)=TOTAL # OF WORDS IN DEPENDENT DATA ARRAY C INFO(3)=SIZE OF FIRST DIMENSION OF DATA ARRAY C INFO(4)=SIZE OF SECOND DIMENSION OF DATA ARRAY (IF APPLICABLE) C ... C INFO(8)=NUMBER OF SCALARS IN DATA FILE C C IER = 0 IF ROUTINE COMPLETES EXECUTION SUCCESSFULLY

1.15 4.0 UFILES Utilities

Graphical Display Routines

There are four routines for the graphical display of data.

ULOOK0 - examine scalar data UGRAF1 - plot f(x) data UGRAF2 - plot f(x,y) data UGRAF3 - plot slices of f(x,y,z) data

All use PPPL's SG graphics library, which is an enhancement of the Tektronix Advanced Graphics library. See http://w3.pppl.gov/~pshare/help/sg_hlp.html.

Ufiles Generalized Data Averaging Programs

There are three utility programs provided for averaging UFILES data.

GAVER0 - used to average scalar data files GAVER1 - used to average one dimensional data files GAVER2 - used to average two dimensional data files

Ufiles Generalized Data Smoothing Programs

These utilities allow "smoothing", "de-glitching", units transformations, axes swaps, and other manipulations of Ufiles data.

GSMOO1 - smoothes one dimensional data GSMOO2 - smoothes two dimensional data

UFILES 1d to 2d Generalized Concatenating Program

Concatenation consists of joining a sequence of related one dimensional data functions f(x) with scalars identifying y-values, to form a two dimensional data function f(x,y). The utility program for this operation is called CONCAT.

UFILES 2d to 1d OR 2D Generalized Extractor

EXTRAC is used to extract slices or sub-domains of 2d UFILES input data. Slice extraction consists of taking out of the 2d data f(x,y) the variation of f vs. x at fixed y, or, the variation of f vs. y at fixed x. Also, 2d subsets of f(x,y) can be extracted, with the x and/or y axes reduced.

UFILES format analyzer / converter program

UFTYPE reads named Ufiles and prints out summaries of their format (i.e. ascii, binary, NetCDF or HDF), dimensionality and shape. UFTYPE can also be used to convert Ufiles from one format to another. The UFTYPE .CONCAT option can be used to pack a sequence of Ufiles into a concatenated ascii file, e.g. for shipment to another computer; the UFTYPE .UNPACK option unpacks a concatenated ascii file back into its original Ufile constituents (ascii format). Scripts "uf_pack" and "uf_unpack", included in UFILES software distributions, provide especially convenient access to this functionality.

All UFILES utilities use a terminal I/O and scripting package called UREAD. This is not part of UFILES itself. UREAD supports powerful scripting and automation capabilities. UREAD is described elsewhere: see http://w3.pppl.gov/~pshare/help/uread_hlp.html

Table of Contents

1.0 0 Introduction
1.1 1 Overview
1.2 2.0 Description of the UFILES System
1.3 2.1 File Identification, OPEN and CLOSE
1.4 2.2 Scalar data file structure and access
1.5 2.3 One-dimensional data file structure and access
1.6 2.4 Two-dimensional data file structure and access
1.7 2.5 Three-dimensional data file structure and access
1.8 2.6 Comment Spooling
1.9 2.7 Error Messages
1.10 3.0 Enhancements
1.11 3.1 Format Variants
1.12 3.1.1 Setup
1.13 3.1.2 Comments
1.14 3.1.3 UFHDCK - read UFILES header
1.15 4.0 UFILES Utilities

About this document

• 1. Section One
• 1.1 Subsection One-One
• ...
• 1.2 Subsection One-Two
• 1.2.1 Subsubsection One-Two-One
• 1.2.2 Subsubsection One-Two-Two
• 1.2.3 Subsubsection One-Two-Three     <== Current Position
• 1.2.4 Subsubsection One-Two-Four
• 1.3 Subsection One-Three
• ...
• 1.4 Subsection One-Four