Importing and exporting data

NCL supports import and export for several data types including netCDF and HDF. Here is a list of supported operations:

Support for importing GRIB files has been add in Version 4.0.1 of the package.

ASCII files

Import

Support for importing ASCII files is provided by the NCL asciiread function. This function returns all data that is read in to an NCL variable. For example, suppose a local file named "float_data" looked like:

-1. 0. 1. 2.
then executing the NCL commands:

input_data = asciiread("float_data",4,"float")
print(input_data)

yields:

Variable: input_data
Type: float
Total Size: 16 bytes
            4 values
Number of Dimensions: 1
Dimensions and sizes:   [4]
Coordinates: 
Number Of Attributes: 1
_FillValue 
(0)     -1
(1)     0
(2)     1
(3)     2

For numeric data, the format of the data file is free-form and all non-numeric characters are ignored. So, for example, if the file "int_data" looked like:

 1 <--First data item, 2 <--Second data item
 3, 4,
 5XXX6YYY
then executing the NCL commands:
input_data = asciiread("int_data",(/2,3/),"integer")
print(input_data)

yields:

Variable: input_data
Type: integer
Total Size: 24 bytes
            6 values
Number of Dimensions: 2
Dimensions and sizes:   [2] x [3]
Coordinates: 
Number Of Attributes: 1
_FillValue 
(0,0)   1
(0,1)   2
(0,2)   3
(1,0)   4
(1,1)   5
(1,2)   6

If there are more data values in the file than are requested, then only the requested number of values are used. If there are fewer data values in the file than are requested, then the missing values are assigned the default missing value for the given data type. If the data type in an asciiread is "string", then each line of the input file is returned as a single data value.

Export

Support for exporting ASCII files is provided by the NCL asciiwrite procedure. For example, executing the following NCL commands:
a = (/1,2,3/)
asciiwrite("int_file",a)
would create a file named "int_file" that looks like:
1
2
3

Binary files

Import

Binary files created in C blocked binary I/O format (by a call to the C procedure fwrite for example) can be imported directly into NCL by using the NCL cbinread function. For example, if "c_bin_data" is a binary file as described that contains three floating point numbers, then the NCL command

fval = cbinread("c_bin_data",3,"float")

would store the three values in "c_bin_data" in the NCL array fval. If there are more values in the data file than specified in the second argument to cbinread, then the requested number of data values will be returned; if there are not as many values in the data file as requested, then the missing values are assigned the default missing value for the given data type.

Note: The data file read in by cbinread must have been created on a machine of the same architecture as the one that NCL is being run on. For example, if you created a binary file on a machine with a 64-bit word size and tried to read it back on a 32-bit machine, you will not get what you want.

Binary files created using Fortran unformatted writes can be imported directly into NCL by using the NCL fbinread function. For example, if "f_bin_data" is a binary file as described that contains three floating point numbers, then the NCL command


fval = fbinread("f_bin_data",3,"float")

would store the three values in "f_bin_data" in the NCL array fval. If there are more values in the data file than specified in the second argument to fbinread, then the requested number of data values will be returned; if there are not as many values in the data file as requested, then an error will result.

Note: The data file read in by fbinread must have been created on a machine of the same architecture as the one that NCL is being run on.

Export

The NCL procedure cbinwrite can be used to write a file in C blocked I/O format from any NCL numeric variable. For example:

fval = (/ 1., 2., 3. /)
cbinwrite("c_bin_data",fval)

would write the binary file "c_bin_data". NCL automatically figures out how many values to write and the data type being written.

Writing binary files using a Fortran unformatted write with fbinwrite is entirely analogous to using cbinwrite.

netCDF files

netCDF (network Common Data Form) is an interface for scientific data access and a library that provides an implementation of the interface. The netCDF library also defines a machine-independent format for representing scientific data. Together, the interface, library, and format support the creation, access, and sharing of scientific data. The netCDF software was developed at the Unidata Program Center in Boulder, Colorado.

Import

netCDF files are imported into NCL via the addfile function. NCL has several operators that apply to netCDF files; these operators are the metadata operators discussed in the operator summary table of the Expressions and operators module. Any netCDF file must have a suffix of ".nc" or ".cdf". NCL uses these suffixes to determine that the file is a netCDF file.

As an example, suppose the file "simple.cdl" (cdl files are ASCII versions of netCDF files that can be converted to netCDF files by using the netCDF library function ncgen) contains the following simple file:


netcdf simple {  // A very simple file for demo purposes.

dimensions:
    xdim = 5;

variables:

    float       xarray(xdim);
                xarray:long_name = "A small array";
                xarray:units = "dimensionless";
                xarray:valid_range = 0.f, 7.f ;
                xarray:_FillValue = -9999.0f;

    float       xdim(xdim);
                xdim:long_name = "X array dimension";
                xdim:units = "dimensionless";

    :history = "created by SCD for demo purposes";
    :title = "Simple demo";

data:

 xdim = -20, -10,   0,  10,  20;

 xarray = -1., 0., 1., 2., 3.;
}

Let simple.cdf be the netCDF file created from the simple.cdl file by using ncgen. The following NCL interactive session illustrates importing simple.cdf into NCL and using some of the metadata operators:

ncl 0> file1 = addfile("simple.cdf","r")
ncl 1> print (file1)

filename:       simple
path:   simple.cdf
        dimensions:
                xdim = 5
        variables:
                float xarray(xdim)
                float xdim(xdim)
ncl 2> print(file1->xarray)


Variable: xarray (file variable)
Type: float
Total Size: 20 bytes
            5 values
Number of Dimensions: 1
Dimensions and sizes:   [xdim | 5]
Coordinates: 
            xdim: [xx..xx]
Number Of Attributes: 4
long_name 
units 
valid_range 
_FillValue 
(0)     -1
(1)     0
(2)     1
(3)     2
(4)     3
ncl 3> print(file1!0)
(0)     xdim
ncl 4> print(file1@history)
(0)     created by SCD for demo purposes
ncl 5> print(file1->xarray({-10:10}))


Variable: xarray (file variable subsection)
Type: float
Total Size: 12 bytes
            3 values
Number of Dimensions: 1
Dimensions and sizes:   [xdim | 3]
Coordinates: 
            xdim: [xx..xx]
Number Of Attributes: 4
long_name 
units 
valid_range 
_FillValue 
(0)     0
(1)     1
(2)     2

Export

netCDF files can be created directly from NCL by specifying the "c" option in an addfile command. Consider the following NCL session:

ncl 0> file1 = addfile("file1.cdf","c")
ncl 1> file1->xarray = (/2.,3.,4.,5.,6./)
ncl 2> file1!0 = "xdim"
ncl 3> file1&xdim = (/-20,-10,0,10,20/)
ncl 4> file1->xarray@units = "millimeters"
ncl 5> file1@title = "NCL-created netCDF"
ncl 6> quit

This creates a netCDF file "file1.cdf". A dump of this netCDF file obtained by using the "ngdump" utility in the netCDF library yields:

netcdf file1 {
dimensions:
        xdim = 5 ;

variables:
        float xarray(xdim) ;
                xarray:units = "millimeters" ;
        long xdim(xdim) ;

// global attributes:
                :title = "NCL-created netCDF" ;

data:

 xarray = 2 , 3 , 4 , 5 , 6  ;

 xdim = -20, -10, 0, 10, 20 ;
}

Existing netCDF files can be modified by importing them using "addfile" with a final argument of "w" and making the desired changes. The imported netCDF file will be overwritten when NCL is terminated.

HDF files

HDF (Hierarchical Data Format) is a multi-object file format, developed at NCSA, that facilitates the transfer of various types of data between machines and operating systems.

Import

Importing HDF files is entirely analogous to importing netCDF files as described above, with some minor exceptions. First, if variable names contain spaces or non-alphanumeric characters these characters are replaced with the '_' character when listed from NCL and are referenced from NCL in this fashion.

Export

Although HDF and netCDF share the same NCL interface and can be accessed through the same types of NCL statements, exporting HDF files from NCL is somewhat limited currently. Only the values of variables and their names are written to files in the HDF format. Attributes and dimension names can not currently be written from NCL. Attributes and dimension names will generate warning messages when attempting to write an NCL variable to an HDF file.

Also there is currently no way to access 8 bit and 24 bit HDF images from NCL.

GRIB import

GRIB was added with release 4.0.1. See Information on Supported Data Formats section of the reference guide for more information.

User Guide Control Panel

NG4.1 Home, Index, Examples, Glossary, Feedback, UG Contents, UG WhereAmI?


$Revision: 1.8 $ $Date: 1998/06/15 22:08:49 $