The Parallel Virtual File System (PVFS) is a high-performance and scalable parallel file system for Linux clusters. File data stored on PVFS can be created/accessed via three APIs: PVFS native API, UNIX I/O API, and ROMIO MPI-IO. Additionally, standard UNIX commands (such as cp, ls, rm, df, etc), as well as similar commands and several useful utilities provided with the PVFS distribution, may be used with the PVFS file system.
Currently at PPPL, PVFS is running on the Pared cluster utilizing 8 I/O servers creating a file system capacity of about 300 Gigabytes which can be accessed by any of 14 clients. From these clients, data can be stored in parallel on PVFS in the HDF5 format, using a collection of functions written to allow easy migration from standard print commands. The HDF5 standard provides an efficient format for the storage of large data, and supports parallel input and output natively. There already exist functionality within AVS/Express 6.1, and IDL for the reading HDF5 data, and an OpenDX module is being developed for the same function. Advanced features of HDF5 include the ability to read in "hyperslabs" of data, and to selectively import downsampled data.
Mike Miller and Vaughn Wernersbach, are students working in the PPPL high resolution display wall / visualization lab.