MPI files are not like ordinary files and so MPI provides special functions for opening them, closing them and doing other things with them.
The first thing you always want to do with a file is to open it. The call to do that in C is:
and in Fortranint MPI_File_open (MPI_Comm comm, char *filename, int amode, MPI_Info info, MPI_File *fh);
This call must be issued by all processes participating in the communicator. It is a blocking call and a barrier call. This call sets a default view of the file, about which more later.MPI_FILE_OPEN(INTEGER COMM, CHARACTER FILENAME(*), INTEGER AMODE, & INTEGER INFO, INTEGER FH, INTEGER IERROR)
There are 3 arguments in this call the likes of which we haven't encountered
The first one is
amode, which stands for the access mode. All processes
opening the file must open it in the same access mode.
The access modes can be as follows
and in Fortran you can use:MPI_MODE_WRONLY | MPI_MODE_CREATE | MPI_MODE_EXCL
The second argument that is specific toIOR(MPI_MODE_WRONLY, MPI_MODE_CREATE)
info. If you use GPFS to write on then this parameter should be left empty. Simply specify:
If you use HPSS then there is quite a lot that you can put ininfo = MPI_INFO_NULL
infoobject can be loaded with information about how the file should be striped, what permissions it should be created with, how many parallel processes will typically access it, and so on. You can also load it with HPSS specific information of which the most important is HPSS class of service, HPSS type of storage class, and hints about the anticipated size of the file.
The info object is created by calling function
in C and there is no support for Fortran interface in the HPSS version of MPI/IO. Once you have created your info object then you can begin loading it by calling functionint MPI_Info_create(MPI_Info *info);
MPI_Info_set. For example:
Once you no longer need an info object you can free it withMPI_Info_set(info, "hpss_cos", "3"); MPI_Info_set(info, "hpss_sclasstype", "DISK"); MPI_Info_set(info, "access_style", "write_once");
There are many other things you can do with an info object. You can enquire about the number of key-value pairs. You can enquire about the keys and then ask about the values. You can duplicate an info object and you can modify any value in a selected key-value pair. There are functions for all that. But, since none of that stuff is currently supported on GPFS, we won't dwell on it any more.MPI_Info_free(info);
MPI_File_open is a blocking call. The function does not return
until the file has been opened and the file handle returned. This may
take a very long time, especially on a parallel file system, and
an even longer time with HPSS, where numerous transactions have to
be exchanged with Encina in order to open a new file. It would be
good if you could send a file open request to the system
and then go away and do other things, occasionally checking if the
file is ready.
Once you have opened a file you will probably write on it or read from it, but this is a complex affair in MPI, so we'll postpone the discussion of this until later.
To close an MPI file simply say:
in C and in Fortranint MPI_File_Close (MPI_File *fh);
You must ensure that all requests associated withMPI_FILE_CLOSE (FH, IERROR) INTEGER FH, IERROR
fhhave completed before you call
MPI_File_Close. This is a collective barrier operation.
To delete an MPI file call function
in C andint MPI_File_delete (char *filename, MPI_Info info);
As withMPI_FILE_DELETE(FILENAME, INFO, IERROR) CHARACTER(*) FILENAME INTEGER INFO, IERROR
openyou can pass various hints to the file system you work with by loading them into
info. GPFS, as I said before, doesn't support this feature at this stage, so here you have to use
MPI_INFO_NULL. And HPSS doesn't have any special hints for deleting either, so here again, you just use
MPI files can be statically sized. This is important because if multiple processes write on the same file in parallel, you must have a static file map, so that the processes don't write on each other's blocks. You set a size of an MPI file with
in C andint MPI_File_set_size (MPI_File fh, MPI_Offset size);
in Fortran. Observe thatMPI_FILE_SET_SIZE (FH, SIZE, IERROR) INTEGER FH INTEGER(KIND=MPI_OFFSET_KIND) SIZE, INTEGER IERROR
sizeis of type
MPI_Offset. This is going to be a 64 bit integer on GPFS and HPSS (
long longin C and
INTEGER(KIND=8)in Fortran). There must be no pending I/O operations when you call this function. You can use this function both to increase and to decrease the size of the file.
If you have a file which is already opened and sized and don't remember what the size was use
in C andint MPI_File_get_size (MPI_File fh, MPI_Offset size);
in Fortran.MPI_FILE_GET_SIZE (FH, SIZE, IERROR) INTEGER FH INTEGER(KIND=MPI_OFFSET_KIND) SIZE, INTEGER IERROR
You can also alter an
info record on a file by calling
MPI_File_set_info and you can read the info record associated
with an open file by calling