SPMD Programmer’s Guide

Version 1.0

 

Introduction.

SPMD consists of several commands that allow the user to use the same parallel program initiation interface for PVM, LAM MPI, or MPICH programs. This interface was developed for the HIVE (http://newton.gsfc.nasa.gov/thehive). The HIVE consists, a 64 dual processor cluster of PCs and 2 host nodes. All users using SPMD start a parallel program from a host and SPMD distributes them across the nodes of the cluster. This document gives the step necessary to install SPMD.

Simple Changes.

Once a parallel program has been written for PVM or MPI the following statements must be added to a program if it is to be run under SPMD.

In the case of PVM the following statement must be executed before any PVM commands:
 

Start_SPMD ( &argc, &argv );

where argc and argv are the input arguments of main:

 

main ( int argc, char **argv )

And before the program exits it must execute the statement:
 

End_SPMD ();

which replaces the normal pvm exit procedure:
 

pvm_exit ();

In the case of MPI, "Start_SPMD ( &argc, &argv );" replaces, "MPI_Init ( &argc, &argv );" and, "End_SPMD ();" replaces, "MPI_Finalize();".

Other than the above stated changes, SPMD changes the way an MPI or PVM program works in that at least one more process is running than without SPMD. That is the global program execution group contains one more process. This process is the SPMD server process which started all program group processes. It provide other useful functionality, yet are not require by PVM or MPI.

In the case of MPI, the SPMD server is process 0 in the process group corresponding to the communicator MPI_COMM_WORLD. In both MPI and PVM, SPMD defines three groups that each process belongs to the SYSTEM, USER, and WORK groups. Of these three groups, the SPMD server belongs only to the SYSTEM group and has the group ID of 0 in this group. All other processes belong to the USER group as well as the SYSTEM group. The WORK group of a process only contains processes executing the same program.

Predefined Groups and Communicators.

SPMD provides predefined groups for both MPI and PVM and provide predefined communicators for MPI. Each user process belongs to the SYSTEM group, the USER group, and the WORK group. The text names of the groups that a process belongs to are defined respectively by the variables:
 

char * SYSTEM_name ;
 

char * USER_name ;
 

char * WORK_name ;

The groups in PVM are defined:
 

char * SPMD_Group_SYSTEM;
 

char * SPMD_Group_USER;
 

char * SPMD_Group_WORK;

and in MPI by:
 

MPI_Group SPMD_Group_SYSTEM;
 

MPI_Group SPMD_Group_USER;
 

MPI_Group SPMD_Group_WORK;

For the purpose of PVM and MPI barrier synchronization the groups are defined in PVM by:
 

char * SPMD_SYSTEM;
 

char * SPMD_USER;
 

char * SPMD_WORK;

and in MPI by:
 

MPI_Comm SPMD_SYSTEM;
 

MPI_Comm SPMD_USER;

MPI_Comm SPMD_WORK;

MPI communicators are define for each group by:

MPI_Comm SPMD_Comm_SYSTEM;

MPI_Comm SPMD_Comm_USER;

MPI_Comm SPMD_Comm_WORK;

These predefined values may be used any where in the program where groups or communicators are normally expected. Variables are also provided that the contain the group identifier for each group a process is contained:
 

int SYSTEM_gid, USER_gid, WORK_gid ;

The variable nproc and iproc are also provided. iproc is equal to WORK_gid and nproc is the number of processes in SPMD_Group_WORK.

Predefined Functions.

SPMD provides additional functions such as, monitoring total execution time, total floating point operation count and graphical display of data on a X Windows terminal. The understanding of these will be left to the ambitious programmer who wishes to search the examples and source code. :-)