Академический Документы
Профессиональный Документы
Культура Документы
Introduction
In previous sections, you learned how to send and receive messages in which all the data was of a single type intrinsic to MPI and stored contiguously in memory. While your data may occasionally be that well behaved, it is likely that you will need to transmit collections of data of mixed types defined by the program, or data that are scattered in memory. In this chapter, you will learn strategies and features in MPI that allow you to address these circumstances.
Multiple Messages
M
NN N
Submatrix to be sent
Disadvantage:
Overhead. A fixed overhead is associated with the sending and receiving of a message, whatever long or short it is. If you replace one long message with multiple short messages, you slow down your program by greatly increasing your overhead.
If you are working in a portion of your program that will be executed infrequently and the number of additional messages is small, the total amount of added overhead may be small enough to ignore. However, for most programs there won't be any advantage in limiting yourself to this strategy. You will have to learn other techniques for the heavily executed portions.
Notes:
This approach eliminates the excessive messages of the previous approach, at the cost of extra memory for the buffer and extra CPU time to perform the copy into the buffer. The obvious limitation of this approach is that it still handles only one type of data at a time.
The MPI_PACK routine allows you to fill a buffer "the right way". You call MPI_PACK with arguments that describe the buffer you are filling and with most of the arguments you would have provided to MPI_SEND in our simplest approach. MPI_PACK copies your data into the buffer and, if necessary, translates it into a standard intermediate representation. After all the data you want to send have been placed in the buffer by MPI_PACK, you can send the buffer (giving its type as MPI_PACKED) and no further translations will be performed.
COUNT is initially set to zero to indicate you are starting the construction of a new message and have an empty buffer. The successive calls to MPI_PACK update COUNT to reflect the data that have been added to the buffer. The final value of COUNT is then used in the call to MPI_SEND as the amount of data to send.
Packing
Notes:
As a direct replacement for pack and unpack operations, MPI derived type operations are usually more efficient but somewhat more verbose, because of the need to explicitly create, commit, and free MPI type indicators. If one is packing data that doesn't remain in existence until the time the packed buffer is sent (e.g., if successive values to be packed are computed into the same variables), derived type operations lose their efficiency advantage because you must institute some other form of buffering to retain those values until they can be sent.
/* the 3 arrays used to describe an MPI derived type */ /* their size reflects the number of components in SparseElt */ int lena[2]; MPI_Aint loca[2]; MPI_Datatype typa[2]; MPI_Aint baseaddress;
/* a variable to hold the MPI type indicator for SparseElt */ MPI_Datatype MPI_SparseElt;
Before a derived datatype can be used in a communication, the program must create it. This is done in two stages.
Construct the datatype (Step 1). New datatype definitions are built up from existing datatypes (either derived or basic) using a call, or a recursive series of calls, to the following routines:
MPI_TYPE_CONTIGUOUS, MPI_TYPE_VECTOR, MPI_TYPE_HVECTOR, MPI_TYPE_INDEXED MPI_TYPE_HINDEXED, MPI_TYPE_STRUCT.
Commit the datatype (Step 2). The new datatype is "committed" with a call to MPI_TYPE_COMMIT. It can then be used in any number of communications. The form of MPI_TYPE_COMMIT is: MPI_TYPE_COMMIT (datatype)
Finally (Step 3), there is a complementary routine to MPI_TYPE_COMMIT, namely MPI_TYPE_FREE, which marks a datatype for de-allocation.
MPI_TYPE_FREE (datatype)
Any datatypes derived from datatype are unaffected when it is freed, as are any communications which are using the datatype at the time of freeing. datatype is returned as MPI_DATATYPE_NULL.
Return a new datatype that represents equally spaced blocks. The spacing between the start of each block is given in:
unit of extent from oldtype(count)
int MPI_Type_vector(count,blocklength,stride,oldtype,newtype);
bytes
int MPI_Type_hvector(count,blocklength,stride,oldtype,newtype);
Extension of vector: varying block lenghth an d strides. Return a new datatype that represents count block. Each block is defined by an entry of array_of_bocklength and array_of_displacement. Displacement between successive blocks are expressed in:
unit of oldtype.
int MPI_Type_indexed(count,array_of_blocklength,array_of_displacement, oldtype,newtype);
bytes
int MPI_Type_hindexed(count,array_of_blocklength,array_of_displacement, oldtype,newtype);
Use pseudo types MPI_LB to set the lower bound of a new datatype that beginning with one or more empty slots(with blocklength=1), and MPI_UB to round up the upper bound. Pseudo types MPI_LB and MPI_UB occupy no space (size=0) Return size(in bytes) of datatype (e.g. MPI_REAL=4)
int MPI_Type_extent(datatype,isize)
Deallocate datatype
int MPI_Type_free(newtype)
MPI_Type_struct
MPI_TYPE_STRUCT : Used to build a general derived type, allowing each block to consist of replications of different datatypes. MPI_TYPE_STRUCT is the most general way to construct an MPI derived type because it allows the length, location, and type of each component to be specified independently.
MPI_Type_struct(count,array_blocklength,array_location,array_types,newtype);
with count=2,array_blocklength={1,3},array_types={MPI_INT,MPI_DOUBLE}
MPI_INT MPI_DOUBLE
block 0 block 1
newtype
array_location[0] array_location[1]
MPI_Type_contiguous
MPI_TYPE_CONTIGUOUS: Used to create a new datatype showing a maptype consisting of the copies of a datatype into contiguous locations. MPI_TYPE_CONTIGUOUS is the simplest of these, describing a contiguous sequence of values in memory. For example,
MPI_Type_contiguous(2,MPI_DOUBLE,&MPI_2D_POINT); MPI_Type_contiguous(3,MPI_DOUBLE,&MPI_3D_POINT);
creates new type indicators MPI_2D_POINT and MPI_3D_POINT. These type indicators allow you to treat consecutive pairs of doubles as point coordinates in a 2-dimensional space and sequences of three doubles as point coordinates in a 3-dimensional space.
MPI_2D_POINT MPI_3D_POINT
Example: type_contiguous.c
MPI_Type_vector
MPI_TYPE_VECTOR: Used to build a derived type consisting of replicated datatypes in blocks uniformly displaced "stride" units apart, where one unit is the extent of old type.
MPI_TYPE_VECTOR describes several such sequences evenly spaced but not consecutive in memory. With it, you can reduce the submatrix example to
MPI_Type_vector(N, M, MM, MPI_DOUBLE, &MY_MPI_TYPE); MPI_Type_commit(&MY_MPI_TYPE); MPI_Send(A[K,L], 1, MY_MPI_TYPE, DEST, TAG, MPI_COMM_WORLD); MPI_Type_free(&MY_MPI_TYPE);
The consecutive blocks are the rows of the submatrix, each M elements long. There are N of them. These columns start MM elements apart because that is the declared row size of the array containing the submatrix.
(with count=2,blocklength=3,stride=5) 5 element stride between blocks
oldtype
newtype
MPI_Type_hvector
MPI_TYPE_HVECTOR : Used to build a derived type consisting of count copies of blocks of old type uniformly displaced stride bytes apart. H stands for heterogeneous. MPI_TYPE_HVECTOR is similar to MPI_TYPE_VECTOR except that the distance between successive blocks is specified in bytes rather than elements. The most common reason for specifying this distance in bytes would be that elements of some other type are interspersed in memory with the elements of interest. For example, if you had an array of type SparseElt, you could use MPI_TYPE_HVECTOR to describe the "array" of value components.
MPI_Type_indexed
MPI_TYPE_INDEXED: Used to build a derived type consisting of an old type replicated into a sequence of blocks (each block is a concatenation of the old datatype) of varying lengths and displacements, which are measured by the extent of old type.
MPI_TYPE_INDEXED describes sequences that may vary both in length and in spacing. Because the location of these sequences is measured in elements rather than bytes, it is most appropriate for identifying arbitrary parts of a single array.
Example: type_indexed.c
MPI_Type_hindexed
MPI_TYPE_HINDEXED: Used to build a derived type consisting of replicated datatypes with a variety of block lengths and displacements measured in bytes. MPI_TYPE_HINDEXED is similar to MPI_TYPE_INDEXED except that the locations are specified in bytes rather than elements. It allows the identification of arbitrary parts of arbitrary arrays, subject only to the requirement that they all have the same type.
Summary of Constructors
int MPI_Type_contiguous( int MPI_Datatype MPI_Datatype* count, oldtype, newtype)
Use when elements are contiguous in an array int MPI_Type_vector( int int int MPI_Datatype MPI_Datatype* count, blocklength, stride, oldtype, newtype)
Use when elements are equally spaced in an array int MPI_Type_indexed( int int int MPI_Datatype MPI_Datatype* count, blocklength[], disp[], oldtype, newtype)
15 DOUBLE
P1:MPI_Recv(MPI_2D_POINTs,&stat) P1:MPI_Get_count(&stat,MPI_2D_POINTs,&count)
LENA[0] = 1; MPI_Address(&X[0], &LOCA[0]); TYPA[0] = MPI_DOUBLE; LENA[1] = 1; MPI_Address(&Y[0], &LOCA[1]); TYPA[1] = MPI_DOUBLE; LENA[2] = 1; MPI_Address(&Z[0], &LOCA[2]); TYPA[2] = MPI_DOUBLE; LENA[3] = 1; MPI_Address(&X[0], &LOCA[3]); TYPA[3] = MPI_LB; Size of type MPI_LB and MPI_UB =0 LENA[4] = 1; MPI_Address(&X[1], &LOCA[4]); TYPA[4] = MPI_UB; MPI_Type_struct(5, LENA, LOCA, TYPA, &MY_TYPE); MPI_Type_commit(&MY_TYPE); MPI_Send(MPI_BOTTOM, N, MY_TYPE, DEST, TAG, MPI_COMM_WORLD); MPI_Type_free(&MY_TYPE);
Definition of MPI_TYPE_UB
Used to find the upper bound of "datatype"; returned in bytes, relative to the datatype origin.
Definition of MPI_TYPE_EXTENT
Used to return the extent of primitive and derived datatypes.
Definition of Extent
The distance between the lower bound and upper bound of a type.
Definition of MPI_TYPE_SIZE
Used to return the total size in bytes of the type signature of datatype, the data's total size that would be created by the datatype.
END
Reference: http://foxtrot.ncsa.uiuc.edu:8900/public/MPI/