Daniel,
Your understanding of he MPI standard requirement with regard to
MPI_Alltoallv is now 100% accurate. The send count and datatype should
match what the receiver expect. You can always use an MPI_Alltoall
before the MPI_Alltoallv to exchange the lengths that you expect.
george.
On Sep 10, 2008, at 1:46 PM, Daniel Spångberg wrote:
Dear all,
First some background, the real question is at the end of this
(longish) mail.
I have a problem where I need to exchange data between all
processes. The data is unevenly distributed and I thought at first I
could use MPI_Alltoallv to transfer the data. However, in my case,
the receivers do not know how many data items the senders will send,
but it is relatively easy to set up so the receiver can figure out
the maximum number of items the sender will send, so I set the
recvcounts to the maximum possible, and the sendcounts to the actual
number of elements (smaller than recvcounts).
The mpi-forum description (from http://www.mpi-forum.org/docs/mpi21-report/node99.htm)
describes the following:
MPI_ALLTOALLV(sendbuf, sendcounts, sdispls, sendtype, recvbuf,
recvcounts, rdispls, recvtype, comm)
IN sendbuf starting address of send buffer (choice)
IN sendcounts integer array equal to the group size specifying the
number of elements to send to each processor
IN sdispls integer array (of length group size). Entry j specifies
the displacement (relative to sendbuf) from which to take the
outgoing data destined for process j
IN sendtype data type of send buffer elements (handle)
OUT recvbuf address of receive buffer (choice)
IN recvcounts integer array equal to the group size specifying the
number of elements that can be received from each processor
IN rdispls integer array (of length group size). Entry i specifies
the displacement (relative to recvbuf) at which to place the
incoming data from process i
IN recvtype data type of receive buffer elements (handle)
IN comm communicator (handle)
In particular the wording is "the number of elements that can be
received from each processor" for recvcounts, and does not say that
this must be exactly the same as the number of elements sent.
It also mentions that it should work similarly as a number of
independent MPI_Send/MPI_Recv calls. The amount of data sent in such
a case does not need to exactly match the amount of data received.
I, unfortunately, missed the following:
The type signature associated with sendcounts[j], sendtypes[j] at
process i must be equal to the type signature associated with
recvcounts[i], recvtypes[i] at process j. This implies that the
amount of data sent must be equal to the amount of data received,
pairwise between every pair of processes. Distinct type maps between
sender and receiver are still allowed.
And the openmpi man page shows
When a pair of processes exchanges data, each may pass
different ele-
ment count and datatype arguments so long as the sender
specifies the
same amount of data to send (in bytes) as the receiver
expects to
receive.
I did test my program on different send/recv counts, and while it
sometimes works, sometimes it does not. Even if it worked I would
not be comfortable using it anyway.
The question is: If there is no way of determining the length of the
data sent by the sender on the receiving end, I see two options:
Either always transmit too much data using MPI_Alltoall(v) or cook
up my own routine based on PTP calls, probably MPI_Sendrecv is the
best option. Am I missing something?
--
Daniel Spångberg
Materials Chemistry
Uppsala University
Sweden
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
