Hi again,
Let me clarify the context of the problem. I'm implementing a MPI piggyback
mechanism that should allow for attaching extra data to any MPI message. The
idea is to wrap MPI communication calls with PMPI interface (or with dynamic
instrumentation or whatsoever) and add/receive extra data in a non expensive
way. The best solution I have found so far is dynamic datatype wrapping.
That is when a user calls MPI_Send (datatype, count) I create dynamically a
new structure type that contains an array [count] of datatype and extra
data. To avoid copying the original send buffer I use absolute addresses to
define displacaments in the structure. This works fine for all P2P calls and
MPI_Bcast. And definitely it has performance benefits when compared to
copying bufferers or sending an additional message in a different
communicator. Or would you expect something different?
The only problem are collective calls like MPI_Gather when a root process
receives an array of data items. There is no problem to wrap the message on
the sender side (for each task), but the question is how to define a
datatype that points both to original receive buffer and extra buffer for
piggybacked data AND has an adecuate extent to work as an array element.
The real problem is that a structure datatype { original data, extra data}
does not have a constant displacement between the original data and extra
data. Eg. consider original data = receive buffer in MPI_Gather and extra
data is an array of ints somewhere in memory). So it cannot be directly used
as an array datatype.
Any solution? It could be complex, I don't mind ;)
On 11/1/07, George Bosilca <bosi...@eecs.utk.edu> wrote:
>
> The MPI standard defines the upper bound and the upper bound for
> similar problems. However, even with all the functions in the MPI
> standard we cannot describe all types of data. There is always a
> solution, but sometimes one has to ask if the performance gain is
> worth the complexity introduced.
As I said there is always a solution. In fact there are 2 solution,
> one somehow optimal the other ... as bad as you can imagine.
>
> The bad approach:
> 1. Use an MPI_Type_struct to create exactly what you want, element
> by element (i.e single pair). This can work in all cases.
2. If the sizeof(int) == sizeof(double) then the displacement inside
> each tuple (double_i, int_i) is constant. Therefore, you can start by
> creating one "single element" type and then use for each send the
> correct displacement in the array (added to the send buffer,
> respectively to the receive one).
>
> george.
>
> On Oct 31, 2007, at 1:40 PM, Oleg Morajko wrote:
>
> > Hello,
> >
> > I have the following problem. There areI two arrays somewere in the
> > program:
> >
> > double weights [MAX_SIZE];
> > ...
> > int values [MAX_SIZE];
> > ...
> >
> > I need to be able to send a single pair { weights [i], values [i] }
> > with a single MPI_Send call Or receive it directly into both arrays
> > at at given index i. How can I define a datatype that spans this
> > pair over both arrays?
> >
> > The only additional constraint it the fact that the memory location
> > of both arrays is fixed and cannot be changed and I should avoid
> > extra copies.
> >
> > Is it possible?
> >
> > Any help welcome,
> > Oleg Morajko
> >
> >
> > _______________________________________________
> > users mailing list
> > us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
