Let's say you have multi-threaded MPI processes, you request
MPI_THREAD_MULTIPLE and get MPI_THREAD_MULTIPLE, and you use the
self,sm,tcp BTLs (which have some degree of threading support). Is it
okay to have an [MPI_Isend|MPI_Irecv] on one thread be completed by an
MPI_Wait on another thread? I'm assuming some sort of synchronization
and memory barrier/flush in between to protect against funny race
conditions.
If it makes things any easier on you, we can do this multiple-choice style:
1) Forbidden by the MPI standard.
2) Not forbidden by the MPI standard, but will not work with OMPI (not
even with the BTLs that claim to be multi-threaded).
3) Works well with OMPI (provided you use a BTL that's multi-threaded).
It's looking like #2 to me, but I'm not sure.