Now that this discussion has gone way off into the MPI standard woods :).
Was your test using Open MPI 1.2.4 or 1.2.5 (the one with the segfault)?
There was definitely a bug in 1.2.4 that could cause exactly the behavior
you are describing when using the shared memory BTL, due to a silly
delayed initialization bug/optimization.
If you are using the OB1 PML (the default), you will still have the
possibility of running the receiver out of memory if the unexpected queue
grows without bounds. I'll withold my opinion on what the standard says
so that we can perhaps actually help you solve your problem and stay out
of the weeds :). Note however, that in general unexpected messages are a
bad idea and thousands of them from one peer to another should be avoided
at all costs -- this is just good MPI programming practice.
Now, if you are using MX, you can replicate MPICH/MX's behavior (including
the very slow part) by using the CM PML (--mca pml cm on the mpirun
command line), which will use the MX library message matching and
unexpected queue and therefore behave exactly like MPICH/MX.
Brian
On Sat, 2 Feb 2008, 8mj6tc...@sneakemail.com wrote:
That would make sense. I able to break OpenMPI by having Node A wait for
messages from Node B. Node B is in fact sleeping while Node C bombards
Node A with a few thousand messages. After a while Node B wakes up and
sends Node A the message it's been waiting on, but Node A has long since
been buried and seg faults. If I decrease the number of messages C is
sending, it works properly. This was on OpenMPI 1.2.4 (using I think the
SM BTL (might have been MX or TCP, but certainly not infiniband. I could
dig up the test and try again if anyone is seriously curious).
Trying the same test on MPICH/MX went very very slow (I don't think they
have any clever buffer management) but it didn't crash.
Sacerdoti, Federico Federico.Sacerdoti-at-deshaw.com
|openmpi-users/Allow| wrote:
Hi,
I am readying an openmpi 1.2.5 software stack for use with a
many-thousand core cluster. I have a question about sending small
messages that I hope can be answered on this list.
I was under the impression that if node A wants to send a small MPI
message to node B, it must have a credit to do so. The credit assures A
that B has enough buffer space to accept the message. Credits are
required by the mpi layer regardless of the BTL transport layer used.
I have been told by a Voltaire tech that this is not so, the credits are
used by the infiniband transport layer to reliably send a message, and
is not an openmpi feature.
Thanks,
Federico
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
