Re: [OMPI users] Bad behavior in Allgatherv when a count is 0

2008-02-04 Thread George Bosilca
Kenneth, I cannot replicate this weird behavior with the current version in the trunk. I guess it has been fixed since 1.2.4. Thanks, george. On Dec 13, 2007, at 6:58 PM, Moreland, Kenneth wrote: I have found that on rare occasion Allgatherv fails to pass the data to all processes.

Re: [OMPI users] overlapping memcpy in ompi_coll_tuned_allgather_intra_bruck

2008-02-04 Thread George Bosilca
On Feb 4, 2008, at 11:56 AM, Number Cruncher wrote: George Bosilca wrote: Now, the overlapping case is a real exception. Obviously, it happened for at least two peoples (as per mailing list search) in about 4 years, but without affecting the correctness of the application. Is that a reason

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread George Bosilca
Richard, You're absolutely right. What a shame :) If I have spent less time drawing the boxes around the code I might have noticed the typo. The Send should be an Isend. george. On Feb 4, 2008, at 5:32 PM, Richard Treumann wrote: Hi George Sorry - This is not a valid MPI program. It v

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Richard Treumann
Hi George Sorry - This is not a valid MPI program. It violates the requirement that a program not depend on there being any system buffering. See page 32-33 of MPI 1.1 Lets simplify to: Task 0: MPI_Recv( from 1 with tag 1) MPI_Recv( from 1 with tag 0) Task 1: MPI_Send(to 0 with tag 0) MPI_Se

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread George Bosilca
Please allow me to slightly modify your example. It still follow the rules from the MPI standard, so I think it's a 100% standard compliant parallel application. ++ | task 0:| +

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Brightwell, Ronald
On Mon Feb 4, 2008 14:23:13... Sacerdoti, Federico wrote > To keep this out of the weeds, I have attached a program called "bug3" > that illustrates this problem on openmpi 1.2.5 using the openib BTL. In > bug3 process with rank 0 uses all available memory buffering > "unexpected" messages from it

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Sacerdoti, Federico
To keep this out of the weeds, I have attached a program called "bug3" that illustrates this problem on openmpi 1.2.5 using the openib BTL. In bug3 process with rank 0 uses all available memory buffering "unexpected" messages from its neighbors. Bug3 is a test-case derived from a real, scalable ap

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Brightwell, Ronald
> > I'm looking at a network where the number of endpoints is large enough that > > everybody can't have a credit to start with, and the "offender" isn't any > > single process, but rather a combination of processes doing N-to-1 where N > > is sufficiently large. I can't just tell one process to s

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Patrick Geoffray
Brightwell, Ronald wrote: I'm looking at a network where the number of endpoints is large enough that everybody can't have a credit to start with, and the "offender" isn't any single process, but rather a combination of processes doing N-to-1 where N is sufficiently large. I can't just tell one

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Gleb Natapov
On Mon, Feb 04, 2008 at 02:54:46PM -0500, Richard Treumann wrote: > In my example, each sender task 1 to n-1 will have one rendezvous message > to task 0 at a time. The MPI standard suggests descriptors be small enough > and there be enough descriptor space for reasonable programs . The > standar

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Brian W. Barrett
Now that this discussion has gone way off into the MPI standard woods :). Was your test using Open MPI 1.2.4 or 1.2.5 (the one with the segfault)? There was definitely a bug in 1.2.4 that could cause exactly the behavior you are describing when using the shared memory BTL, due to a silly delay

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Richard Treumann
Gleb In my example, each sender task 1 to n-1 will have one rendezvous message to task 0 at a time. The MPI standard suggests descriptors be small enough and there be enough descriptor space for reasonable programs . The standard is clear that unreasonable programs can run out of space and fail

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Brightwell, Ronald
> > Not to muddy the point, but if there's enough ambiguity in the Standard > > for people to ignore the progress rule, then I think (hope) there's enough > > ambiguity for people to ignore the sender throttling issue too ;) > > I understand your position, and I used to agree until I was forced to

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Patrick Geoffray
Ron, Brightwell, Ronald wrote: Not to muddy the point, but if there's enough ambiguity in the Standard for people to ignore the progress rule, then I think (hope) there's enough ambiguity for people to ignore the sender throttling issue too ;) I understand your position, and I used to agree un

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Gleb Natapov
On Mon, Feb 04, 2008 at 09:08:45AM -0500, Richard Treumann wrote: > To me, the MPI standard is clear that a program like this: > > task 0: > MPI_Init > sleep(3000); > start receiving messages > > each of tasks 1 to n-1: > MPI_Init > loop 5000 times >MPI_Send(small message to 0) > end loop >

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Brightwell, Ronald
> > I am well aware of the scaling problems related to the standard > send requirements in MPI. I t is a very difficult issue. > > However, here is what the standard says: MPI 1.2, page 32 lines 29-37 > > [...] I'm well aware of those words. They are highlighted (in pink no less) in on page 50

[OMPI users] Fw: openmpi credits for eager messages

2008-02-04 Thread Richard Treumann
Sorry for typo - The reference is MPI 1.1 Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 - Forwarded by Richard Treumann/Poughkeepsie/IBM on 02/04/2008 01:36 PM ---

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Richard Treumann
Hi Ron - I am well aware of the scaling problems related to the standard send requirements in MPI. I t is a very difficult issue. However, here is what the standard says: MPI 1.2, page 32 lines 29-37 === a standard send operation that cannot complete because of lack of buffer space will me

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Ron Brightwell
> Is what George says accurate? If so, it sounds to me like OpenMPI > does not comply with the MPI standard on the behavior of eager > protocol. MPICH is getting dinged in this discussion because they > have complied with the requirements of the MPI standard. IBM MPI > also complies with the stand

Re: [OMPI users] overlapping memcpy in ompi_coll_tuned_allgather_intra_bruck

2008-02-04 Thread Number Cruncher
George Bosilca wrote: Now, the overlapping case is a real exception. Obviously, it happened for at least two peoples (as per mailing list search) in about 4 years, but without affecting the correctness of the application. Is that a reason good enough to effect the overall performance of all p

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Christian Bell
The words 'eager', 'rendezvous' or 'credit' have a specific resonance only for implementors and i think it's correct that the MPI specification sidestep these words since they are artifacts of implementation. All implementations make their own guarantees and run into their own different limitation

Re: [OMPI users] Dynamically changing machine file data

2008-02-04 Thread Christian Bell
On Fri, 01 Feb 2008, Keith Hellman wrote: > I'm working on a rather eccentric application of openmpi. Is there > any way to dynamically change the information obtained from the > machine or hosts file provided on the command line to mpirun? > > Perhaps something like: > MPI_Reread_Hosts( cons

Re: [OMPI users] openmpi credits for eager messages

2008-02-04 Thread Richard Treumann
Is what George says accurate? If so, it sounds to me like OpenMPI does not comply with the MPI standard on the behavior of eager protocol. MPICH is getting dinged in this discussion because they have complied with the requirements of the MPI standard. IBM MPI also complies with the standard. If