Re: [O-MPI users] direct openib btl and latency

2006-02-09 Thread Sayantan Sur
Galen Shipman wrote: >Hi Jean, > >You probably are not seeing overhead costs so much as you are seeing >the difference between using send/recv for small messages, which Open >MPI uses, and RDMA for small messages. If you are comparing against >another implementation that uses RDMA for small messa

Re: [O-MPI users] Firewall ports and Mac OS X 10.4.4

2006-02-09 Thread Brian Barrett
On Feb 9, 2006, at 6:50 PM, James Conway wrote: I couldn't find any information on firewall ports to open up for using OpenMPI. I have compiled and successfully run simple commands (eg mpirun with "uname -n") on the localhost, but including remote hosts caused a hang. Statements in the remote .c

Re: [O-MPI users] direct openib btl and latency

2006-02-09 Thread Jean-Christophe Hugly
On Thu, 2006-02-09 at 16:37 -0700, Brightwell, Ronald wrote: > I apologize if it seems like I'm picking on you. No offense taken. > I'm hypersensitive to > people trying to make judgements based on micro-benchmark performance. > I've been trying to make an argument that two-node ping-pong laten

[O-MPI users] Firewall ports and Mac OS X 10.4.4

2006-02-09 Thread James Conway
I couldn't find any information on firewall ports to open up for using OpenMPI. I have compiled and successfully run simple commands (eg mpirun with "uname -n") on the localhost, but including remote hosts caused a hang. Statements in the remote .cshrc to echo would be returned, but nothing

Re: [O-MPI users] direct openib btl and latency

2006-02-09 Thread Brightwell, Ronald
> > No, I assumed it based on comparisions between doing and not doing small > msg rdma at various scales, from a paper Galen pointed out to me. > http://www.cs.unm.edu/~treport/tr/05-10/Infiniband.pdf > Actually, I wasn't so much concerned with how you jumped to your conclusion. I just wanted t

[O-MPI users] Job fails with mpiP

2006-02-09 Thread Aniruddha Shet
Hi, I am trying to profile an Open MPI job using the mpiP profiling library. Running the job without the library completes successfully. When I link the profiling library into the executable, the job fails to run. I am able to build the job with mpiP, but the execution fails. Please see the attach

Re: [O-MPI users] direct openib btl and latency

2006-02-09 Thread Galen Shipman
On Feb 9, 2006, at 3:03 PM, Jean-Christophe Hugly wrote: On Thu, 2006-02-09 at 14:05 -0700, Ron Brightwell wrote: [...] From an adoption perspective, though, the ability to shine in micro-benchmarks is important, even if it means using an ad-hoc tuning. There is some justification for it af

Re: [O-MPI users] direct openib btl and latency

2006-02-09 Thread Jean-Christophe Hugly
On Thu, 2006-02-09 at 14:05 -0700, Ron Brightwell wrote: > > [...] > > > > >From an adoption perspective, though, the ability to shine in > > micro-benchmarks is important, even if it means using an ad-hoc tuning. > > There is some justification for it after all. There are small clusters > > out t

Re: [O-MPI users] direct openib btl and latency

2006-02-09 Thread Galen Shipman
When do you plan on having the small-msg rdma option available ? I would expect this in the very near future, we will be discussing schedules next week. Thanks, Galen J-C -- Jean-Christophe Hugly PANTA

Re: [O-MPI users] direct openib btl and latency

2006-02-09 Thread Ron Brightwell
> [...] > > >From an adoption perspective, though, the ability to shine in > micro-benchmarks is important, even if it means using an ad-hoc tuning. > There is some justification for it after all. There are small clusters > out there (many more than big ones, in fact) so taking maximum advantage >

Re: [O-MPI users] direct openib btl and latency

2006-02-09 Thread Jean-Christophe Hugly
> > > So far, the best latency I got from ompi is 5.24 us, and the best I > > got from mvapich is 3.15. > > I am perfectly ready to accept that ompi scales better and that may be > > more important (except to the marketing dept :-) ), but I do not > > understand your explanation based on small-

Re: [O-MPI users] direct openib btl and latency

2006-02-09 Thread Galen Shipman
I would recommend reading the following tech report, it should shed some light to how these things work : http://www.cs.unm.edu/research/search_technical_reports_by_keyword/? string=infiniband 1 - It does not seem that mvapich does RDMA for small messages. It will do RDMA for any message