Re: [OMPI users] strange IMB runs

2009-08-14 Thread Michael Di Domenico
> One of the differences among MPI implementations is the default placement of > processes within the node. E.g., should processes by default be collocated > on cores of the same socket or on cores of different sockets? I don't know > if that issue is applicable here (that is, HP MPI vs Open MPI

Re: [OMPI users] strange IMB runs

2009-08-14 Thread Michael Di Domenico
On Thu, Aug 13, 2009 at 1:51 AM, Eugene Loh wrote: > Also, I'm puzzled why you should see better results by changing > btl_sm_eager_limit. That shouldn't change long-message bandwidth, but only > the message size at which one transitions from short to long messages. If > anything, tweaking btl_sm

Re: [OMPI users] strange IMB runs

2009-08-13 Thread Michael Di Domenico
On Thu, Aug 13, 2009 at 1:51 AM, Eugene Loh wrote: >>Is this behavior expected? Are there any tunables to get the OpenMPI >>sockets up near HP-MPI? > > First, I want to understand the configuration. It's just a single node. No > interconnect (InfiniBand or Ethernet or anything). Right? Yes, th

Re: [OMPI users] strange IMB runs

2009-08-13 Thread Ralph Castain
Just a couple of data points: 1. so we don't confuse folks, there is no legal thing about a space in OpenMPI. Heck, most of us developers drop the space in our discussions. It was put in there to avoid confusion with OpenMP. While the more marketing oriented worry about it, the rest of the

Re: [OMPI users] strange IMB runs

2009-08-13 Thread Eugene Loh
I was away on vacation for two weeks and therefore missed most of this thread, but I'm quite interested. Michael Di Domenico wrote: >I'm not sure I understand what's actually happened here. I'm running >IMB on an HP superdome, just comparing the PingPong benchmark > >HP-MPI v2.3 >Max ~ 700-800

Re: [OMPI users] strange IMB runs

2009-08-12 Thread Michael Di Domenico
So pushing this along a little more running with openmpi-1.3 svn rev 20295 mpirun -np 2 -mca btl sm,self -mca mpi_paffinity_alone 1 -mca mpi_leave_pinned 1 -mca btl_sm_eager_limit 8192 $PWD/IMB-MPI1 pingpong Yields ~390MB/sec So we're getting there, but still only about half speed On

Re: [OMPI users] strange IMB runs

2009-08-06 Thread Michael Di Domenico
Here's an interesting data point. I installed the RHEL rpm version of OpenMPI 1.2.7-6 for ia64 mpirun -np 2 -mca btl self,sm -mca mpi_paffinity_alone 1 -mca mpi_leave_pinned 1 $PWD/IMB-MPI1 pingpong With v1.3 and -mca btl self,sm i get ~150MB/sec With v1.3 and -mca btl self,tcp i get ~550MB/sec

Re: [OMPI users] strange IMB runs

2009-07-31 Thread Edgar Gabriel
Michael Di Domenico wrote: mpi_leave_pinned didn't help still at ~145MB/sec btl_sm_eager_limit from 4096 to 8192 pushes me upto ~212MB/sec, but pushing it past that doesn't change it anymore Are there any intelligent programs that can go through and test all the different permutations of tunable

Re: [OMPI users] strange IMB runs

2009-07-31 Thread Michael Di Domenico
mpi_leave_pinned didn't help still at ~145MB/sec btl_sm_eager_limit from 4096 to 8192 pushes me upto ~212MB/sec, but pushing it past that doesn't change it anymore Are there any intelligent programs that can go through and test all the different permutations of tunables for openmpi? Outside of me

Re: [OMPI users] strange IMB runs

2009-07-30 Thread Michael Di Domenico
On Thu, Jul 30, 2009 at 10:08 AM, George Bosilca wrote: > The leave pinned will not help in this context. It can only help for devices > capable of real RMA operations and that require pinned memory, which > unfortunately is not the case for TCP. What is [really] strange about your > results is tha

Re: [OMPI users] strange IMB runs

2009-07-30 Thread George Bosilca
The leave pinned will not help in this context. It can only help for devices capable of real RMA operations and that require pinned memory, which unfortunately is not the case for TCP. What is [really] strange about your results is that you get a 4 times better bandwidth over TCP than over

Re: [OMPI users] strange IMB runs

2009-07-29 Thread Dorian Krause
Hi, --mca mpi_leave_pinned 1 might help. Take a look at the FAQ for various tuning parameters. Michael Di Domenico wrote: I'm not sure I understand what's actually happened here. I'm running IMB on an HP superdome, just comparing the PingPong benchmark HP-MPI v2.3 Max ~ 700-800MB/sec OpenM