Re: [OMPI users] using OpenMPI + SGE in a heterogeneous network

2008-06-06 Thread Reuti
Hi, Am 07.06.2008 um 01:07 schrieb John Hearns: On Fri, 2008-06-06 at 17:56 +0100, SLIM H.A. wrote: Hi I want to use SGE to run jobs on a cluster with mx and infiniband nodes. By dividing the nodes into two host groups SGE will submit to either interconnect. The interconnect can be specif

Re: [OMPI users] using OpenMPI + SGE in a heterogeneous network

2008-06-06 Thread George Bosilca
How do you know the BTL is used ? I run applications in such context, and as soon as I force the BTLs everything works like a charm. Please try "--mca pml ob1" to the command line, this will do the trick george. On Jun 6, 2008, at 6:06 PM, SLIM H.A. wrote: The btl interface is used. The no

Re: [OMPI users] using OpenMPI + SGE in a heterogeneous network

2008-06-06 Thread John Hearns
On Fri, 2008-06-06 at 17:56 +0100, SLIM H.A. wrote: > Hi > > I want to use SGE to run jobs on a cluster with mx and infiniband nodes. > By dividing the nodes into two host groups SGE will submit to either > interconnect. > > The interconnect can be specified in the mpirun command with the --mca >

Re: [OMPI users] using OpenMPI + SGE in a heterogeneous network

2008-06-06 Thread Reuti
Am 07.06.2008 um 00:06 schrieb SLIM H.A.: The btl interface is used. The nodes in both subclusters have the same architecture so no need for two binaries. The MPI libraries for the interconnect can be selected at runtime, for example through the value of the OMPI_MCA_btl environment variabl

Re: [OMPI users] using OpenMPI + SGE in a heterogeneous network

2008-06-06 Thread SLIM H.A.
The btl interface is used. The nodes in both subclusters have the same architecture so no need for two binaries. The MPI libraries for the interconnect can be selected at runtime, for example through the value of the OMPI_MCA_btl environment variable. However that variable should then somehow be

Re: [OMPI users] using OpenMPI + SGE in a heterogeneous network

2008-06-06 Thread Reuti
Am 06.06.2008 um 19:31 schrieb Patrick Geoffray: SLIM H.A. wrote: I would be grateful for any advice Just to check, you are not using the MTL for MX, right ? Only the BTL interface allow to choose between several devices at run time. At least there would be the option to built two binaries

Re: [OMPI users] using OpenMPI + SGE in a heterogeneous network

2008-06-06 Thread Patrick Geoffray
SLIM H.A. wrote: I would be grateful for any advice Just to check, you are not using the MTL for MX, right ? Only the BTL interface allow to choose between several devices at run time. Patrick

Re: [OMPI users] OpenMPI scaling > 512 cores

2008-06-06 Thread Scott Shaw
Thank you to all that replied regarding my questions. I have tried all the options suggested but unfortunately I still run into the same problem. I am at a point were I have exhausted all of the options available with the OpenMPI v1.2.2 release and moving to v1.2.6 later today. Hopefully mca_oo

[OMPI users] using OpenMPI + SGE in a heterogeneous network

2008-06-06 Thread SLIM H.A.
Hi I want to use SGE to run jobs on a cluster with mx and infiniband nodes. By dividing the nodes into two host groups SGE will submit to either interconnect. The interconnect can be specified in the mpirun command with the --mca btl parameter. However users would then have to decide at runtime

Re: [OMPI users] Question about RDMA

2008-06-06 Thread Gabriele Fatigati
Hi Jeff, thanks for you reply. I did understand previous questions about RDMA. Ever with SKaMPI, i tried to run with mpi_leave_pinned = 1, as you have suggested. But also in this case, execution time is very similar to previous case. Does it means that SKaMPI, reallocates buffer every time ? For

Re: [OMPI users] OpenIB problem: error polling HP CQ...

2008-06-06 Thread Matt Hughes
2008/6/4 Jeff Squyres : > Would it be possible for you to try a trunk nightly tarball snapshot, > perchance? I have attempted to use openmpi-1.3a1r18569. After some pain getting MPI_Comm_spawn to work (I will write about that in a separate message), I was able to get my app started. If segfaults