Re: [OMPI users] OpenMPI with openib partitions

2008-10-07 Thread Jeff Squyres
FWIW, if this configuration is for all of your users, you might want to specify these MCA params in the default MCA param file, or the environment, ...etc. Just so that you don't have to specify it on every mpirun command line. See http://www.open-mpi.org/faq/?category=tuning#setting-mca-p

Re: [OMPI users] OpenMPI with openib partitions

2008-10-07 Thread Pavel Shamis (Pasha)
Matt, I guess that you have some problem with partition configuration. Can you share with us your partition configuration file (by default opensm use /etc/opensm/partitions.conf) and guid from your machines ( ibstat | grep GUID ) ? Regards, Pasha Matt Burgess wrote: Hi, I'm trying to get o

Re: [OMPI users] OpenMPI with openib partitions

2008-10-07 Thread Lenny Verkhovsky
Sorry, misunderstood the question, thanks for Pasha the right command line will be -mca btl openib,self -mca btl_openib_of_pkey_val 0x8109 -mca btl_openib_of_pkey_ix 1 ex. #mpirun -np 2 -H witch2,witch3 -mca btl openib,self -mca btl_openib_of_pkey_val 0x8001 -mca btl_openib_of_pkey_ix 1 ./mpi_p

Re: [OMPI users] OpenMPI with openib partitions

2008-10-06 Thread Jeff Squyres
On Oct 5, 2008, at 1:22 PM, Lenny Verkhovsky wrote: you should probably use -mca tcp,self -mca btl_openib_if_include ib0.8109 Really? I thought we only took OpenFabrics device names in the openib_if_include MCA param...? It looks like ib0.8109 is an IPoIB device name. Lenny. On

Re: [OMPI users] OpenMPI with openib partitions

2008-10-05 Thread Lenny Verkhovsky
Hi, you should probably use -mca tcp,self -mca btl_openib_if_include ib0.8109 Lenny. On 10/3/08, Matt Burgess wrote: > > Hi, > > > I'm trying to get openmpi working over openib partitions. On this cluster, > the partition number is 0x109. The ib interfaces are pingable over the > appropriate i