Re: [OMPI users] OpenMPI fails to initalize the openib btl when run from SGE

2007-08-22 Thread Noam Meltzer
Hi, Thanks all who answered. The problem was indeed in the max. locked memory limitation. Though, changing it in /default/common/settings.sh was not enough. I also had to add ". /default/common/settings.sh" to /default/common/sgeexecd (and to /etc/init.d/sgeexecd on the compute nodes) as when

Re: [OMPI users] OpenMPI fails to initalize the openib btl when run from SGE

2007-08-22 Thread Jeff Squyres
I suspect that your SGE daemons are not starting with the proper locked memory limits (and therefore jobs started under SGE get severely limited locked memory limits). See these FAQ entries -- the issues described for SLURM are applicable to all resource managers (to include SGE): http://

Re: [OMPI users] OpenMPI fails to initalize the openib btl when run from SGE

2007-08-22 Thread Gleb Natapov
On Wed, Aug 22, 2007 at 03:31:20PM +0300, Noam Meltzer wrote: > Hi, > > I am running openmpi-1.2.3 compiled for 64bit on RHEL4u4. > I also have a Voltaire InfiniBand interconnect. > When I manually run jobs using the following command: > > /opt/local/openmpi-1.2.3-gcc4/bin/orterun -np 8 -hostfile

[OMPI users] OpenMPI fails to initalize the openib btl when run from SGE

2007-08-22 Thread Noam Meltzer
Hi, I am running openmpi-1.2.3 compiled for 64bit on RHEL4u4. I also have a Voltaire InfiniBand interconnect. When I manually run jobs using the following command: /opt/local/openmpi-1.2.3-gcc4/bin/orterun -np 8 -hostfile ~/myHostList -mca btl self,openib /tcc/eandm/performance/igor/main.exe.op