Re: [OMPI users] OpenMPI 1.6.3 and Memory Issues

2012-11-29 Thread Joseph Farran
Hi again. I am using /etc/modprobe.d/mofed.conf, otherwise I get: WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/ But I am still getting the memory errors after making the changes and rebooting: $ cat /etc/modprobe.d/mofed.conf options mlx4_co

Re: [OMPI users] OpenMPI 1.6.3 and Memory Issues

2012-11-29 Thread Yevgeny Kliteynik
You can also set these parameters in /etc/modprobe.conf: options mlx4_core log_num_mtt=24 log_mtts_per_seg=1 -- YK On 11/30/2012 2:12 AM, Yevgeny Kliteynik wrote: > On 11/30/2012 12:47 AM, Joseph Farran wrote: >> I'll assume: /etc/modprobe.d/mlx4_en.conf > > Add these to /etc/modprobe.d/mofed

Re: [OMPI users] OpenMPI 1.6.3 and Memory Issues

2012-11-29 Thread Yevgeny Kliteynik
On 11/30/2012 12:47 AM, Joseph Farran wrote: > I'll assume: /etc/modprobe.d/mlx4_en.conf Add these to /etc/modprobe.d/mofed.conf: options mlx4_core log_num_mtt=24 options mlx4_core log_mtts_per_seg=1 And then restart the driver. You need to do it on all the machines. -- YK > > On 11/29/2012 0

Re: [OMPI users] OpenMPI 1.6.3 and Memory Issues

2012-11-29 Thread Joseph Farran
I'll assume: /etc/modprobe.d/mlx4_en.conf On 11/29/2012 02:34 PM, Joseph Farran wrote: Where do change those mellanox settings? On 11/29/2012 02:23 PM, Jeff Squyres wrote: See http://www.open-mpi.org/faq/?category=openfabrics#ib-low-reg-mem. On Nov 29, 2012, at 5:21 PM, Joseph Farran wrote:

Re: [OMPI users] OpenMPI 1.6.3 and Memory Issues

2012-11-29 Thread Joseph Farran
Where do change those mellanox settings? On 11/29/2012 02:23 PM, Jeff Squyres wrote: See http://www.open-mpi.org/faq/?category=openfabrics#ib-low-reg-mem. On Nov 29, 2012, at 5:21 PM, Joseph Farran wrote:

Re: [OMPI users] OpenMPI 1.6.3 and Memory Issues

2012-11-29 Thread Jeff Squyres
See http://www.open-mpi.org/faq/?category=openfabrics#ib-low-reg-mem. On Nov 29, 2012, at 5:21 PM, Joseph Farran wrote: > Hi All. > > In compiling a simple Hello world with OpenMPI 1.6.3 and mpirun the hello > program, I am getting: > > $ ulimit -l unlimited > $ mpirun -np 2 hello > --

[OMPI users] OpenMPI 1.6.3 and Memory Issues

2012-11-29 Thread Joseph Farran
Hi All. In compiling a simple Hello world with OpenMPI 1.6.3 and mpirun the hello program, I am getting: $ ulimit -l unlimited $ mpirun -np 2 hello -- WARNING: It appears that your OpenFabrics subsystem is configured to onl