Re: [OMPI users] memory per core/process

2013-03-30 Thread Duke Nguyen
I should really have asked earlier. Thanks for all the helps. D. On 3/30/13 10:28 PM, Ralph Castain wrote: FWIW: there is an MCA param that helps with such problems: opal_set_max_sys_limits "Set to non-zero to automatically set any system-imposed limits to the maxim

Re: [OMPI users] OpenMPI + BLCR + LSF integration

2013-03-30 Thread Ralph Castain
It is quite likely that the lsf integration on the 1.6 series is broken. We don't have a way to test it any more (all our LSF access is gone). I recently was briefly given access to an LSF machine and fixed it for the 1.7 series, but that series doesn't support checkpoint/restart. On Mar 30, 2

Re: [OMPI users] memory per core/process

2013-03-30 Thread Ralph Castain
FWIW: there is an MCA param that helps with such problems: opal_set_max_sys_limits "Set to non-zero to automatically set any system-imposed limits to the maximum allowed", At the moment, it only sets the limits on number of files open, and max size of a file we can crea

Re: [OMPI users] memory per core/process

2013-03-30 Thread Gustavo Correa
On Mar 30, 2013, at 10:02 AM, Duke Nguyen wrote: > On 3/30/13 8:20 PM, Reuti wrote: >> Am 30.03.2013 um 13:26 schrieb Tim Prince: >> >>> On 03/30/2013 06:36 AM, Duke Nguyen wrote: On 3/30/13 5:22 PM, Duke Nguyen wrote: > On 3/30/13 3:13 PM, Patrick Bégou wrote: >> I do not know abou

Re: [OMPI users] memory per core/process

2013-03-30 Thread Duke Nguyen
On 3/30/13 8:20 PM, Reuti wrote: Am 30.03.2013 um 13:26 schrieb Tim Prince: On 03/30/2013 06:36 AM, Duke Nguyen wrote: On 3/30/13 5:22 PM, Duke Nguyen wrote: On 3/30/13 3:13 PM, Patrick Bégou wrote: I do not know about your code but: 1) did you check stack limitations ? Typically intel fort

Re: [OMPI users] memory per core/process

2013-03-30 Thread Patrick Bégou
Ok, so your problem is identified as a stack size problem. I went into these limitations using Intel fortran compilers on large data problems. First, it seems you can increase your stack size as "ulimit -s unlimited" works (you didn't enforce the system hard limit). The best way is to set thi

Re: [OMPI users] memory per core/process

2013-03-30 Thread Reuti
Am 30.03.2013 um 13:26 schrieb Tim Prince: > On 03/30/2013 06:36 AM, Duke Nguyen wrote: >> On 3/30/13 5:22 PM, Duke Nguyen wrote: >>> On 3/30/13 3:13 PM, Patrick Bégou wrote: I do not know about your code but: 1) did you check stack limitations ? Typically intel fortran codes needs

Re: [OMPI users] memory per core/process

2013-03-30 Thread Tim Prince
On 03/30/2013 06:36 AM, Duke Nguyen wrote: On 3/30/13 5:22 PM, Duke Nguyen wrote: On 3/30/13 3:13 PM, Patrick Bégou wrote: I do not know about your code but: 1) did you check stack limitations ? Typically intel fortran codes needs large amount of stack when the problem size increase. Check u

Re: [OMPI users] memory per core/process

2013-03-30 Thread Duke Nguyen
On 3/30/13 5:22 PM, Duke Nguyen wrote: On 3/30/13 3:13 PM, Patrick Bégou wrote: I do not know about your code but: 1) did you check stack limitations ? Typically intel fortran codes needs large amount of stack when the problem size increase. Check ulimit -a First time I heard of stack limit

Re: [OMPI users] memory per core/process

2013-03-30 Thread Reuti
Am 30.03.2013 um 05:21 schrieb Duke Nguyen: > Hi folks, > > I am sorry if this question had been asked before, but after ten days of > searching/working on the system, I surrender :(. We try to use mpirun to run > abinit (abinit.org) which in turns will call an input file to run some > simulat

Re: [OMPI users] memory per core/process

2013-03-30 Thread Duke Nguyen
On 3/30/13 3:13 PM, Patrick Bégou wrote: I do not know about your code but: 1) did you check stack limitations ? Typically intel fortran codes needs large amount of stack when the problem size increase. Check ulimit -a First time I heard of stack limitations. Anyway, ulimit -a gives $ ulimi

Re: [OMPI users] memory per core/process

2013-03-30 Thread Patrick Bégou
I do not know about your code but: 1) did you check stack limitations ? Typically intel fortran codes needs large amount of stack when the problem size increase. Check ulimit -a 2) did your node uses cpuset and memory limitation like fake numa to set the maximum amount of memory available for

[OMPI users] OpenMPI + BLCR + LSF integration

2013-03-30 Thread Jorge Naranjo Bouzas
Hello! We are having problems integrating BLCR + OpenMPI + LSF in a linux cluster with Infiniband We compiled OpenMPI version 1.6 with gcc version 4.6.0 ... The configure line was like: ./configure --prefix=/opt/share/mpi-openmpi/1.6-gcc-4.6.0/el6/x86_64 --with-lsf --with-openib --with-blcr=/opt

[OMPI users] memory per core/process

2013-03-30 Thread Duke Nguyen
Hi folks, I am sorry if this question had been asked before, but after ten days of searching/working on the system, I surrender :(. We try to use mpirun to run abinit (abinit.org) which in turns will call an input file to run some simulation. The command to run is pretty simple $ mpirun -np