Re: [OMPI users] Clarification about OpenMPI, slurm and PMI interface

2014-08-21 Thread Filippo Spiga
Dear Mike, it sounds good... the description fits my purposes... I really miss this when I was reading srun man page! I will give it a try Thanks to everybody for the help and support! F > On Aug 21, 2014, at 7:58 PM, Mike Dubman wrote: > > Hi FIlippo, > > I think you can use SLURM_LOCALID v

Re: [OMPI users] Clarification about OpenMPI, slurm and PMI interface

2014-08-21 Thread Mike Dubman
Hi FIlippo, I think you can use SLURM_LOCALID var (at least with slurm v14.03.4-2) $srun -N2 --ntasks-per-node 3 env |grep SLURM_LOCALID SLURM_LOCALID=1 SLURM_LOCALID=2 SLURM_LOCALID=0 SLURM_LOCALID=0 SLURM_LOCALID=1 SLURM_LOCALID=2 $ Kind Regards, M On Thu, Aug 21, 2014 at 9:27 PM, Ralph Cas

Re: [OMPI users] Clarification about OpenMPI, slurm and PMI interface

2014-08-21 Thread Ralph Castain
On Aug 21, 2014, at 10:58 AM, Filippo Spiga wrote: > Dear Ralph > > On Aug 21, 2014, at 2:30 PM, Ralph Castain wrote: >> I'm afraid that none of the mapping or binding options would be available >> under srun as those only work via mpirun. You can pass MCA params in the >> environment of cou

Re: [OMPI users] Clarification about OpenMPI, slurm and PMI interface

2014-08-21 Thread Filippo Spiga
Dear Ralph On Aug 21, 2014, at 2:30 PM, Ralph Castain wrote: > I'm afraid that none of the mapping or binding options would be available > under srun as those only work via mpirun. You can pass MCA params in the > environment of course, or in default MCA param files. I understand. I hopefully

Re: [OMPI users] Clarification about OpenMPI, slurm and PMI interface

2014-08-21 Thread Ralph Castain
On Aug 20, 2014, at 11:46 PM, Filippo Spiga wrote: > Hi Joshua, > > On Aug 21, 2014, at 12:28 AM, Joshua Ladd wrote: >> When launching with mpirun in a SLURM environment, srun is only being used >> to launch the ORTE daemons (orteds.) Since the daemon will already exist on >> the node from

Re: [OMPI users] Clarification about OpenMPI, slurm and PMI interface

2014-08-21 Thread Filippo Spiga
Hi Joshua, On Aug 21, 2014, at 12:28 AM, Joshua Ladd wrote: > When launching with mpirun in a SLURM environment, srun is only being used to > launch the ORTE daemons (orteds.) Since the daemon will already exist on the > node from which you invoked mpirun, this node will not be included in the

Re: [OMPI users] Clarification about OpenMPI, slurm and PMI interface

2014-08-20 Thread Ralph Castain
Or you can add -nolocal|--nolocalDo not run any MPI applications on the local node to your mpirun command line and we won't run any application procs on the node where mpirun is executing On Aug 20, 2014, at 4:28 PM, Joshua Ladd wrote: > Hi, Filippo > > When launching with mpirun in

Re: [OMPI users] Clarification about OpenMPI, slurm and PMI interface

2014-08-20 Thread Joshua Ladd
Hi, Filippo When launching with mpirun in a SLURM environment, srun is only being used to launch the ORTE daemons (orteds.) Since the daemon will already exist on the node from which you invoked mpirun, this node will not be included in the list of nodes. SLURM's PMI library is not involved (that

[OMPI users] Clarification about OpenMPI, slurm and PMI interface

2014-08-20 Thread Filippo Spiga
Dear Open MPI experts, I have a problem that is related to the integration of OpenMPI, slurm and PMI interface. I spent some time today with a colleague of mine trying to figure out why we were not able to obtain all H5 profile files (generated by acct_gather_profile) using Open MPI. When I say