Re: [OMPI users] SLURM environment variables at runtime

2011-02-24 Thread Ralph Castain
I guess I wasn't clear earlier - I don't know anything about how HP-MPI works. I was only theorizing that perhaps they did something different that results in some other slurm vars showing up in Brent's tests. From Brent's comments, I guess they don't - but they launch jobs in a different manner th

Re: [OMPI users] SLURM environment variables at runtime

2011-02-24 Thread Jeff Squyres
On Feb 24, 2011, at 2:59 PM, Henderson, Brent wrote: > [snip] > They really can't be all SLURM_PROCID=0 - that is supposed to be unique for > the job - right? It appears that the SLURM_PROCID is inherited from the > orted parent - which makes a fair amount of sense given how things are > launc

Re: [OMPI users] SLURM environment variables at runtime

2011-02-24 Thread Henderson, Brent
> -Original Message- > From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On > Behalf Of Jeff Squyres > Sent: Thursday, February 24, 2011 10:20 AM > To: Open MPI Users > Subject: Re: [OMPI users] SLURM environment variables at runtime > > On F

Re: [OMPI users] SLURM environment variables at runtime

2011-02-24 Thread Jeff Squyres
On Feb 24, 2011, at 11:15 AM, Henderson, Brent wrote: > Note that the parent of the sleep processes is orted and that orted was > started by slurmstepd. Unless orted is updating the slurm variables for the > children (which is doubtful) then they will not contain the specific settings > that I

Re: [OMPI users] SLURM environment variables at runtime

2011-02-24 Thread Jeff Squyres
ORT=33347 >> SLURM_SRUN_COMM_PORT=33347 >> SLURM_STEP_ID=8 >> SLURM_STEP_ID=8 >> SLURM_STEPID=8 >> SLURM_STEPID=8 >> SLURM_STEP_LAUNCHER_PORT=33347 >> SLURM_STEP_LAUNCHER_PORT=33347 > 59,60c61,62 > < SLURM_TASK_PID=1381 > < SLURM_TASK_PID

Re: [OMPI users] SLURM environment variables at runtime

2011-02-24 Thread Henderson, Brent
-mpi.org] On Behalf Of Ralph Castain Sent: Thursday, February 24, 2011 10:05 AM To: Open MPI Users Subject: Re: [OMPI users] SLURM environment variables at runtime I would talk to the slurm folks about it - I don't know anything about the internals of HP-MPI, but I do know the relevant

Re: [OMPI users] SLURM environment variables at runtime

2011-02-24 Thread Ralph Castain
7 > > SLURM_STEP_ID=8 > > SLURM_STEP_ID=8 > > SLURM_STEPID=8 > > SLURM_STEPID=8 > > SLURM_STEP_LAUNCHER_PORT=33347 > > SLURM_STEP_LAUNCHER_PORT=33347 > 59,60c61,62 > < SLURM_TASK_PID=1381 > < SLURM_TASK_PID=2288 > --- > > SLURM_TASK_PID=1592 &g

Re: [OMPI users] SLURM environment variables at runtime

2011-02-24 Thread Henderson, Brent
2 mpi]$ [brent@node2 mpi]$ [brent@node2 mpi]$ grep SLURM_PROCID srun.out SLURM_PROCID=0 SLURM_PROCID=1 [brent@node2 mpi]$ grep SLURM_PROCID mpirun.out SLURM_PROCID=0 [brent@node2 mpi]$ grep SLURM_PROCID hpmpi.out SLURM_PROCID=0 SLURM_PROCID=1 [brent@node2 mpi]$ > -Original Message- > F

Re: [OMPI users] SLURM environment variables at runtime

2011-02-24 Thread Ralph Castain
> > > SLURM_LOCALID=0 > > > SLURM_LOCALID=1 > > > SLURM_LOCALID=1 > > > SLURM_NODEID=0 > > > SLURM_NODEID=0 > > > SLURM_NODEID=1 > > > SLURM_NODEID=1 > > > SLURM_PROCID=0 > > > SLURM_PROCID=1 > > > SLUR

Re: [OMPI users] SLURM environment variables at runtime

2011-02-24 Thread Jeff Squyres
ODEID=0 > > SLURM_NODEID=0 > > SLURM_NODEID=1 > > SLURM_NODEID=1 > > SLURM_PROCID=0 > > SLURM_PROCID=1 > > SLURM_PROCID=2 > > SLURM_PROCID=3 > > [brent@node1 mpi]$ > > > > Since srun is not supported currently by OpenMPI, I have to use sallo

Re: [OMPI users] SLURM environment variables at runtime

2011-02-24 Thread Ralph Castain
t; > > Since srun is not supported currently by OpenMPI, I have to use salloc – > right? In this case, it is up to OpenMPI to interpret the SLURM environment > variables it sees in the one process that is launched and ‘do the right > thing’ – whatever that means in this case. How does Op

Re: [OMPI users] SLURM environment variables at runtime

2011-02-24 Thread Jeff Squyres
hostfile and launch as you would outside SLURM, …) This may be the > difference between HP-MPI and OpenMPI. > > Thanks, > > Brent > > > From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On > Behalf Of Ralph Castain > Sent: Wednesday, February

Re: [OMPI users] SLURM environment variables at runtime

2011-02-23 Thread Ralph Castain
Behalf Of > Ralph Castain > Sent: Wednesday, February 23, 2011 10:07 AM > To: Open MPI Users > Subject: Re: [OMPI users] SLURM environment variables at runtime Resource > managers generally frown on the idea of any program passing RM-managed envars > from one node to another,

Re: [OMPI users] SLURM environment variables at runtime

2011-02-23 Thread Henderson, Brent
en-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf Of Ralph Castain Sent: Wednesday, February 23, 2011 10:07 AM To: Open MPI Users Subject: Re: [OMPI users] SLURM environment variables at runtime Resource managers generally frown on the idea of any program passing RM-managed envars from one node

Re: [OMPI users] SLURM environment variables at runtime

2011-02-23 Thread Ralph Castain
Resource managers generally frown on the idea of any program passing RM-managed envars from one node to another, and this is certainly true of slurm. The reason is that the RM reserves those values for its own use when managing remote nodes. For example, if you got an allocation and then used mpiru

[OMPI users] SLURM environment variables at runtime

2011-02-23 Thread Henderson, Brent
Hi Everyone, I have an OpenMPI/SLURM specific question, I'm using MPI as a launcher for another application I'm working on and it is dependent on the SLURM environment variables making their way into the a.out's environment. This works as I need if I use HP-MPI/PMPI, but when I use OpenMPI, it