On Nov 28, 2011, at 6:56 PM, Ralph Castain wrote: > I'm afraid that example is incorrect - you were running under slurm on your > cluster, not rsh.
Ummm... right. Duh. > If you look at the actual code, you will see that we slurp up the envars into > the environment of each app_context, and then send that to the backend. Ah, right -- here's running under rsh (SVN trunk): ----- [16:26] svbu-mpi:~ % cat run #!/bin/csh -f echo on `hostname`, foo is: $OMPI_MCA_foo exit 0 [16:26] svbu-mpi:~ % mpirun -np 2 --host svbu-mpi043,svbu-mpi044 run OMPI_MCA_foo: Undefined variable. OMPI_MCA_foo: Undefined variable. ------------------------------------------------------- While the primary job terminated normally, 2 processes returned non-zero exit codes.. Further examination may be required. ------------------------------------------------------- [16:26] svbu-mpi:~ % setenv OMPI_MCA_foo bar [16:27] svbu-mpi:~ % mpirun -np 2 --host svbu-mpi043,svbu-mpi044 run on svbu-mpi044, foo is: bar on svbu-mpi043, foo is: bar [16:27] svbu-mpi:~ % ----- (the "MCA_" here is superfluous -- I looked at the code and the man page that Paul cited, and see that we grab all env vars OMPI_*, not OMPI_MCA_*). > In environments like slurm, we can also apply those envars to the launch of > the orteds as we pass the env to the API that launches the orteds. You cannot > do that with rsh, as you know. Right-o. Knew there was something I forgot... > So on rsh, we do not put envar mca params onto the orted cmd line. This has > been noted repeatedly on the user and devel lists, so it really has always > been the case. So they're sent as part of the launch command (i.e., out of band -- not on the rsh/ssh command line), right? Meaning that per my output from above, what Paul was trying should have worked, no? I.e., setenv'ing OMPI_<whatever>, and those env vars should magically show up in the launched process. >>>> [after performing some RTFM...] >>>> at least the man page of mpiexec says, the OMPI_ environment variables are >>>> always provided and thus treated *differently* than other envvars: >>>> >>>> $ man mpiexec >>>> .... >>>> Exported Environment Variables >>>> All environment variables that are named in the form OMPI_* will >>>> automatically be exported to new processes on the local and remote nodes. >>>> >>>> So, tells the man page lies, or this is an removed feature, or something >>>> else? -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
