Sorry folks - for some reason (probably timing for getting 2.1.0 out), the fix for this got pushed to v2.1.1 - see the PR here: https://github.com/open-mpi/ompi/pull/3163 <https://github.com/open-mpi/ompi/pull/3163>
> On Mar 22, 2017, at 7:49 AM, Reuti <re...@staff.uni-marburg.de> wrote: > >> >> Am 22.03.2017 um 15:31 schrieb Heinz-Ado Arnolds >> <arno...@mpa-garching.mpg.de <mailto:arno...@mpa-garching.mpg.de>>: >> >> Dear Reuti, >> >> thanks a lot, you're right! But why did the default behavior change but not >> the value of this parameter: >> >> 2.1.0: MCA plm rsh: parameter "plm_rsh_agent" (current value: "ssh : rsh", >> data source: default, level: 2 user/detail, type: string, synonyms: >> pls_rsh_agent, orte_rsh_agent) >> The command used to launch executables on remote >> nodes (typically either "ssh" or "rsh") >> >> 1.10.6: MCA plm: parameter "plm_rsh_agent" (current value: "ssh : rsh", >> data source: default, level: 2 user/detail, type: string, synonyms: >> pls_rsh_agent, orte_rsh_agent) >> The command used to launch executables on remote >> nodes (typically either "ssh" or "rsh") >> >> That means there must have been changes in the code regarding that, perhaps >> for detecting SGE? Do you know of a way to revert to the old style (e.g. >> configure option)? Otherwise all my users have to add this option. > > There was a discussion in https://github.com/open-mpi/ompi/issues/2947 > <https://github.com/open-mpi/ompi/issues/2947> > > For now you can make use of > https://www.open-mpi.org/faq/?category=tuning#setting-mca-params > <https://www.open-mpi.org/faq/?category=tuning#setting-mca-params> > > Essentially to have it set for all users automatically, put: > > plm_rsh_agent=foo > > in $prefix/etc/openmpi-mca-params.conf of your central Open MPI 2.1.0 > installation. > > -- Reuti > > >> Thanks again, and have a nice day >> >> Ado Arnolds >> >> On 22.03.2017 13:58, Reuti wrote: >>> Hi, >>> >>>> Am 22.03.2017 um 10:44 schrieb Heinz-Ado Arnolds >>>> <arno...@mpa-garching.mpg.de>: >>>> >>>> Dear users and developers, >>>> >>>> first of all many thanks for all the great work you have done for OpenMPI! >>>> >>>> Up to OpenMPI-1.10.6 the mechanism for starting orted was to use SGE/qrsh: >>>> mpirun -np 8 --map-by ppr:4:node ./myid >>>> /opt/sge-8.1.8/bin/lx-amd64/qrsh -inherit -nostdin -V <DNS-Name of Remote >>>> Machine> orted --hnp-topo-sig 2N:2S:2L3:20L2:20L1:20C:40H:x86_64 -mca ess >>>> "env" -mca orte_ess_jobid "1621884928" -mca orte_ess_vpid 1 -mca >>>> orte_ess_num_procs "2" -mca orte_hnp_uri "1621884928.0;tcp://<IP-addr of >>>> Master>:41031" -mca plm "rsh" -mca rmaps_base_mapping_policy "ppr:4:node" >>>> --tree-spawn >>>> >>>> Now with OpenMPI-2.1.0 (and the release candidates) "ssh" is used to start >>>> orted: >>>> mpirun -np 8 --map-by ppr:4:node -mca mca_base_env_list OMP_NUM_THREADS=5 >>>> ./myid >>>> /usr/bin/ssh -x <DNS-Name of Remote Machine> >>>> PATH=/afs/...../openmpi-2.1.0/bin:$PATH ; export PATH ; >>>> LD_LIBRARY_PATH=/afs/...../openmpi-2.1.0/lib:$LD_LIBRARY_PATH ; export >>>> LD_LIBRARY_PATH ; >>>> DYLD_LIBRARY_PATH=/afs/...../openmpi-2.1.0/lib:$DYLD_LIBRARY_PATH ; export >>>> DYLD_LIBRARY_PATH ; /afs/...../openmpi-2.1.0/bin/orted --hnp-topo-sig >>>> 2N:2S:2L3:20L2:20L1:20C:40H:x86_64 -mca ess "env" -mca ess_base_jobid >>>> "1626013696" -mca ess_base_vpid 1 -mca ess_base_num_procs "2" -mca >>>> orte_hnp_uri "1626013696.0;usock;tcp://<IP-addr of Master>:43019" -mca >>>> plm_rsh_args "-x" -mca plm "rsh" -mca rmaps_base_mapping_policy >>>> "ppr:4:node" -mca pmix "^s1,s2,cray" >>>> >>>> qrsh set the environment properly on the remote side, so that environment >>>> variables from job scripts are properly transferred. With the ssh variant >>>> the environment is not set properly on the remote side, and it seems that >>>> there are handling problems with Kerberos tickets and/or AFS tokens. >>>> >>>> Is there any way to revert the 2.1.0 behavior to the 1.10.6 (use SGE/qrsh) >>>> one? Are there mca params to set this? >>>> >>>> If you need more info, please let me know. (Job submitting machine and >>>> target cluster are the same with all tests. SW is residing in AFS >>>> directories visible on all machines. Parameter "plm_rsh_disable_qrsh" >>>> current value: "false") >>> >>> It looks like `mpirun` still needs: >>> >>> -mca plm_rsh_agent foo >>> >>> to allow SGE to be detected. >>> >>> -- Reuti >>> >>> >>> >>> _______________________________________________ >>> users mailing list >>> users@lists.open-mpi.org >>> https://rfd.newmexicoconsortium.org/mailman/listinfo/users >>> >> _______________________________________________ >> users mailing list >> users@lists.open-mpi.org >> https://rfd.newmexicoconsortium.org/mailman/listinfo/users > > _______________________________________________ > users mailing list > users@lists.open-mpi.org <mailto:users@lists.open-mpi.org> > https://rfd.newmexicoconsortium.org/mailman/listinfo/users > <https://rfd.newmexicoconsortium.org/mailman/listinfo/users>
_______________________________________________ users mailing list users@lists.open-mpi.org https://rfd.newmexicoconsortium.org/mailman/listinfo/users
