I think one has to quote everything after the gid, sg 25000 "ls -l" should work.
I tried something similar to the orted wrapper but also couldn't get it to
work, but I didn't have time yet to dig into it. Might be worth trying it with
both quotes:
#!/bin/sh
exec sg 25000 "orted ${@}"
Also, Ralph,
I think I skipped one step of logic in asking you yesterday to consider adding
the capability to pass a multi-word argument to orte_launch_agent. This itself
seems like it could have uses, so I like the idea. But what I'd like to have,
in the end, is the possibility to launch all processes with the same group or
gid (for the situation where all hosts share the same passwd/group database),
without relying on any launcher other than rsh/ssh. I think it should be
possible. The best way to do this might not be the idea of using sg, so
perhaps this should be discussed first before you spend time adding multi-word
support for the orte_launch_agent MCA parameter. But if you do it I will
certainly test it.
Thanks again
Ed
-----Original Message-----
From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On Behalf
Of Reuti
Sent: Thursday, September 15, 2011 4:37 AM
To: Open MPI Users
Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the processes
created by mpirun?
Am 15.09.2011 um 01:15 schrieb Blosch, Edwin L:
> I would appreciate trying to fix the multi-word argument to
> orte_launch_agent, unless you've already fixed it in a later series in which
> case I could update. My workaround with the setuid applied to a copied
> executable doesn't work for other applications that run on our clusters ( I
> mean their job scripts would also have to be modified similarly).
I played around with `sg` and it looks like having some pitfalls:
$ sg 25000 ls -l
The -l will go to `sg`, even double quotes fo " and ' don't help and there is
no --:
$ sg 2600 ls "'-l'"
gives an error too.
(I was about to suggest an orted wrapper doing:
#!/bin/sh
exec sg 25000 orted ${@}"
but it failed.)
> I think OpenMPI should be able to launch all processes with the same uid and
> gid even when the user has launched the job while using one of his secondary
> groups. Whether it should be a change request or bug (unexpected,
> counter-intuitive), I'll leave that to your judgment.
Yes, but for now the "username" is used, not the "uid" - right? So it could
have a different uid/gid on the machines, but with the new feature they must be
the same. Okay, in a cluster they are most likely unique across all machines
anyway. But just to note as a side effect.
-- Reuti
> Thank you again for the support
>
>
>
> From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On
> Behalf Of Ralph Castain
> Sent: Wednesday, September 14, 2011 5:10 PM
> To: Open MPI Users
> Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the processes
> created by mpirun?
>
> Hi Ed
>
> I finally had some time to dig around a little. I believe we could make this
> work if I fix the multi-word cmd line param issue. Either "newgrp" or "sg"
> should resolve the problem - both are user-level cmds.
>
> Up to you - let me know if you want me to pursue this.
> Ralph
>
>
> On Sep 14, 2011, at 3:31 PM, Blosch, Edwin L wrote:
>
>
> Thank you - I did pursue this kind of workaround, and it worked, but you'll
> be happy to know that nothing had to be owned by root.
>
> ASIDE ----
> Just to remind: The job script is a shell script that invokes mpirun; the
> job script itself is run with as the correct user, but the group id may be
> changed to whatever the user requests of the job-scheduling system. I think
> it may not be uncommon to have jobs that request a specific Unix group, for
> many reasons, but in our case the group is an input for the scheduler's
> prioritization policy.
>
> Outcome: rank 0 runs user:group2 as the user requested, but the launched
> child processes run user:group1 where group 1 is the user's primary group.
> The peculiarity of this application is that each of the processes writes a
> file to disk, so the resulting group ownership of rank 0 files is group2, but
> the group ownership of all other ranks' files is group1. That was the
> original problem I'm trying to work around.
> --- END ASIDE
>
> Fortunately for me, there is another peculiarity of this application -- the
> executable gets copied out to /tmp (local space) on each of the hosts to be
> used in the job. We found this helped prevent some crashes during test
> phases where an executable gets overwritten while in use. Definitely a
> special behavior. But as a result of this peculiarity, the mpirun command
> ends up launching the copied executable, and I took advantage of that.
>
> I had the job script do chown user:group2 on the copied executables and then
> chmod 6711, and then I observed that the child processes ran as user:group2,
> same as the rank 0 process, so the files they created had the desired group
> ownership.
>
> I will explore Reuti's guidance as well.
>
> Thank you
>
>
> From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On
> Behalf Of Randall Svancara
> Sent: Wednesday, September 14, 2011 3:07 PM
> To: Open MPI Users
> Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the processes
> created by mpirun?
>
> You could set the setuid bit on the application and chown it to root?? It is
> about as secure as anything else that has been described thus far. As a
> system admin, I cringe at the thought of anything that would allow something
> to run as someone else, so there would have to be a pretty good
> justification for such unique use case as yours.
>
> Randall
>
> On Wed, Sep 14, 2011 at 12:56 PM, Reuti <re...@staff.uni-marburg.de> wrote:
> Am 14.09.2011 um 19:02 schrieb Blosch, Edwin L:
>
> > Thanks for trying.
> >
> > Do you feel that this is an impossible request without the assistance of
> > some process running as root, for example, as Reuti mentioned, the daemons
> > of a job scheduler? Or are you saying it will just not be as
> > straightforward as calling setgid as you had hoped?
> >
> > Also, do you think there is a way I could make use of the sg command below?
> > Perhaps there is a way to have the rsh/ssh launcher start the application
> > processes with a command like 'sg <group> <executable name>'?
>
> What about a half-tight integration (or call it: classic tight integration),
> i.e. no recompilation necessary?
>
> - setup your mpiexec call in the jobscript to use a plain rsh for the remote
> startup (no path given): -mca plm_rsh_agent rsh
>
> - the PE of SGE needs the argument -catch_rsh in start_proc_args and the
> supplied script in $SGE_ROOT/mpi/startmpi.sh
>
> (SGE will create a symbolic link in $TMPDIR therein [which will be called
> first this way] to the rsh-wrapper in $SGE_ROOT/mpi [pitfall: some
> applications need a -V to be added in the lines woth "qrsh", i.e. "qrsh
> -inherit -V ..." to send all environment variables to the slaves])
>
> - what is your setting of qrsh_daemon/qrsh_command in `qconf -sconf`? This
> will then be used finally to reach the node and should be builtin or point to
> the SGE supplied rsh/rshd (no rshd necessary to install, no rshd is running
> all the time, no rshd will be started by xinet.d or alike)
>
> - like you do already: switch off the built-in SGE starter in your mpiexec
> call: -mca plm_rsh_disable_qrsh 1
>
> -- Reuti
>
> PS: To avoid misunderstandings: you could also set "-mca plm_rsh_agent
> foobar" and in $SGE_ROOT/mpi/startmpi.sh you change it to create a symbolic
> link called "foobar " in $TMPDIR. It's just a name at this stage of startup.
>
>
> > Ed
> >
> >
> > NAME
> > sg - execute command as different group ID
> >
> > SYNOPSIS
> > sg [-] [group [-c ] command]
> >
> > DESCRIPTION
> > The sg command works similar to newgrp but accepts a command. The
> > command will be executed with the /bin/sh shell. With most shells you
> > may run sg from, you need to enclose multi-word commands in quotes.
> > Another difference between newgrp and sg is that some shells treat
> > newgrp specially, replacing themselves with a new instance of a shell
> > that newgrp creates. This doesn't happen with sg, so upon exit from a
> > sg command you are returned to your previous group ID.
> >
> >
> >
> >
> > -----Original Message-----
> > From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On
> > Behalf Of Ralph Castain
> > Sent: Wednesday, September 14, 2011 11:33 AM
> > To: Open MPI Users
> > Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the
> > processes created by mpirun?
> >
> >
> > On Sep 14, 2011, at 9:39 AM, Blosch, Edwin L wrote:
> >
> >> Thanks, Ralph,
> >>
> >> I get the failure messages, unfortunately:
> >>
> >> setgid FAILED
> >> setgid FAILED
> >> setgid FAILED
> >>
> >> I actually had attempted to call setgid from within the application
> >> previously, which looks similar to what you've done, but it failed. That
> >> was when I initiated the post to the mailing list. My conclusion, a guess
> >> really, was that Linux would not let me setgid from within my program
> >> because I was not root.
> >
> > I was afraid of that - the documentation seemed to indicate that would be
> > the case, but I figured it was worth a quick try. Sorry I can't be of help.
> >
> >
> >>
> >>
> >> -----Original Message-----
> >> From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On
> >> Behalf Of Ralph Castain
> >> Sent: Wednesday, September 14, 2011 8:15 AM
> >> To: Open MPI Users
> >> Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the
> >> processes created by mpirun?
> >>
> >> The attached should set the gid of the remote daemons (and their children)
> >> to the gid of mpirun. No cmd line option or anything is required - it will
> >> just always do it.
> >>
> >> Would you mind giving it a try?
> >>
> >> Please let me know if/how it works.
> >>
> >> _______________________________________________
> >> users mailing list
> >> us...@open-mpi.org
> >> http://www.open-mpi.org/mailman/listinfo.cgi/users
> >
> >
> > _______________________________________________
> > users mailing list
> > us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
> > _______________________________________________
> > users mailing list
> > us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
>
> --
> Randall Svancara
> http://knowyourlinux.com/
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
