Am 15.09.2011 um 16:31 schrieb Blosch, Edwin L:
> I think one has to quote everything after the gid, sg 25000 "ls -l" should
> work.
>
> I tried something similar to the orted wrapper but also couldn't get it to
> work, but I didn't have time yet to dig into it. Might be worth trying it
> with both quotes:
> #!/bin/sh
> exec sg 25000 "orted ${@}"
Thx for the reply.
It looks like it depends on the Version of `sg`. On Scientific Linux I *must*
use the group name, no go with the group id:
$ sg 537 ls
unknown group: 537
On openSUSE you can use either of them.
The version on Scientific Linux accepts any number of arguments, but all except
the first will be ignored:
$ sg my_group "echo" "1" "2" "3"
(Yes, an empty line only) while on openSUSE:
$ sg 24000 "echo" "1" "2" "3"
$ sg: Too many arguments.
Try `sg --help' or `sg --usage' for more information.
==
This is now going away from Open MPI but anyway:
What is indeed working is: sg 25000 "ls -l"
But it still fails for:
$ sg 24000 "./tt.sh ${@}"
sg: Too many arguments.
I works again if I use * in place of a @, but this will change how the bash
will assemble the arguments:
$ set "11 12" "21 22"
$ echo $1
11 12
$ echo $2
21 22
$ sg 24000 "./tt.sh ${*}"
12
11
What I expect is:
$ ./tt.sh "${@}"
21 22
11 12
While the script tt.sh lists just:
#!/bin/sh
echo $2
echo $1
-- Reuti
> Also, Ralph,
>
> I think I skipped one step of logic in asking you yesterday to consider
> adding the capability to pass a multi-word argument to orte_launch_agent.
> This itself seems like it could have uses, so I like the idea. But what I'd
> like to have, in the end, is the possibility to launch all processes with the
> same group or gid (for the situation where all hosts share the same
> passwd/group database), without relying on any launcher other than rsh/ssh.
> I think it should be possible. The best way to do this might not be the idea
> of using sg, so perhaps this should be discussed first before you spend time
> adding multi-word support for the orte_launch_agent MCA parameter. But if
> you do it I will certainly test it.
>
> Thanks again
>
>
> Ed
>
> -----Original Message-----
> From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On
> Behalf Of Reuti
> Sent: Thursday, September 15, 2011 4:37 AM
> To: Open MPI Users
> Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the processes
> created by mpirun?
>
> Am 15.09.2011 um 01:15 schrieb Blosch, Edwin L:
>
>> I would appreciate trying to fix the multi-word argument to
>> orte_launch_agent, unless you've already fixed it in a later series in which
>> case I could update. My workaround with the setuid applied to a copied
>> executable doesn't work for other applications that run on our clusters ( I
>> mean their job scripts would also have to be modified similarly).
>
> I played around with `sg` and it looks like having some pitfalls:
>
> $ sg 25000 ls -l
>
> The -l will go to `sg`, even double quotes fo " and ' don't help and there is
> no --:
>
> $ sg 2600 ls "'-l'"
>
> gives an error too.
>
> (I was about to suggest an orted wrapper doing:
>
> #!/bin/sh
> exec sg 25000 orted ${@}"
>
> but it failed.)
>
>
>> I think OpenMPI should be able to launch all processes with the same uid and
>> gid even when the user has launched the job while using one of his secondary
>> groups. Whether it should be a change request or bug (unexpected,
>> counter-intuitive), I'll leave that to your judgment.
>
> Yes, but for now the "username" is used, not the "uid" - right? So it could
> have a different uid/gid on the machines, but with the new feature they must
> be the same. Okay, in a cluster they are most likely unique across all
> machines anyway. But just to note as a side effect.
>
> -- Reuti
>
>
>> Thank you again for the support
>>
>>
>>
>> From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On
>> Behalf Of Ralph Castain
>> Sent: Wednesday, September 14, 2011 5:10 PM
>> To: Open MPI Users
>> Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the processes
>> created by mpirun?
>>
>> Hi Ed
>>
>> I finally had some time to dig around a little. I believe we could make this
>> work if I fix the multi-word cmd line param issue. Either "newgrp" or "sg"
>> should resolve the problem - both are user-level cmds.
>>
>> Up to you - let me know if you want me to pursue this.
>> Ralph
>>
>>
>> On Sep 14, 2011, at 3:31 PM, Blosch, Edwin L wrote:
>>
>>
>> Thank you - I did pursue this kind of workaround, and it worked, but you'll
>> be happy to know that nothing had to be owned by root.
>>
>> ASIDE ----
>> Just to remind: The job script is a shell script that invokes mpirun; the
>> job script itself is run with as the correct user, but the group id may be
>> changed to whatever the user requests of the job-scheduling system. I think
>> it may not be uncommon to have jobs that request a specific Unix group, for
>> many reasons, but in our case the group is an input for the scheduler's
>> prioritization policy.
>>
>> Outcome: rank 0 runs user:group2 as the user requested, but the launched
>> child processes run user:group1 where group 1 is the user's primary group.
>> The peculiarity of this application is that each of the processes writes a
>> file to disk, so the resulting group ownership of rank 0 files is group2,
>> but the group ownership of all other ranks' files is group1. That was the
>> original problem I'm trying to work around.
>> --- END ASIDE
>>
>> Fortunately for me, there is another peculiarity of this application -- the
>> executable gets copied out to /tmp (local space) on each of the hosts to be
>> used in the job. We found this helped prevent some crashes during test
>> phases where an executable gets overwritten while in use. Definitely a
>> special behavior. But as a result of this peculiarity, the mpirun command
>> ends up launching the copied executable, and I took advantage of that.
>>
>> I had the job script do chown user:group2 on the copied executables and then
>> chmod 6711, and then I observed that the child processes ran as user:group2,
>> same as the rank 0 process, so the files they created had the desired group
>> ownership.
>>
>> I will explore Reuti's guidance as well.
>>
>> Thank you
>>
>>
>> From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On
>> Behalf Of Randall Svancara
>> Sent: Wednesday, September 14, 2011 3:07 PM
>> To: Open MPI Users
>> Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the processes
>> created by mpirun?
>>
>> You could set the setuid bit on the application and chown it to root?? It
>> is about as secure as anything else that has been described thus far. As a
>> system admin, I cringe at the thought of anything that would allow something
>> to run as someone else, so there would have to be a pretty good
>> justification for such unique use case as yours.
>>
>> Randall
>>
>> On Wed, Sep 14, 2011 at 12:56 PM, Reuti <re...@staff.uni-marburg.de> wrote:
>> Am 14.09.2011 um 19:02 schrieb Blosch, Edwin L:
>>
>>> Thanks for trying.
>>>
>>> Do you feel that this is an impossible request without the assistance of
>>> some process running as root, for example, as Reuti mentioned, the daemons
>>> of a job scheduler? Or are you saying it will just not be as
>>> straightforward as calling setgid as you had hoped?
>>>
>>> Also, do you think there is a way I could make use of the sg command below?
>>> Perhaps there is a way to have the rsh/ssh launcher start the application
>>> processes with a command like 'sg <group> <executable name>'?
>>
>> What about a half-tight integration (or call it: classic tight integration),
>> i.e. no recompilation necessary?
>>
>> - setup your mpiexec call in the jobscript to use a plain rsh for the remote
>> startup (no path given): -mca plm_rsh_agent rsh
>>
>> - the PE of SGE needs the argument -catch_rsh in start_proc_args and the
>> supplied script in $SGE_ROOT/mpi/startmpi.sh
>>
>> (SGE will create a symbolic link in $TMPDIR therein [which will be called
>> first this way] to the rsh-wrapper in $SGE_ROOT/mpi [pitfall: some
>> applications need a -V to be added in the lines woth "qrsh", i.e. "qrsh
>> -inherit -V ..." to send all environment variables to the slaves])
>>
>> - what is your setting of qrsh_daemon/qrsh_command in `qconf -sconf`? This
>> will then be used finally to reach the node and should be builtin or point
>> to the SGE supplied rsh/rshd (no rshd necessary to install, no rshd is
>> running all the time, no rshd will be started by xinet.d or alike)
>>
>> - like you do already: switch off the built-in SGE starter in your mpiexec
>> call: -mca plm_rsh_disable_qrsh 1
>>
>> -- Reuti
>>
>> PS: To avoid misunderstandings: you could also set "-mca plm_rsh_agent
>> foobar" and in $SGE_ROOT/mpi/startmpi.sh you change it to create a symbolic
>> link called "foobar " in $TMPDIR. It's just a name at this stage of startup.
>>
>>
>>> Ed
>>>
>>>
>>> NAME
>>> sg - execute command as different group ID
>>>
>>> SYNOPSIS
>>> sg [-] [group [-c ] command]
>>>
>>> DESCRIPTION
>>> The sg command works similar to newgrp but accepts a command. The
>>> command will be executed with the /bin/sh shell. With most shells you
>>> may run sg from, you need to enclose multi-word commands in quotes.
>>> Another difference between newgrp and sg is that some shells treat
>>> newgrp specially, replacing themselves with a new instance of a shell
>>> that newgrp creates. This doesn't happen with sg, so upon exit from a
>>> sg command you are returned to your previous group ID.
>>>
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On
>>> Behalf Of Ralph Castain
>>> Sent: Wednesday, September 14, 2011 11:33 AM
>>> To: Open MPI Users
>>> Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the
>>> processes created by mpirun?
>>>
>>>
>>> On Sep 14, 2011, at 9:39 AM, Blosch, Edwin L wrote:
>>>
>>>> Thanks, Ralph,
>>>>
>>>> I get the failure messages, unfortunately:
>>>>
>>>> setgid FAILED
>>>> setgid FAILED
>>>> setgid FAILED
>>>>
>>>> I actually had attempted to call setgid from within the application
>>>> previously, which looks similar to what you've done, but it failed. That
>>>> was when I initiated the post to the mailing list. My conclusion, a guess
>>>> really, was that Linux would not let me setgid from within my program
>>>> because I was not root.
>>>
>>> I was afraid of that - the documentation seemed to indicate that would be
>>> the case, but I figured it was worth a quick try. Sorry I can't be of help.
>>>
>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: users-boun...@open-mpi.org [mailto:users-boun...@open-mpi.org] On
>>>> Behalf Of Ralph Castain
>>>> Sent: Wednesday, September 14, 2011 8:15 AM
>>>> To: Open MPI Users
>>>> Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the
>>>> processes created by mpirun?
>>>>
>>>> The attached should set the gid of the remote daemons (and their children)
>>>> to the gid of mpirun. No cmd line option or anything is required - it will
>>>> just always do it.
>>>>
>>>> Would you mind giving it a try?
>>>>
>>>> Please let me know if/how it works.
>>>>
>>>> _______________________________________________
>>>> users mailing list
>>>> us...@open-mpi.org
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> us...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> _______________________________________________
>>> users mailing list
>>> us...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>> _______________________________________________
>> users mailing list
>> us...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>>
>> --
>> Randall Svancara
>> http://knowyourlinux.com/
>> _______________________________________________
>> users mailing list
>> us...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>> _______________________________________________
>> users mailing list
>> us...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
