Hello All,
Last week I posted a question about a potential '--execute' option . Dave
Dykstra replied and pointed me to a 2 year old thread on the topic. I would
like to revive the discussion and add my own comments on the current rsync
security model, user authentication and finally, implementation of the
--execute option.
I have reproduced the entirety of that dialogue up to the present for
reference, since it is two years old. My commentary will follow that;
Feb. 07, 2001:
Henry: My problem is that I often want to execute post-distribution scripts
on the endpoints (daemons) such as 'bounce_the_server'. This is similar to
Interwoven's OpenDeploy feature "deploy_and_run". It seems trivial to do
this when running rsync over rsh or ssh since we already have .rhosts trust
to run rsync itself and thus can run other rsh commands. But when running
in client-server mode there would need to be a '--execute=/path/to/doit.sh'
Option to do this.
Has anybody thought about or working on this type of functionality?
Dave: Yes, it has been thought about, and I think it is a good idea, but
nobody
has done anything about it. I found an archive of a discussion from two
years ago in the rsync bugs tracker
Following is the two year old thread between the following players:
Marc Evans [EMAIL PROTECTED]
Dave Dykstra [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
Andrew Tridgell [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
Marc: I would like to see the rsyncd.conf(5) file extended to support the
concept of running an arbitrary program after a successful (or failed?)
transfer takes place. For example, I would like to have the program
/usr/local/sbin/foo run each time a new /usr/local/etc/foo.dat is uploaded
to my host machine.
Dave: I personally think that a general, simple mechanism for invoking a
program via rsyncd.conf is a good idea, but we'd have to see what Tridge
thinks about it. I know that the idea of rsync invoking other programs has
come up in the past, and if I recall correctly I don't think Tridge was in
favor of it. For non-daemon mode I am opposed because people can always
write wrappers around rsync, but they can't do that for a daemon mode
server.
If Tridge does agree with the concept I expect it will probably still
have to come as contribution from a user.
Marc: OK, cranking out the code is pretty trivial... Probably my biggest
hurtle would be understanding any constraints that I need to conform to in
the creation of this new code. Are there any guidelines that you can
provide, and I guess I should also wait for Tridge to
respond before creating the code, since I don't want to waste mine/your time
if this won't be integrated.
Dave: I suggest making it a per-module option. Decide whether or not it
should run under the user/group id for that module. It may be tricky to
give the program a non-chrooted environment, which it will probably want;
you'll probably need to keep an extra parent process around. There should
be some mechanism for (optionally?) passing a list of files that have
changed to the program. Perhaps that would best be a file that has a copy
of the log messages for just that run.
Tridge: the main problem is that it is incompatible with the chroot() that
rsync currently does when run as a daemon.
Marc: Agreed.
Dave: It doesn't have to be. That's why I suggested keeping an extra
parent process; the parent process will fork and the child will do the
chroot, and the parent will do the exec after the child has finished. This
also has the advantage of being able to exec the program as root if we want
even if the child does a setuid/getuid to 'nobody' or something like that.
Tridge: We'd need to do a pretty careful security review before we dropped
the chroot. I know you've added an option to not chroot but that is a far
cry from making it the default.
Marc: Agreed too.
Dave: I agree we should not drop chroot as the default.
Tridge: I suppose we could say that the exec options only work if "use
chroot = no".
What do you think?
Marc: I think that is an acceptable limitation, at least until a very
careful review of the security were performed.
Dave: I think we should support both. If "use chroot = no", don't fork
the extra process.
Tridge: if we did add exec options then I think we would want to add general
purpose macro expansion code as well (like was recently
discussed). That would allow for:
post exec = /usr/local/bin/doit %f
where %f expands to the full path name. We'd want a bunch of macros (client,
service etc)
Dave: I think that the program may very likely want to be able to have
access to a list of the files that changed for that run. I'm not sure that
other things would be very useful. The post-processing commands in my nsbd
are always passed a single parameter which is the name of a file that
contains the list of changed files and deleted files and a few other pieces
of information which aren't relevant in rsync.
I suggest that we instead have an option "exec format" which can contain the
exact same things as "log format". When it is specified, rsync will create
a temporary file with the log information and pass the name of the temporary
file to the post exec program. The program should be able to extract
anything it could want out of the log file, especially after we improve the
logging to have a little more information like files deleted, etc.
Marc: That would be nice, though in my opinion may not be required. Since
the program being invoked is being done so as a direct result of
configuration in the rsyncd.conf system, a simple implementation could
assume that the program is knowledgeable about what needs to be done without
a list of files. I know in my particular cases, all I really care about is
invoking make in a particular directory. That is a trivial situation. Others
could probably make use of command line options like the %f above, but could
be reserved for a second iteration of the code.
Dave: There are some situations such as the one you speak of where the
file
names won't be needed, but I don't think we should put it in at all without
the ability to give the program more relevant information. It should be
quite easy to implement the "exec format" option I suggest.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Following are my comments regarding the above dialogue:
Henry: Just as a little background on myself; one of my ongoing projects is
the distribution of content and code to several thousand web servers
globally. This involves sourcing and permissioning thousands of 'branches',
'channels', 'webs'; whatever you want to call them, and distributing them
through hubs to endpoint servers and to synchronized farms of servers. I am
writing a wrapper to do this and would like to use rsync as the underlying
point-to-point engine because it is so powerful.
And some background on my distribution model. The overall architecture I am
assuming uses clients a sources and servers as targets for content
distribution. I know this goes against more traditional client-server use,
however it is the best way I can scale my needs. There are many uses for
the pull model as used by ftp servers or by any schema where endpoints check
for updates on a server. My needs are more along those of content
publishing and code releases, whereby content and code are pushed to
endpoints on demand. For many reasons not outlined here, a push model makes
more sense for my project. Therefore, assume that all distributions are
initiated and permissioned from the content source and pushed to the hubs
and endpoints. Daemons listen on those hubs and endpoints, ready to accept
a push.
Finally, Please do not take any of my comments critically or personally. I
appreciate the work that has gone into building and maintaining a program
such as rsync. I only wish to suggest improvements and I am willing to help
implement any community recommended changes. So, on we go..
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In the above thread about the --execute option much of the concern has been
about security. I believe that with small changes in the way rsync handles
security and user authentication the worries over --execute will go away and
thus --execute will be easy to implement. As such, I will comment on
security and user authentication first before finishing with --execute.
Security model:
In the current working model an rsync daemon running as root will spawn a
connection child as nobody by default. However, if 'uid, gid=#', the child
process will spawn as # and it will do it's copies with the permission of
#. In the case uid, gid=0 it will have root permission, write files as
root and have the ability to preserve client side file attributes.
This means Jil may now write files on a daemon server with ownership root
without native operating system root privileges. She may also say '--owner'
and write files owned by Jak onto the daemon server as Jak without assuming
the username Jak. This violates standard industry (or Unix) audit
requirements whereby a user should only be able to do what that user is
allowed to do on the native operating system. By allowing users to do root
stuff or allow one user to do operations as another user overrides the
native OS security model.
To prevent this we can turn off 'uid=' but now Jil can only write files as
nobody and can not do anything, even as herself on the daemon host. Well,
we can fix this by permissioning a module to uid='Jil'. But that of course
requires that we know in advance Jill will be logging on, deciding what
directories she has exclusive ownership, and writing a new server config in
advance.
All this is extremely inconvenient. Security is already enforced by the OS
and users should be allowed to do what-and-only-what they are allowed to do
on that host. FTP, Telnet and rsh-rcp all have this functionality.. You may
only do what you are allowed to do as if you were actually on the host. This
includes login, filesystem and execute permissions.
Why are we trying to implement our own permissioning system, over-riding the
existing OS system that works. Why not just do a proper authentication of
the user, and let them be themselves? I think this fundamental change to the
rsync security model can be implemented by one small change to the way
children are spawned.
Rather than spawn the child as nobody or as the user in the 'uid=' option,
spawn the child as the username running the client. We could leave the
'uid=' option to override the client. This way the user could just run as
the user. The identity of the client is already supplied in the rsync
protocol in the USER environment variable or on the command line
'user@host::' As long as we are happy with the strength of our
authentication there is no need to impose any other rules on the user
(unless we want to), since they could presumably log onto the host and do
those things anyway.
-Authentication:
If we are going to spawn the child as the client user, we need to be assured
the user is who he or she claims to be. Currently, rsyncd uses a password
file with arbitrary users (users not necessarily on the host) and a clear
text password, or secrets file. It is secured by 'root read only'
permissions. Modules may permission allowed users from that list. We all
recognize that the secrets file is a good, but antique idea and does not
pass audits any longer. My only recommendation is that we upgrade the login
functionality to allow native OS usernames and passwords. Then perhaps, we
extend this later to include LDAP, radius, kerberos, PKI, etc.
--execute
Once we have child daemons securely spawning as normal users we can have no
worries about adding an --execute option. I say this because we are not
granting any new privileges to the user which they do not already have by
logging onto the box directly. Jil can not do things as Jak, Like send
nasty mail to his boss, and can not do things as root like, rm /.
-Summary
I think --execute could become a safe and extremely powerful addition to
rsync by making two relatively simple changes to the existing daemon.
1.) When 'uid=' is not specified in a module spawn the child as
the client username
2.) Update the login function to allow native OS username and
password. 'login=native'
Now we can create an --execute feature to allow execution of arbitrary
scripts either before or after file deployment and either 'on success' or
'on failure'.
--execute="/path/doit -q arg" after_deploy on_success
Perhaps it be required that if the --execute option is used then either the
uid= or login= be enforced. That is I don't think --execute should be
allowed in anonymous mode.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If we could see this new --execute feature added, I think rsync will find
much wider acceptability and many more applications. The underlying
protocol of rsync is the best around, yet we see limited or 'experimental
only' use, especially in the financial sectors, because it will not pass an
audit. By making a few small, but key changes to the existing rsync code
base, that is, by cleaning up security and user authentication, and by
adding --execute we will be able to build scalable enterprise solutions on
rsync.
I am interested in all your comments on my ideas, and I am willing to work
with anybody to help build the improvements.
Thanks for reading this far,
--Henry
Regards,
Henry Wrieth
> CREDIT | FIRST
> SUISSE | BOSTON
Global Internet Intranet Services /
Global Web Services (New York)
> Voice: 212.325.7714
Fax: 212.325.9596
E-mail: [EMAIL PROTECTED]
Page: 888.977.8523 or
[EMAIL PROTECTED]
<<Wrieth, Henry.vcf>>
This message is for the named person's use only. It may contain
confidential, proprietary or legally privileged information. No
confidentiality or privilege is waived or lost by any mistransmission.
If you receive this message in error, please immediately delete it and all
copies of it from your system, destroy any hard copies of it and notify the
sender. You must not, directly or indirectly, use, disclose, distribute,
print, or copy any part of this message if you are not the intended
recipient. CREDIT SUISSE GROUP and each of its subsidiaries each reserve
the right to monitor all e-mail communications through its networks. Any
views expressed in this message are those of the individual sender, except
where the message states otherwise and the sender is authorised to state
them to be the views of any such entity.
Unless otherwise stated, any pricing information given in this message is
indicative only, is subject to change and does not constitute an offer to
deal at any price quoted.
Any reference to the terms of executed transactions should be treated as
preliminary only and subject to our formal written confirmation.
Wrieth, Henry.vcf
