avoid mkdir/selinux failure when mknod is a shell built-in
This test would fail not only because the built-in mknod doesn't support -Z, but because it doesn't know about 'p' pipes. tests: avoid mkdir/selinux failure when mknod is a shell built-in * tests/mkdir/selinux: Skip the mknod test if it's a built-in. diff --git a/tests/mkdir/selinux b/tests/mkdir/selinux index fcd45a8..adbaa6a 100755 --- a/tests/mkdir/selinux +++ b/tests/mkdir/selinux @@ -30,9 +30,15 @@ fi c=invalid-selinux-context msg="failed to set default file creation context to \`$c':" +# In OpenBSD's /bin/sh, mknod is a shell built-in. +# In that case, just skip the mknod test. +( mknod --version 2>&1 ) | grep 'GNU coreutils' \ +&& mknod='mknod b p' \ +|| mknod='mkdir dir' + # Test each of mkdir, mknod, mkfifo with "-Z invalid-context". -for cmd_w_arg in 'mkdir dir' 'mknod b p' 'mkfifo f'; do +for cmd_w_arg in 'mkdir dir' "$mknod" 'mkfifo f'; do $cmd_w_arg -Z $c 2> out && fail=1 set $cmd_w_arg; cmd=$1 echo "$cmd: $msg" > exp || fail=1 -- 1.5.5.56.gbcf7f ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to Jim Meyering on 4/16/2008 2:33 AM: | This test would fail not only because the built-in mknod | doesn't support -Z, but because it doesn't know about 'p' pipes. | | tests: avoid mkdir/selinux failure when mknod is a shell built-in | * tests/mkdir/selinux: Skip the mknod test if it's a built-in. Couldn't you try to defeat the shell builtin by using a subshell and exec to force the PATH lookup? For example, on bash: $ printf --version | head -n1 bash: printf: --: invalid option printf: usage: printf [-v var] format [arguments] $ (exec printf --version) | head -n1 printf (GNU coreutils) 6.10 Doesn't this also mean that you need to patch mknod.c to print USAGE_BUILTIN_WARNING? - -- Don't work too hard, make some time for fun as well! Eric Blake [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Cygwin) Comment: Public key at home.comcast.net/~ericblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkgF61YACgkQ84KuGfSFAYDxgQCfcCM73K1gHf3zzTLsJrkqLIY+ T44AnRKNgfzjU4x7mMOzysT2zHCHnR+T =TABh -END PGP SIGNATURE- ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
Eric Blake <[EMAIL PROTECTED]> wrote: > According to Jim Meyering on 4/16/2008 2:33 AM: > | This test would fail not only because the built-in mknod > | doesn't support -Z, but because it doesn't know about 'p' pipes. > | > | tests: avoid mkdir/selinux failure when mknod is a shell built-in > | * tests/mkdir/selinux: Skip the mknod test if it's a built-in. > > Couldn't you try to defeat the shell builtin by using a subshell and exec > to force the PATH lookup? For example, on bash: > > $ printf --version | head -n1 > bash: printf: --: invalid option > printf: usage: printf [-v var] format [arguments] > $ (exec printf --version) | head -n1 > printf (GNU coreutils) 6.10 My first reaction was "great! that looks much better". Unfortunately, the technique doesn't work with that shell: openbsd$ ./mknod --version|head -1 mknod (GNU coreutils) 6.10.188-7cb24 openbsd$ PATH=. /bin/sh -c 'mknod --version'|head -1 /bin/sh: mknod: --: unknown option /bin/sh: mknod: usage: mknod [-m mode] name [b | c] major minor /bin/sh: mknod: usage: mknod [-m mode] name p However, if m4/gl_POSIX_SHELL were to test for this, I could clean up this test as you suggest, and simply skip it whenever POSIX_SHELL is empty. Volunteers? > Doesn't this also mean that you need to patch mknod.c to print > USAGE_BUILTIN_WARNING? Yes. Good catch! Thanks! mknod --help: note that this command may be a shell built-in * src/mknod.c (usage): Print USAGE_BUILTIN_WARNING. Suggestion from Eric Blake. diff --git a/src/mknod.c b/src/mknod.c index 8a1718d..445117c 100644 --- a/src/mknod.c +++ b/src/mknod.c @@ -83,6 +83,7 @@ otherwise, as decimal. TYPE may be:\n\ c, u create a character (unbuffered) special file\n\ p create a FIFO\n\ "), stdout); + printf (USAGE_BUILTIN_WARNING, PROGRAM_NAME); emit_bug_reporting_address (); } exit (status); -- 1.5.5.56.gbcf7f ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
Hello! On Wed, Apr 16, 2008 at 02:30:57PM +0200, Jim Meyering wrote: > Eric Blake <[EMAIL PROTECTED]> wrote: > > According to Jim Meyering on 4/16/2008 2:33 AM: > > | This test would fail not only because the built-in mknod > > | doesn't support -Z, but because it doesn't know about 'p' pipes. > > | > > | tests: avoid mkdir/selinux failure when mknod is a shell built-in > > | * tests/mkdir/selinux: Skip the mknod test if it's a built-in. > > > > Couldn't you try to defeat the shell builtin by using a subshell and exec > > to force the PATH lookup? For example, on bash: > > > > $ printf --version | head -n1 > > bash: printf: --: invalid option > > printf: usage: printf [-v var] format [arguments] > > $ (exec printf --version) | head -n1 > > printf (GNU coreutils) 6.10 > > My first reaction was "great! that looks much better". > Unfortunately, the technique doesn't work with that shell: > > openbsd$ ./mknod --version|head -1 > mknod (GNU coreutils) 6.10.188-7cb24 > openbsd$ PATH=. /bin/sh -c 'mknod --version'|head -1 ^ exec? Regards, Thomas signature.asc Description: Digital signature ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to Jim Meyering on 4/16/2008 6:30 AM: | My first reaction was "great! that looks much better". | Unfortunately, the technique doesn't work with that shell: | | openbsd$ ./mknod --version|head -1 | mknod (GNU coreutils) 6.10.188-7cb24 | openbsd$ PATH=. /bin/sh -c 'mknod --version'|head -1 | /bin/sh: mknod: --: unknown option | /bin/sh: mknod: usage: mknod [-m mode] name [b | c] major minor | /bin/sh: mknod: usage: mknod [-m mode] name p That still uses a builtin. You need an exec in the loop to force a PATH search (even command, which bypasses functions, won't bypass builtins). Or perhaps a forwarding command will help; do any of these work? $ /bin/sh -c '(exec mknod --version)' | head -1 $ /bin/sh -c 'nice mknod --version' | head -1 $ /bin/sh -c 'nohup mknod --version' | head -1 - -- Don't work too hard, make some time for fun as well! Eric Blake [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Cygwin) Comment: Public key at home.comcast.net/~ericblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkgF9RMACgkQ84KuGfSFAYD3eACeJSYwRzZXjUCknA/wnqsOMHCU hMAAoKLQv1qEKOn07Gkc0mq7El/iJcgx =0FHy -END PGP SIGNATURE- ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
Jim Meyering <[EMAIL PROTECTED]> writes: > Unfortunately, the technique doesn't work with that shell: > > openbsd$ ./mknod --version|head -1 > mknod (GNU coreutils) 6.10.188-7cb24 > openbsd$ PATH=. /bin/sh -c 'mknod --version'|head -1 What about /bin/sh -c 'exec mknod --version'? Andreas. -- Andreas Schwab, SuSE Labs, [EMAIL PROTECTED] SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany PGP key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
Thomas Schwinge <[EMAIL PROTECTED]> wrote: > On Wed, Apr 16, 2008 at 02:30:57PM +0200, Jim Meyering wrote: ... >> My first reaction was "great! that looks much better". >> Unfortunately, the technique doesn't work with that shell: >> >> openbsd$ ./mknod --version|head -1 >> mknod (GNU coreutils) 6.10.188-7cb24 >> openbsd$ PATH=. /bin/sh -c 'mknod --version'|head -1 > > ^ exec? Of course. My stand-alone example was wrong. However, with 'exec' it fails the same way. On the other hand, Eric's suggestions to use nice or nohup do work: $ PATH=. /bin/sh -c 'exec mknod --version'|head -1 /bin/sh: mknod: --: unknown option /bin/sh: mknod: usage: mknod [-m mode] name [b | c] major minor /bin/sh: mknod: usage: mknod [-m mode] name p $ PATH=. /bin/sh -c '(exec mknod --version)' | head -1 /bin/sh: mknod: --: unknown option /bin/sh: mknod: usage: mknod [-m mode] name [b | c] major minor /bin/sh: mknod: usage: mknod [-m mode] name p $ PATH=. /bin/sh -c 'nice mknod --version' | head -1 mknod (GNU coreutils) 6.10.188-7cb24 $ PATH=. /bin/sh -c 'nohup mknod --version' | head -1 nohup: ignoring input and redirecting stderr to stdout mknod (GNU coreutils) 6.10.188-7cb24 $ So I'll go with nice. Thanks, Eric! ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to Jim Meyering on 4/16/2008 6:57 AM: | $ PATH=. /bin/sh -c 'exec mknod --version'|head -1 | /bin/sh: mknod: --: unknown option Ouch - this looks like a POSIX compliance bug in exec; I'm adding bug-autoconf to the distribution in case we want to document this corner case bug in the shell portability section. POSIX states that exec is supposed to bypass shell builtins (and while special shell builtins, like 'exit', give undefined behavior when passed to exec, regular shell builtins, like 'fg', are required to exist in PATH even if they can't quite do as much work as their builtin counterpart). Do you know which shell heritage the OpenBSD /bin/sh derives from? | $ PATH=. /bin/sh -c 'nice mknod --version' | head -1 | mknod (GNU coreutils) 6.10.188-7cb24 | So I'll go with nice. Sounds nice to me (sorry, couldn't resist :) - -- Don't work too hard, make some time for fun as well! Eric Blake [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Cygwin) Comment: Public key at home.comcast.net/~ericblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkgF+h4ACgkQ84KuGfSFAYBgwQCgkm+chKbhFZ+zkxx7U6Mn6QT1 NCUAoKAcBXl5/fg6KyURTn9fKu2kxPFL =SDPR -END PGP SIGNATURE- ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
Eric Blake <[EMAIL PROTECTED]> wrote: > According to Jim Meyering on 4/16/2008 6:57 AM: > | $ PATH=. /bin/sh -c 'exec mknod --version'|head -1 > | /bin/sh: mknod: --: unknown option > > Ouch - this looks like a POSIX compliance bug in exec; I'm adding > bug-autoconf to the distribution in case we want to document this corner > case bug in the shell portability section. POSIX states that exec is > supposed to bypass shell builtins (and while special shell builtins, like > 'exit', give undefined behavior when passed to exec, regular shell > builtins, like 'fg', are required to exist in PATH even if they can't > quite do as much work as their builtin counterpart). Do you know which > shell heritage the OpenBSD /bin/sh derives from? This is on 3.9 GENERIC#617 i386 OpenBSD, and /bin/sh is a hard link to /bin/ksh. "strings" says it's PDKSH: PD KSH v5.2.14 99/07/13.2 > | $ PATH=. /bin/sh -c 'nice mknod --version' | head -1 > | mknod (GNU coreutils) 6.10.188-7cb24 > > | So I'll go with nice. > > Sounds nice to me (sorry, couldn't resist :) :) ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
Eric Blake <[EMAIL PROTECTED]> wrote: > According to Jim Meyering on 4/16/2008 6:57 AM: > | $ PATH=. /bin/sh -c 'exec mknod --version'|head -1 > | /bin/sh: mknod: --: unknown option > > Ouch - this looks like a POSIX compliance bug in exec; I'm adding > bug-autoconf to the distribution in case we want to document this corner > case bug in the shell portability section. POSIX states that exec is > supposed to bypass shell builtins (and while special shell builtins, like > 'exit', give undefined behavior when passed to exec, regular shell > builtins, like 'fg', are required to exist in PATH even if they can't > quite do as much work as their builtin counterpart). Do you know which > shell heritage the OpenBSD /bin/sh derives from? > > | $ PATH=. /bin/sh -c 'nice mknod --version' | head -1 > | mknod (GNU coreutils) 6.10.188-7cb24 > > | So I'll go with nice. > > Sounds nice to me (sorry, couldn't resist :) Here's the patch I've just pushed: >From 57ae7da07593ab0134e120ea79819fdd069563a1 Mon Sep 17 00:00:00 2001 From: Jim Meyering <[EMAIL PROTECTED]> Date: Wed, 16 Apr 2008 15:28:33 +0200 Subject: [PATCH] tests: accommodate built-in mknod more cleanly * tests/mkdir/selinux: Undo most of previous change, bc22dbbf844f31ddaf2e68b167d0128a985d73ab, and instead invoke the command via "nice". Using "exec" should be enough, but isn't with OpenBSD's PD KSH v5.2.14 99/07/13.2. Eric Blake suggested using nice. Signed-off-by: Jim Meyering <[EMAIL PROTECTED]> --- tests/mkdir/selinux | 12 1 files changed, 4 insertions(+), 8 deletions(-) diff --git a/tests/mkdir/selinux b/tests/mkdir/selinux index 4363275..45c2399 100755 --- a/tests/mkdir/selinux +++ b/tests/mkdir/selinux @@ -34,16 +34,12 @@ fi c=invalid-selinux-context msg="failed to set default file creation context to \`$c':" -# In OpenBSD's /bin/sh, mknod is a shell built-in. -# In that case, just skip the mknod test. -( mknod --version 2>&1 ) | grep 'GNU coreutils' \ -&& mknod='mknod b p' \ -|| mknod='mkdir dir' - # Test each of mkdir, mknod, mkfifo with "-Z invalid-context". -for cmd_w_arg in 'mkdir dir' "$mknod" 'mkfifo f'; do - $cmd_w_arg -Z $c 2> out && fail=1 +for cmd_w_arg in 'mkdir dir' 'mknod b p' 'mkfifo f'; do + # In OpenBSD's /bin/sh, mknod is a shell built-in. + # Running via "nice" ensures we run our program and not the built-in. + nice -- $cmd_w_arg -Z $c 2> out && fail=1 set $cmd_w_arg; cmd=$1 echo "$cmd: $msg" > exp || fail=1 -- 1.5.5.56.gbcf7f ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
Hi Eric,
* Eric Blake wrote on Wed, Apr 16, 2008 at 03:07:42PM CEST:
> According to Jim Meyering on 4/16/2008 6:57 AM:
> | $ PATH=. /bin/sh -c 'exec mknod --version'|head -1
> | /bin/sh: mknod: --: unknown option
>
> Ouch - this looks like a POSIX compliance bug in exec; I'm adding
> bug-autoconf to the distribution in case we want to document this corner
> case bug in the shell portability section. POSIX states that exec is
> supposed to bypass shell builtins (and while special shell builtins, like
> 'exit', give undefined behavior when passed to exec, regular shell
> builtins, like 'fg', are required to exist in PATH even if they can't
> quite do as much work as their builtin counterpart).
Can you please point me to where POSIX specifies these things? I cannot
find in the page describing exec, that it has to bypass built-ins.
Also, I cannot find at all that regular shell builtins have to exist as
independent programs in $PATH. The only thing I found wrt. special
built-ins was
The special built-in utilities in this section need not be provided
in a manner accessible via the exec family of functions defined in
the System Interfaces volume of IEEE Std 1003.1-2001.
which isn't a requirement on the implementation.
I'm looking at SUSv3, by the way.
* Eric Blake wrote on Wed, Apr 16, 2008 at 06:14:35PM CEST:
>
> Any comments before I check it in?
> [EMAIL PROTECTED] @command{exec}
> +Posix divides the set of shell built-ins into two groups. Special
> +built-ins (such as @command{exit}) must impact the environment of the
> +current shell, and need not be available through @command{exec}.
> +Regular built-ins (such as @command{echo} or @command{cd}) must not
> +propogate variable assignments to the environment of the current shell,
propagate
> +On the other hand, @command{pdksh} 5.2.14 refuses to execute the
> +executable replacement, using the built-in no matter what:
OpenBSD's /bin/sh is heavily modified from pdksh. While it seems that
both share this bug, in general one should not be quick to assume both
share all features.
> +This has the interest effect that @command{pdksh} can use @command{exec}
s/interest/interesting/
> +on special built-ins, even though Posix doesn't require it:
Thanks,
Ralf
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
Matthew Woehlke <[EMAIL PROTECTED]> wrote: > Eric Blake wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> According to Jim Meyering on 4/16/2008 6:30 AM: >> | My first reaction was "great! that looks much better". >> | Unfortunately, the technique doesn't work with that shell: >> | >> | openbsd$ ./mknod --version|head -1 >> | mknod (GNU coreutils) 6.10.188-7cb24 >> | openbsd$ PATH=. /bin/sh -c 'mknod --version'|head -1 >> | /bin/sh: mknod: --: unknown option >> | /bin/sh: mknod: usage: mknod [-m mode] name [b | c] major minor >> | /bin/sh: mknod: usage: mknod [-m mode] name p >> >> That still uses a builtin. You need an exec in the loop to force a PATH >> search (even command, which bypasses functions, won't bypass builtins). >> Or perhaps a forwarding command will help; do any of these work? >> >> $ /bin/sh -c '(exec mknod --version)' | head -1 >> $ /bin/sh -c 'nice mknod --version' | head -1 >> $ /bin/sh -c 'nohup mknod --version' | head -1 > > I realize you already pushed something, but for the record, wouldn't > env' work as well (and without the side effects)? Ha! Good call. That's even better ;-) ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
Matthew Woehlke <[EMAIL PROTECTED]> wrote: >> $ /bin/sh -c '(exec mknod --version)' | head -1 >> $ /bin/sh -c 'nice mknod --version' | head -1 >> $ /bin/sh -c 'nohup mknod --version' | head -1 > > I realize you already pushed something, but for the record, wouldn't > env' work as well (and without the side effects)? This "simple" test has certainly been beaten to death. I confirmed that it works. I'll push this soon. Holler if the attribution should be different. >From 0d03baa068c20f09bcba3a915fd756db7359a7fa Mon Sep 17 00:00:00 2001 From: Matthew Woehlke <[EMAIL PROTECTED]> Date: Wed, 16 Apr 2008 19:15:05 +0200 Subject: [PATCH] tests: accommodate built-in mknod more cleanly still * tests/mkdir/selinux: Use "env" rather than "nice". Signed-off-by: Jim Meyering <[EMAIL PROTECTED]> --- tests/mkdir/selinux |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/mkdir/selinux b/tests/mkdir/selinux index 45c2399..91b23b1 100755 --- a/tests/mkdir/selinux +++ b/tests/mkdir/selinux @@ -38,8 +38,8 @@ msg="failed to set default file creation context to \`$c':" for cmd_w_arg in 'mkdir dir' 'mknod b p' 'mkfifo f'; do # In OpenBSD's /bin/sh, mknod is a shell built-in. - # Running via "nice" ensures we run our program and not the built-in. - nice -- $cmd_w_arg -Z $c 2> out && fail=1 + # Running via "env" ensures we run our program and not the built-in. + env -- $cmd_w_arg -Z $c 2> out && fail=1 set $cmd_w_arg; cmd=$1 echo "$cmd: $msg" > exp || fail=1 -- 1.5.5.56.gbcf7f ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
Eric Blake wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to Jim Meyering on 4/16/2008 6:30 AM: | My first reaction was "great! that looks much better". | Unfortunately, the technique doesn't work with that shell: | | openbsd$ ./mknod --version|head -1 | mknod (GNU coreutils) 6.10.188-7cb24 | openbsd$ PATH=. /bin/sh -c 'mknod --version'|head -1 | /bin/sh: mknod: --: unknown option | /bin/sh: mknod: usage: mknod [-m mode] name [b | c] major minor | /bin/sh: mknod: usage: mknod [-m mode] name p That still uses a builtin. You need an exec in the loop to force a PATH search (even command, which bypasses functions, won't bypass builtins). Or perhaps a forwarding command will help; do any of these work? $ /bin/sh -c '(exec mknod --version)' | head -1 $ /bin/sh -c 'nice mknod --version' | head -1 $ /bin/sh -c 'nohup mknod --version' | head -1 I realize you already pushed something, but for the record, wouldn't 'env' work as well (and without the side effects)? -- Matthew > pinotree uses the large trout on tsdgeos and PutHuhn :) > PutHuhn runs > tsdgeos lights a fire and eats the trout (with apologies to Pino Toscano, PutHuhn and Albert Astals Cid, who came up with this entirely on their own) ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: avoid mkdir/selinux failure when mknod is a shell built-in
Ralf Wildenhues gmx.de> writes: > > case bug in the shell portability section. POSIX states that exec is > > supposed to bypass shell builtins (and while special shell builtins, like > > 'exit', give undefined behavior when passed to exec, regular shell > > builtins, like 'fg', are required to exist in PATH even if they can't > > quite do as much work as their builtin counterpart). > > Can you please point me to where POSIX specifies these things? I cannot > find in the page describing exec, that it has to bypass built-ins. Point taken. XCU 2.9.1.1 Command Search and Execution gives this order: 1. special built-ins 2. user-provided shell functions 3. certain utilities (for example, true or command) (the point of this list is to guarantee that regardless of how badly PATH is munged, these utilities will still work rather than executing a trojan implementation earlier on the path; useful since command can then be used to generate a repaired PATH) 4. PATH search, including shell-provided functions and regular builtins: "the command shall be searched for using the PATH environment variable as described in XBD Chapter 8 (on page 173): i. If the search is successful: a. If the system has implemented the utility as a regular built-in or as a shell function, it shall be invoked at this point in the path search." This makes it sound like the shell cannot use the builtin unless the PATH search succeeds, but once successful, the shell can then invoke its builtin instead of calling execv() on what was found on PATH. So I guess POSIX does not require that exec must bypass builtins, and therefore, pdksh is not buggy, just different. > Also, I cannot find at all that regular shell builtins have to exist as > independent programs in $PATH. The only thing I found wrt. special > built-ins was > >The special built-in utilities in this section need not be provided >in a manner accessible via the exec family of functions defined in >the System Interfaces volume of IEEE Std 1003.1-2001. > > which isn't a requirement on the implementation. In addition to the above wording about simple command execution requiring a successful path search, I think this wording in command is most applicable here: http://www.opengroup.org/onlinepubs/009695399/utilities/command.html "Two types of regular built-ins could be encountered on a system and these are described separately by command. The description of command search in Command Search and Execution allows for a standard utility to be implemented as a regular built-in as long as it is found in the appropriate place in a PATH search. So, for example, command -v true might yield /bin/true or some similar pathname. Other implementation-defined utilities that are not defined by this volume of IEEE Std 1003.1-2001 might exist only as built-ins and have no pathname associated with them. These produce output identified as (regular) built-ins. Applications encountering these are not able to count on execing them, using them with nohup, overriding them with a different PATH , and so on." So mknod qualifies as a candidate for a regular builtin that need not be PATH- based (since POSIX doesn't require mknod(1)). I would almost claim that true MUST exist in the PATH, except that true is on the list of 17 utilities that must work regardless of PATH search (hmm - the Posix example of command -v true yielding /bin/true is thus suspect). But printf is a good example - it is a POSIX-mandated utility, not on the special list of required utilities, but often implemented as a builtin; even if it is a builtin, this statement says it must also exist on PATH. > I'm looking at SUSv3, by the way. As am I. > > * Eric Blake wrote on Wed, Apr 16, 2008 at 06:14:35PM CEST: > > > > Any comments before I check it in? > Thanks for the nitpicks. How about this attempt at rewording things? (I dropped mention of exec special-builtin, since that is already explicitly undefined by Posix) >From 9651f4a0e1b80c55da23a5ceaa0237121e0b95f2 Mon Sep 17 00:00:00 2001 From: Eric Blake <[EMAIL PROTECTED]> Date: Wed, 16 Apr 2008 10:10:31 -0600 Subject: [PATCH] Document pdksh exec behavior. * doc/autoconf.texi (Limitations of Builtins) : New subsection. Discovered by Jim Meyering. Signed-off-by: Eric Blake <[EMAIL PROTECTED]> --- ChangeLog |7 +++ doc/autoconf.texi | 48 2 files changed, 55 insertions(+), 0 deletions(-) diff --git a/ChangeLog b/ChangeLog index b1d9a4e..9d9d04e 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,10 @@ +2008-04-16 Eric Blake <[EMAIL PROTECTED]> + + Document pdksh exec behavior. + * doc/autoconf.texi (Limitations of Builtins) : New + subsection. + Discovered by Jim Meyering. + 2008-04-14 Ralf Wildenhues <[EMAIL PROTECTED]> * tests/autotest.at (AT_CHECK_AT): Allow to pass additional diff --git a/doc/autoconf.texi b/doc/autoconf.texi index 8b1
PATCH: md5sum --check did not work with "\r\n" and "\r" line endings
Hello list, md5sum with the -c option did not work on files with "\r\n" and "\r" line endings. A patch is attached. With best regards, Simon Hengel. === modified file 'src/md5sum.c' --- src/md5sum.c 2008-04-16 15:51:12 + +++ src/md5sum.c 2008-04-16 16:27:50 + @@ -465,7 +465,12 @@ continue; /* Remove any trailing newline. */ - if (line[line_length - 1] == '\n') + while (line_length && + ( + (line[line_length - 1] == '\n') || + (line[line_length - 1] == '\r') + ) + ) line[--line_length] = '\0'; if (! (split_3 (line, line_length, &hex_digest, &binary, &filename) ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Enhancement request -- performance improvement
Hello, Running du -sk on an apache disk cache containing 10GB of data and 30,000 directories and files I see du using maybe .03% of the cpu. It takes an hour for it to complete. Are there any plans to make du multi-threaded? Or otherwise improve it's performance? Is it filesystem sensitive? We are using ext3. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
DF utility
Utility: df Didn't know where to send these suggestions but two things that would be nice... 1. Colour. Show different file system types (ie nfs) in a different colour 2. Adjustable width. I have my screen width at about 142 characters wide.That way the nfs mounts aren't taking up two lines. Michael -- *G. Michael Carter* *Projects* .. GarageSales http://www.garagesaleorganizer.ca Blog/Diarieshttp://theshadow.carterfamily.ca Family Pictures http://photoalbum.carterfamily.ca GeoCaching http://www.geocache.carterfamily.ca *Contact Info* Office 1-905-890-8100 x 28494 Work Site http://www.oracle.com 1-905-267-8494 Personalhttp://www.carterfamily.ca Cell519-215-1869AIM MikeyCarter Fax 1-519-941-0009 OIM Michael.Carter ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
[PATCH] Use a hash rather than a linked-list for cycle check in cp
This addresses a FIXME in src/copy.c:
-/* FIXME: rewrite this to use a hash table so we avoid the quadratic
- performance hit that's probably noticeable only on trees deeper
- than a few hundred levels. See use of active_dir_map in remove.c */
The performance benefit is there, but on my machine with a PATH_MAX of
4096 it's hard to see, because the userland work `cp' does is dwarfed
by the work the kernel does on its behalf:
$ time src/cp.master -r a b
real0m54.032s
user0m3.236s<-- coreutils HEAD
sys 0m47.335s
$ time src/cp -r a c
real0m53.475s
user0m0.624s<-- with patch
sys 0m48.639s
Thanks,
Bo
From 224328e4bda44aa25cd5c98b1c13751ecea865c7 Mon Sep 17 00:00:00 2001
From: Bo Borgerson <[EMAIL PROTECTED]>
Date: Wed, 16 Apr 2008 13:44:36 -0400
Subject: [PATCH] Use a hash rather than a linked-list for cycle check in cp
* NEWS: List the change of cycle check behavior.
* src/copy.c (struct dir_info): These go in the hash.
(static Hash_table *ancestry_hash): This is the hash.
(static size_t dir_info_hasher): Simple hasher based on INO.
(static bool dir_info_comparator): Checks equivalence of INO and DEV.
(static bool ancestry_insert): Insert a dir_info entry for the current dir.
(static bool ancestry_delete): Delete the hash entry for the current dir.
(copy_internal): Now calls ancestry_insert and ancestry_delete instead of
managing the linked-list inline and calling is_ancestor for the cycle check.
Signed-off-by: Bo Borgerson <[EMAIL PROTECTED]>
---
NEWS |2 +
src/copy.c | 106 +++
2 files changed, 72 insertions(+), 36 deletions(-)
diff --git a/NEWS b/NEWS
index 3a584e9..111e0c1 100644
--- a/NEWS
+++ b/NEWS
@@ -69,6 +69,8 @@ GNU coreutils NEWS-*- outline -*-
seq gives better diagnostics for invalid formats.
+ cp now uses a hash for cycle checks rather than a linked-list of ancestry.
+
** Portability
rm now works properly even on systems like BeOS and Haiku,
diff --git a/src/copy.c b/src/copy.c
index c2f21a3..8e673f0 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -83,19 +83,74 @@ rpl_mkfifo (char const *file, mode_t mode)
#define SAME_GROUP(A, B) ((A).st_gid == (B).st_gid)
#define SAME_OWNER_AND_GROUP(A, B) (SAME_OWNER (A, B) && SAME_GROUP (A, B))
-struct dir_list
+
+/* This is for looking up directories by inode/device to ensure we don't
+ have any cycles. */
+static Hash_table *ancestors_hash;
+
+enum { INIT_ANCESTRY_SIZE = 47 };
+
+struct dir_info
{
- struct dir_list *parent;
ino_t ino;
dev_t dev;
};
+static size_t
+dir_info_hasher (const void *entry, size_t tabsize)
+{
+ const struct dir_info *node = entry;
+ return node->ino % tabsize;
+}
+
+static bool
+dir_info_comparator (const void *e1, const void *e2)
+{
+ const struct dir_info *n1 = e1, *n2 = e2;
+ return n1->ino == n2->ino && n1->dev == n2->dev;
+}
+
+/* First check whether this directory has been seen. If it has then
+ return false because we've encountered a cycle in the graph. If it
+ hasn't, then insert it into the ancestry hash and return true. */
+
+static bool
+ancestry_insert (struct dir_info *dir, struct stat *src_sb)
+{
+ dir->ino = src_sb->st_ino;
+ dir->dev = src_sb->st_dev;
+
+ if (! ancestors_hash)
+{
+ ancestors_hash = hash_initialize (INIT_ANCESTRY_SIZE, NULL,
+ dir_info_hasher,
+ dir_info_comparator, NULL);
+ if (! ancestors_hash)
+ xalloc_die ();
+}
+
+ if (hash_lookup (ancestors_hash, dir))
+return false;
+
+ hash_insert (ancestors_hash, dir);
+
+ return true;
+}
+
+/* This is called when recursion through DIR is complete. Note that
+ `*dir' is actually a pointer into the stack of the caller, so there's
+ no free here. */
+static inline void
+ancestry_delete (const struct dir_info *dir)
+{
+ hash_delete (ancestors_hash, dir);
+}
+
/* Initial size of the cp.dest_info hash table. */
#define DEST_INFO_INITIAL_CAPACITY 61
static bool copy_internal (char const *src_name, char const *dst_name,
bool new_dst, dev_t device,
- struct dir_list *ancestors,
const struct cp_options *x,
bool command_line_arg,
bool *copy_into_self,
@@ -110,23 +165,6 @@ static char const *top_level_dst_name;
/* The invocation name of this program. */
extern char *program_name;
-/* FIXME: describe */
-/* FIXME: rewrite this to use a hash table so we avoid the quadratic
- performance hit that's probably noticeable only on trees deeper
- than a few hundred levels. See use of active_dir_map in remove.c */
-
-static bool
-is_ancestor (const struct stat *sb, const struct dir_list *ancestors)
-{
- while (ancestors != 0)
-{
- if (ancestors->ino == sb->st_ino && ancestors->dev == sb->st_dev)
- return true;
- ancestors = ancestors->parent;
-}
- return false;
-}
-
/* Read the contents of the directory SRC_NAME_IN, and recursively
copy the contents to DST_NA
Re: avoid mkdir/selinux failure when mknod is a shell built-in
[ re-added bug-autoconf ] * Eric Blake wrote on Wed, Apr 16, 2008 at 08:04:23PM CEST: > Subject: [PATCH] Document pdksh exec behavior. > > * doc/autoconf.texi (Limitations of Builtins) : New > subsection. > Discovered by Jim Meyering. This looks good to me, thanks. Cheers, Ralf ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
dd skip bug?
dd handles skip weirdly disk=/dev/sda8 dd if=$disk bs=8M count=1 skip=1000 of=/dev/null #ok dd if=$disk bs=8M count=1 skip=1000K of=/dev/null #reads whole disk! as seek fails I had a 10s look at the source and noticed a comment saying POSIX doesn't specify what we should do when skipping past the end of input. For seekable files though, reading the whole thing is unexpected to me at least. I would expect it to do: if (seekable && !seek(skip_len)) exit(EXIT_FAILURE); Pádraig. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
RM disregards file level permissions and uses directory permissions instead.
We are in the cutover process and one of the DBAs found this behavior. If testfile1 is owned by usera:group1 in a parent directory with permissions 777 owned by usera:group1, userb:group2 can delete testfile1 even if testfile1 has permissions 600. Conversely if the same parent directory has permissions 755 and userb owns testfile2 with 777 permissions they cannot delete their own file from that parent directory. I found this happens on rm version 5.2.1 running on RHEL4 and 5.97 running on RHEL5. It does not specify anything about this behavior in the man pages or other docs. Can you please look into this and let me know about this behavior? Here is more OS level detail: test1 $ ls -lR .: total 8 drwxrwxrwx 2 test1 users 4096 Apr 16 18:15 dir1 drwxr-xr-x 2 test1 users 4096 Apr 16 18:15 dir2 ./dir1: total 0 -rw--- 1 test1 users 0 Apr 16 18:15 testfile1 ./dir2: total 0 -rwxrwxrwx 1 test2 users 0 Apr 16 18:15 testfile2 Now as test2 user: test2 $ rm test1/testfile1 rm: cannot remove `test1/testfile1': No such file or directory test2 $ rm dir1/testfile1 rm: remove write-protected regular empty file `dir1/testfile1'? y test2 $ rm dir2/testfile2 test2 $ ls -lR .: total 8 drwxrwxrwx 2 test1 users 4096 Apr 16 18:16 dir1 drwxr-xr-x 2 test1 users 4096 Apr 16 18:15 dir2 ./dir1: total 0 ./dir2: total 0 -rwxrwxrwx 1 test2 users 0 Apr 16 18:15 testfile2 Thanks! James J Perry ([EMAIL PROTECTED]) DS Waters of America, Inc. 770-933-1401 ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: RM disregards file level permissions and uses directory permissions instead.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to James J. Perry on 4/16/2008 4:25 PM: | We are in the cutover process and one of the DBAs found this behavior. | If testfile1 is owned by usera:group1 in a parent directory with | permissions 777 owned by usera:group1, userb:group2 can delete testfile1 | even if testfile1 has permissions 600. Conversely if the same parent | directory has permissions 755 and userb owns testfile2 with 777 | permissions they cannot delete their own file from that parent | directory. Yes, and POSIX requires this. You probably want to set the directory's sticky bit. | I found this happens on rm version 5.2.1 running on RHEL4 | and 5.97 running on RHEL5. Consider upgrading - the latest stable version is 6.10 (and 6.11 is due out this month). | It does not specify anything about this | behavior in the man pages or other docs. Can you please look into this | and let me know about this behavior? POSIX documents it here: http://www.opengroup.org/onlinepubs/009695399/utilities/rm.html http://www.opengroup.org/onlinepubs/009695399/functions/unlink.html In particular, the EACCES errors on unlink() mention that without the sticky bit, all you need is write access to the directory (and your directory is world writable); with the sticky bit set, you must also own the directory and file. To set the sticky bit, use: chmod a+t dir or chmod 01777 dir - -- Don't work too hard, make some time for fun as well! Eric Blake [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Cygwin) Comment: Public key at home.comcast.net/~ericblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkgGmuoACgkQ84KuGfSFAYAIKACeJYuYGzX8qCLzpq2udRT8XCaz Ov4AnR8tXDVKaO9F5Jq46knYg2E/6s5+ =R2nF -END PGP SIGNATURE- ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: PATCH: md5sum --check did not work with "\r\n" and "\r" line endings
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to Simon Hengel on 4/16/2008 10:37 AM: | Hello list, | md5sum with the -c option did not work on files with "\r\n" and "\r" line | endings. | | A patch is attached. Thanks for the patch, but it corrupts actual \r in file names on platforms where \r is allowed. This issue has been raised previously on this list; what really needs to happen is that \r is always output encoded rather than literally, so that reading can then discard literal \r. I've had it on my TODO list to implement something along those lines, but it hasn't ever percolated to to the top. - -- Don't work too hard, make some time for fun as well! Eric Blake [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Cygwin) Comment: Public key at home.comcast.net/~ericblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkgGna4ACgkQ84KuGfSFAYAVYwCfWAu5WlbL8JIa5/B/zZz114MK X34AoM5VsXP9gjKYjdJMgCJxypCZCuj9 =dHfF -END PGP SIGNATURE- ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Enhancement request -- performance improvement
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to rh on 4/16/2008 12:14 PM: | Hello, | Running du -sk on an apache disk cache containing 10GB of data and | 30,000 directories and files | I see du using maybe .03% of the cpu. It takes an hour for it to complete. Probably because du is I/O intensive; the cpu is probably rather idle because most of your time is spent waiting on the disk to spin to the next inode to inspect. | | Are there any plans to make du multi-threaded? Or otherwise improve | it's performance? Is it filesystem sensitive? We are using ext3. I doubt multi-threading will help an I/O bound process. - -- Don't work too hard, make some time for fun as well! Eric Blake [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Cygwin) Comment: Public key at home.comcast.net/~ericblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkgGnhgACgkQ84KuGfSFAYAvZACcC9t8xv+zSxkiDaSvI1jtlzvt 5DsAoMmidIDE3kz120xUTc5tzHw5UQME =zqqN -END PGP SIGNATURE- ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: RM disregards file level permissions and uses directory permissions instead.
Eric Blake wrote: In particular, the EACCES errors on unlink() mention that without the sticky bit, all you need is write access to the directory (and your directory is world writable); with the sticky bit set, you must also own the directory and file. ^^^ To stave off confusion... I think you meant "or"? Surely POSIX doesn't state that I cannot delete my own file in a root-owned directory with go+w,+t set ;-). (The preceding sentence presumes that "I" am not root, of course.) -- Matthew > pinotree uses the large trout on tsdgeos and PutHuhn :) > PutHuhn runs > tsdgeos lights a fire and eats the trout (with apologies to Pino Toscano, PutHuhn and Albert Astals Cid, who came up with this entirely on their own) ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
su -l -c "command" fails to honor user's path
[EMAIL PROTECTED] build]# su -l zimbra [EMAIL PROTECTED] ~]$ echo $PATH /opt/zimbra/bin:/opt/zimbra/zimbramon:/opt/zimbra/postfix/sbin:/opt/zimbra/openldap/bin:/opt/zimbra/snmp/bin:/opt/zimbra/sleepycat/bin:/opt/zimbra/openssl/bin:/opt/zimbra/java/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin [EMAIL PROTECTED] ~]$ exit logout [EMAIL PROTECTED] build]# su -l zimbra -c "echo $PATH" /usr/kerberos/sbin:/usr/local/java/bin:/usr/local/ant/bin:/usr/local/mysql/bin:/usr/local/ant/bin:/usr/local/java/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/build/bin:/usr/local/p4/bin This means of course, that using "su -l -c command" will fail to execute commands that should be found in the user's path, necessitating that one use a fully qualified path to the command. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: su -l -c "command" fails to honor user's path
On Wed, Apr 16, 2008 at 9:55 PM, Quanah Gibson-Mount <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] build]# su -l zimbra -c "echo $PATH" > /usr/kerberos/sbin:/usr/local/java/bin:/usr/local/ant/bin:/usr/local/mysql/bin:/usr/local/ant/bin:/usr/local/java/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/build/bin:/usr/local/p4/bin > > This means of course, that using "su -l -c command" will fail to > execute commands that should be found in the user's path, necessitating that > one use a fully qualified path to the command. Greetings, You are using a double quote which will expand the $PATH variable before switching to the user zimbra. See this example: [EMAIL PROTECTED] ~]# su -l noland -c "echo $PATH" /usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin [EMAIL PROTECTED] ~]# su -l noland -c 'echo $PATH' /usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/noland/bin Here is another example which shows this: [EMAIL PROTECTED] ~]# echo su -l noland -c "echo $PATH" su -l noland -c echo /usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin [EMAIL PROTECTED] ~]# echo su -l noland -c 'echo $PATH' su -l noland -c echo $PATH Respectfully, Brock ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: su -l -c "command" fails to honor user's path
--On Wednesday, April 16, 2008 10:36 PM -0500 Brock Noland <[EMAIL PROTECTED]> wrote: Respectfully, Got it, thanks. The problem I'm having then, I see, is not related to the coreutils su, but specifically to the BSD su shipped with Darwin. Sorry for the noise. Linux: [EMAIL PROTECTED] quanah]# su - zimbra -c "zmlicense -h" usage: zmlicense [options] -c,--check check if a valid license is installed -h,--helpshow usage summary -i,--installinstall specified license file -l,--ldapinstall on ldap only -p,--print show items in installed license BSD: sh-3.2# su - zimbra -c "zmlicense -h" -bash: zmlicense: command not found --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc Zimbra :: the leader in open source messaging and collaboration ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: dd skip bug?
On Wed, Apr 16, 2008 at 7:01 PM, Pádraig Brady <[EMAIL PROTECTED]> wrote: > dd handles skip weirdly > > disk=/dev/sda8 > dd if=$disk bs=8M count=1 skip=1000 of=/dev/null #ok > dd if=$disk bs=8M count=1 skip=1000K of=/dev/null #reads whole disk! as seek > fails > > I had a 10s look at the source and noticed a comment > saying POSIX doesn't specify what we should do when > skipping past the end of input. For seekable files though, > reading the whole thing is unexpected to me at least. > I would expect it to do: > > if (seekable && !seek(skip_len)) > exit(EXIT_FAILURE); It looks me that the offset is too large? strace shows: _llseek(0, 8589934592000, 0xbff7efb0, SEEK_CUR) = -1 EINVAL (Invalid argument) Where as strace dd if=/dev/hda2 of=/dev/null bs=8M skip=10 shows: _llseek(0, 83886080, [83886080], SEEK_CUR) = 0 I agree that I would expect if the file is seekable and seek fails, dd would exit. But here it looks like we just cannot seek that large of an offset? Respectfully, Brock ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: RM disregards file level permissions and uses directory permissions instead.
Eric Blake wrote: > According to James J. Perry on 4/16/2008 4:25 PM: > | We are in the cutover process and one of the DBAs found this behavior. > | If testfile1 is owned by usera:group1 in a parent directory with > | permissions 777 owned by usera:group1, userb:group2 can delete testfile1 > | even if testfile1 has permissions 600. Conversely if the same parent > | directory has permissions 755 and userb owns testfile2 with 777 > | permissions they cannot delete their own file from that parent > | directory. > > Yes, and POSIX requires this. You probably want to set the directory's > sticky bit. Yes. And maybe. But I really wanted to explain it a different way. Things go into containers. In a Unix filesystem files go into directories. The directory is the container for files. Looking at it this way it should make sense that when creating, renaming/moving, or deleting files that it isn't the permission on the file that is important. It is the permission on the directory that controls all directory operations. The directory is the container that determines whether you can put things into it or take things out of it. If the directory is writable then then you can create, rename/move, delete files in it. If the directory isn't writable then you can't. Directories are really just special files that contain entries that point to other files. Enter the sticky bit. This causes special rules to come into play to restrict the definition of whether something is writable or not. It is "sticky" with regards to the classic Unix kernel and locking executables into swap space. That doesn't really apply any more but the name has stuck, pardon the pun. They needed a bit to indicate that a shared directory such as /tmp must be writable by everyone but must also keep everyone apart too. They reused the sticky bit for this other behavior. Permissions on the file control whether you can read or write the file. This is unrelated to directory permissions. This difference between directory permissions and file permissions is sometimes confusing to people. This is the Unix filesystem which has been standardized by POSIX. Other filesystems exist. The AFS filesystem is an example of a filesystem that doesn't use Unix permissions. AFS uses access control lists for everything and has a different set of permissions. I mention this to illustrate that filesystems implement a security policy layer. Not all are the same. But the Unix filesystem is the most common one on Unix-like systems and all others are very small niches only by comparison. > | I found this happens on rm version 5.2.1 running on RHEL4 > | and 5.97 running on RHEL5. > > Consider upgrading - the latest stable version is 6.10 (and 6.11 is due > out this month). Changing the rm command won't affect this at all. This feature has been stable for decades. It is controlled by the kernel filesystem and not by the rm command. Bob ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: Enhancement request -- performance improvement
rh <[EMAIL PROTECTED]> wrote: > Hello, > Running du -sk on an apache disk cache containing 10GB of data and > 30,000 directories and files > I see du using maybe .03% of the cpu. It takes an hour for it to complete. If you have an old version, and depending on how it's configured, ext3 can be very slow when operating on directories containing many entries. > Are there any plans to make du multi-threaded? No. That wouldn't help. > Or otherwise improve it's performance? Patches welcome, but it's already pretty well-optimized, since it uses fts. > Is it filesystem sensitive? We are using ext3. Yes. File system type and options will definitely affect performance. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: dd skip bug?
On Wed, Apr 16, 2008 at 11:30 PM, Brock Noland <[EMAIL PROTECTED]> wrote: > I agree that I would expect if the file is seekable and seek fails, dd > would exit. But here it looks like we just cannot seek that large of > an offset? I implemented something that seeks in portions and it results in the same error near the end of the file. _llseek(0, 1000, 0xbfe39670, SEEK_CUR) = -1 EINVAL (Invalid argument) As such my original thought was wrong... Respectfully, Brock ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: dd skip bug?
Pádraig Brady <[EMAIL PROTECTED]> wrote: > dd handles skip weirdly > > disk=/dev/sda8 > dd if=$disk bs=8M count=1 skip=1000 of=/dev/null #ok > dd if=$disk bs=8M count=1 skip=1000K of=/dev/null #reads whole disk! as seek > fails > > I had a 10s look at the source and noticed a comment > saying POSIX doesn't specify what we should do when > skipping past the end of input. For seekable files though, > reading the whole thing is unexpected to me at least. > I would expect it to do: > > if (seekable && !seek(skip_len)) > exit(EXIT_FAILURE); Thanks, but the existing behavior is deliberate, and IMHO, necessary. skip=N is required to try to seek, and failing that, position the read pointer by calling read. That is so it works on e.g., redirected stdin as well as on regular files. However, as that comment says, an attempt to go beyond EOF *might* be worth a warning. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
