Re: [PATCH] cp: ignore obscure failure to preserve symlink time stamps,
Jim Meyering wrote:
> The "preserve symlink time stamps" feature in coreutils-7.5
> is causing trouble in Fedora's build system,
>
> http://thread.gmane.org/gmane.linux.redhat.fedora.devel/119834
>
> because they use a kernel without utimensat support, yet configured/built
> coreutils-7.5 in an environment that suggests (via link tests) that the
> function is available. Here is the fix I expect to push soon:
> diff --git a/src/copy.c b/src/copy.c
> index bf9230b..8fc4b68 100644
> --- a/src/copy.c
> +++ b/src/copy.c
> @@ -124,7 +124,13 @@ static inline int
> utimens_symlink (char const *file, struct timespec const *timespec)
> {
> #if HAVE_UTIMENSAT
> - return utimensat (AT_FDCWD, file, timespec, AT_SYMLINK_NOFOLLOW);
> + int err = utimensat (AT_FDCWD, file, timespec, AT_SYMLINK_NOFOLLOW);
> + /* When configuring on a system with new headers and libraries, and
> + running on one with a kernel that is old enough to lack the syscall,
> + utimensat fails with ENOTSUP. Ignore that. */
> + if (err && errno == ENOSYS)
> +err = 0;
> + return err;
I was worried about that but thought it wasn't an issue
as ENOTSUP was not mentioned in the man page :(
http://www.kernel.org/doc/man-pages/online/pages/man2/utimensat.2.html
Michael can we add ENOTSUP to utimensat(2) ?
cheers,
Pádraig.
Re: [PATCH] cp: ignore obscure failure to preserve symlink time stamps,
Pádraig Brady wrote:
> Jim Meyering wrote:
>> The "preserve symlink time stamps" feature in coreutils-7.5
>> is causing trouble in Fedora's build system,
>>
>> http://thread.gmane.org/gmane.linux.redhat.fedora.devel/119834
>>
>> because they use a kernel without utimensat support, yet configured/built
>> coreutils-7.5 in an environment that suggests (via link tests) that the
>> function is available. Here is the fix I expect to push soon:
>
>> diff --git a/src/copy.c b/src/copy.c
>> index bf9230b..8fc4b68 100644
>> --- a/src/copy.c
>> +++ b/src/copy.c
>> @@ -124,7 +124,13 @@ static inline int
>> utimens_symlink (char const *file, struct timespec const *timespec)
>> {
>> #if HAVE_UTIMENSAT
>> - return utimensat (AT_FDCWD, file, timespec, AT_SYMLINK_NOFOLLOW);
>> + int err = utimensat (AT_FDCWD, file, timespec, AT_SYMLINK_NOFOLLOW);
>> + /* When configuring on a system with new headers and libraries, and
>> + running on one with a kernel that is old enough to lack the syscall,
>> + utimensat fails with ENOTSUP. Ignore that. */
>> + if (err && errno == ENOSYS)
>> +err = 0;
>> + return err;
>
> I was worried about that but thought it wasn't an issue
> as ENOTSUP was not mentioned in the man page :(
> http://www.kernel.org/doc/man-pages/online/pages/man2/utimensat.2.html
>
> Michael can we add ENOTSUP to utimensat(2) ?
Actually, the comment I added above has a typo.
s/ENOTSUP/ENOSYS/.
The diagnostic reported corresponds to ENOSYS.
The adjusted patch is here:
http://www.redhat.com/archives/fedora-devel-list/2009-August/msg01220.html
[RFC] new open flag O_NOSTD
Proposal Add a new flag, O_NOSTD, to at least open and pipe2 (and an alternate spelling SOCK_NOSTD for socket, socketpair, accept4), with the following semantics: If the flag is specified and the function is successful, the returned fd (both fds for the pipe2 case) will be at least 3, regardless of whether the standard file descriptors 0, 1, or 2 are currently closed. Rationale = GNU Coreutils tries hard to protect itself from whatever weird environment may be thrown at it. One example is if the user runs: cp a b 2>&- If cp encounters an error, it prints a message to stderr, then regardless of whether the message was successfully printed, cp guarantees a non-zero exit status. In the case where fd 2 starts life closed, however, a naive implementation could end up opening a destination file for writing as fd 2, then encounter an error, such that the first use of stderr to print an error message will incorrectly modify the contents of a completely unrelated file. Therefore, the best approach for cp to take is to ensure that command-line arguments never occupy fd 0, 1, or 2, no matter what the cp process inherited from its parent. Of course, if cp were installed set-user-ID Or set-group-ID, then the OS could guarantee that cp would never start life with fd 0, 1, or 2 closed; but cp should not normally be installed with these permissions, and POSIX does not permit the OS to arbitrarily open these fds if these permissions are not present. One option is for cp to manually guarantee that fd 0, 1, and 2 are opened prior to parsing command line options. At one point, coreutils even used this approach, via a function stdopen: http://git.savannah.gnu.org/cgit/coreutils.git/diff/lib/stdopen.c?id=875cae47 However, this has a couple of drawbacks. It costs several syscalls at startup, even in the common case of all three std descriptors being provided by the parent process. It also ties up otherwise unused open file descriptors (perhaps the user intentionally closed some of the std fds in order to provide room for allowing more simultaneously open files without hitting EMFILE limits). Another option is what cp currently uses, which guarantees that any function call that creates a new fd is wrapped by a *_safer variant, which guarantees that the result will never collide with the standard descriptors. In the common case, the original open() returns 3 or larger, so the wrapper has no additional work to perform. But if the user started cp with fd 0, 1, or 2 closed, then the current implementation of the open_safer wrapper notices that the underlying open() call is in the wrong range, and provides a followup call to fcntl(fd,F_DUPFD,3) and close(fd), such that the overall result is again safely out of the std fd range: http://git.savannah.gnu.org/cgit/gnulib.git/tree/lib/open-safer.c http://git.savannah.gnu.org/cgit/gnulib.git/tree/lib/fd-safer.c http://git.savannah.gnu.org/cgit/gnulib.git/tree/lib/dup-safer.c Notice that with coreutils' current approach, the common case (all std descriptors provided by the parent) uses the minimal number of syscalls. However, in the corner case of starting life with a standard descriptor closed, the number of additional fcntl(F_DUPFD)/close() calls cause noticeable slowdown when copying large hierarchies (especially when compared with the stdopen approach of only suffering an up-front syscall penalty). And while coreutils does not keep fd 0, 1, or 2 tied open on a useless file all the time, it is still putting pressure on these descriptors during the window of the open_safer wrapper, so it has not completely eliminated the EMFILE avoidance. Also, the coreutils' approach works well for a single-threaded application, but it needs modifications to use the recently added POSIX 2008 open(O_CLOEXEC) and fcntl(F_DUPFD_CLOEXEC) flags if it is to avoid leaking a temporary fd 0, 1, or 2 into child process created by a fork/exec in another thread during the time that the first thread is calling open_safer. Therefore it makes sense to move this functionality into the kernel, via the addition of a new open() flag that informs the kernel that a successful fd-creation syscall must behave as if fd 0, 1, and 2 were already open. The idea is not new, since fcntl(fd, F_DUPFD, 3) already does just this. Then, on kernels where this is available, coreutils can alter its open_safer function to pass the new flag to the underlying open() syscall, and avoid having to use fcntl/close to sanitize any returned fd, with the result of no difference in the number of syscalls regardless of whether the parent process started cp with stderr open or closed. It also solves the EMFILE and multithreading fd leak issue, since a temporary fd 0, 1, or 2 is never opened in the first place. The name proposed in this mail is O_NOSTD (implying that a successful result will not be any of the standard file descriptors); other ideas mentioned on the bug-gnulib list were O_SAFER, O_NONSTD, O_NOSTDFD.
Re: [PATCH] cp: ignore obscure failure to preserve symlink time stamps,
On Monday 24 of August 2009 10:46:26 Jim Meyering wrote: > Actually, the comment I added above has a typo. > s/ENOTSUP/ENOSYS/. > > The diagnostic reported corresponds to ENOSYS. > The adjusted patch is here: > > > http://www.redhat.com/archives/fedora-devel-list/2009-August/msg01220.html s/Kamil Dudka/Jeff Garzik/ Kamil
Re: [PATCH] cp: ignore obscure failure to preserve symlink time stamps,
Kamil Dudka wrote: > On Monday 24 of August 2009 10:46:26 Jim Meyering wrote: >> Actually, the comment I added above has a typo. >> s/ENOTSUP/ENOSYS/. >> >> The diagnostic reported corresponds to ENOSYS. >> The adjusted patch is here: >> >> >> http://www.redhat.com/archives/fedora-devel-list/2009-August/msg01220.html > > s/Kamil Dudka/Jeff Garzik/ No. Someone reported a koji build failure last week. If it wasn't you, then it must have been Ondřej. Sorry about that.
Re: [PATCH] cp: ignore obscure failure to preserve symlink time stamps,
On Monday 24 of August 2009 14:45:58 Jim Meyering wrote: > Kamil Dudka wrote: > > On Monday 24 of August 2009 10:46:26 Jim Meyering wrote: > >> Actually, the comment I added above has a typo. > >> s/ENOTSUP/ENOSYS/. > >> > >> The diagnostic reported corresponds to ENOSYS. > >> The adjusted patch is here: > >> > >> > >> http://www.redhat.com/archives/fedora-devel-list/2009-August/msg01220.ht > >>ml > > > > s/Kamil Dudka/Jeff Garzik/ > > No. Someone reported a koji build failure last week. > If it wasn't you, then it must have been Ondřej. > Sorry about that. Then it was Ondra :-) He is working on the new version of Fedora coreutils now... Kamil
Re: [PATCH] cp: ignore obscure failure to preserve symlink time stamps,
Jim Meyering wrote: > Kamil Dudka wrote: > > On Monday 24 of August 2009 10:46:26 Jim Meyering wrote: > >> Actually, the comment I added above has a typo. > >> s/ENOTSUP/ENOSYS/. > >> > >> The diagnostic reported corresponds to ENOSYS. > >> The adjusted patch is here: > >> > >> > >> http://www.redhat.com/archives/fedora-devel-list/2009-August/msg01220.html > > > > s/Kamil Dudka/Jeff Garzik/ > > No. Someone reported a koji build failure last week. > If it wasn't you, then it must have been Ondřej. > Sorry about that. Yep, it was me, I reported that on Friday just before I left office - but actually my workaround for koji caused the build failure reported on fedora-devel-list by Todd and I thought Kamil reported privately that on weekend so I was silent about the patch. Adjusted patch is already built in fedora rawhide so everything should be fine now. Ondřej signature.asc Description: Toto je digitálně podepsaná část zprávy
Re: Bug#505927: just use the date(1) -d library instead of your own poorer date parser
> Regarding http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=505927 > "BSB" == Bernd Siggy Brentrup writes: BSB> Hi, BSB> I'm currently evaluating at's wishlist bugs for my 'at' replacement BSB> 'at-ng' which is a complete rewrite from scratch. Uh oh, "at no good"? :-) BSB> On Mon, Nov 17, 2008 at 05:49 +0800, jida...@jidanni.org wrote: >> Package: at >> Version: 3.1.10.2 >> Severity: wishlist >> >> At should be more flexible. Just use the date libraries instead of >> your own parser. >> $ at -v 'now + 5 years + 11 months' >> syntax error. Last token seen: + >> Garbled time >> $ date -d 'now + 5 years + 11 months' >> Fri Oct 17 03:53:15 CST 2014 >> However to get at to accept such a date, one needs: >> $ at -v $(date --rfc-3339=date -d 'now + 5 years + 11 months') >> Fri Oct 17 03:56:00 2014 >> warning: commands will be executed using /bin/sh >> If at would use the same library as date -d, you could 1. parse lots >> more types of dates. 2. Eliminate maintenance of duplicate code. BSB> If there only were such a library, static or preferrably dynamic! BSB> ldd /bin/date shows there is no dynamic one and dpkg -L coreutils shows BSB> no static version either. BSB> I might get coreutils sources and use the relevant parts but that BSB> deprives me of my freedom to choose a license at my will. IANAL but BSB> in my understanding using GPLed source code means you must release BSB> everything under the GPL. I'll Cc the coreutils people and thus hook you up so you fellows can figure out the best way to reuse code. I'm a big Stallman http://jidanni.org/comp/index.html#rms fan, so any license of his is good with me. BSB> I'm not yet decided what to do, in particular when thinking about BSB> i18n which may demand to cope with cultural differences. How does BSB> date handle these? I recall its output was better than its input, but then on your at(1) man page you can say "if at(1) can't parse Chinese dates yet, blame date(1)"! BSB> Regards BSB> Siggy BSB> [1] https://launchpad.net/~at-ng BSB> not much there for now, but by next week there will be BSB> demos for the cli commands at & friends. Server-side BSB> will take somewhat longer.
