bug#14190: cat command bug
Hello, Ctrl-c is behaving like Ctrl-d. When redirecting from standard-in to an existing file, Ctrl-c empties out the old file contents. In the past, Ctrl-c just aborts cat and leave the existing file unchanged. It happens in Fedora 17. Please fix. Thanks. -- Leow Hock Seng Tel:64463211 Mobile:96740759 http://www.oses.sg
bug#14189: ls -d bug ??
On 04/12/2013 08:06 AM, Bob Proulx wrote: > Some local wordsmithing turned out the following as a better > improvement. It lists what it does in the positive first. And > removes the negative which was seen as being too confusing. > > `-d' > `--directory' > List only the name of directories, not the contents. This is > most typically used with `-l' to list the information for the > directory itself instead of its contents. Do not follow symbolic > links unless the `--dereference-command-line' (`-H'), > `--dereference' (`-L'), or > `--dereference-command-line-symlink-to-dir' options are > specified. Overrides `--recursive', (`-R'). Not bad, but I'm still missing the point that `-d' changes ls's behavior for *directory arguments* only. Furthermore, I don't think mentioning `-l' is of much relevance here. So this would melt down the first two sentences as follows: `-d' `--directory' For directory arguments, list only the information for the directory itself instead of its contents. Do not follow symbolic links unless the `--dereference-command-line' (`-H'), `--dereference' (`-L'), or `--dereference-command-line-symlink-to-dir' options are specified. Overrides `--recursive', (`-R'). And what about the usage() string? I'd bet this is still 95% where users are looking for. Something like the following perhaps? - -d, --directorylist directory entries instead of contents, - and do not dereference symbolic links + -d, --directoryfor directory arguments, list the entry itself + instead of contents, and do not dereference + symbolic links Have a nice day, Berny
bug#14189: ls -d bug ??
Eric Blake writes: > bash-specific: > $ (shopt -s nullglob; ls -d */ .[!.]/ .??*/) $ (shopt -s dotglob; ls -d */) Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."
bug#14190: cat command bug
tag 14190 notabug close 14190 stop On 04/12/2013 03:29 AM, hockseng leow wrote: > Hello, > > Ctrl-c is behaving like Ctrl-d. > > When redirecting from standard-in to an existing file, Ctrl-c empties > out the old file contents. In the past, Ctrl-c just aborts cat and > leave the existing file unchanged. > > It happens in Fedora 17. > > Please fix. Please give your exact command line if you still think this is an issue, but I suspect your shell is truncating the file before cat runs. thanks, Pádraig.
bug#13530: head: memory exhausted when printing all from stdin but last P/E bytes
On 01/23/2013 12:34 PM, Pádraig Brady wrote:
> On 01/23/2013 02:32 AM, Lei Zhang wrote:
>> Hi All,
>>
>> We found a bug in the `head' program of coreutils 8.20:
>>
>> Invoking `head -c -P' or `head -c -E' will cause memory exhaustion.
>>
>> However, smaller units (e.g., b, K, M) work fine; bigger units (e.g., Z, Y)
>> fail properly
>> (by outputing "number of bytes is so large that it is not representable").
>> And `-n' also
>> works fine.
>>
>> A bit dig reveals that the bug is introduced at line 322 of head.c
>> (coreutils 8.20):
>>
>> 204: elide_tail_bytes_pipe (const char *filename, int fd, uintmax_t
>> n_elide_0)
>> 205: {
>> 322: b = xcalloc (n_bufs, sizeof *b); /** this statement fails **/
>> 398: }
>>
>> We also examined n_elide and n_bufs before that statement. Actually, they
>> are very
>> large:
>>
>> n_elide: 1125899906842624
>> n_bufs: 137438953474
>>
>> According to the following comment in the source file:
>>
>>> CAUTION: do not fail (out of memory) when asked to elide
>>> a ridiculous amount, but when given only a small input. */
>>
>> We think this is a bug and bring this issue to your attention. Thanks!
>
> There is the argument that we _should_ allocate
> everything up front to indicate immediately
> that the system can't (currently) support the requested operation,
> but given the 'currently' caveat above I guess it's
> more general to fail when we actually run out of mem?
>
> So to do that we can either realloc our pointer array
> in chunks or use the simpler approach in the patch below,
> and take advantage of kernel overcommit strategies.
> (calloc is problematic for overcommit as zeroing the
> memory fully allocates it to the process).
> The caveat with that though is that it's dependent
> on the overcommit config for the kernel and possibly
> current memory conditions.
>
> thanks,
> Pádraig.
>
> diff --git a/src/head.c b/src/head.c
> index cf84a95..909be04 100644
> --- a/src/head.c
> +++ b/src/head.c
> @@ -197,7 +197,7 @@ copy_fd (int src_fd, FILE *o_stream, uintmax_t n_bytes)
>return COPY_FD_OK;
> }
>
> -/* Print all but the last N_ELIDE lines from the input available via
> +/* Print all but the last N_ELIDE bytes from the input available via
> the non-seekable file descriptor FD. Return true upon success.
> Give a diagnostic and return false upon error. */
> static bool
> @@ -314,18 +314,22 @@ elide_tail_bytes_pipe (const char *filename, int fd,
> uintmax_t n_elide_0)
>size_t n_read;
>bool buffered_enough;
>size_t i, i_next;
> + size_t n_alloc = 0;
>char **b;
>/* Round n_elide up to a multiple of READ_BUFSIZE. */
>size_t rem = READ_BUFSIZE - (n_elide % READ_BUFSIZE);
>size_t n_elide_round = n_elide + rem;
>size_t n_bufs = n_elide_round / READ_BUFSIZE + 1;
> - b = xcalloc (n_bufs, sizeof *b);
> + b = xnmalloc (n_bufs, sizeof *b);
>
>buffered_enough = false;
>for (i = 0, i_next = 1; !eof; i = i_next, i_next = (i_next + 1) %
> n_bufs)
> {
> - if (b[i] == NULL)
> -b[i] = xmalloc (READ_BUFSIZE);
> + if (! buffered_enough)
> +{
> + b[i] = xmalloc (READ_BUFSIZE);
> + n_alloc = i + 1;
> +}
>n_read = full_read (fd, b[i], READ_BUFSIZE);
>if (n_read < READ_BUFSIZE)
> {
> @@ -389,7 +393,7 @@ elide_tail_bytes_pipe (const char *filename, int fd,
> uintmax_t n_elide_0)
> }
>
> free_mem:
> - for (i = 0; i < n_bufs; i++)
> + for (i = 0; i < n_alloc; i++)
> free (b[i]);
>free (b);
I expect to push soon, the attached more complete fix to realloc the array.
thanks,
Pádraig.
>From 57d55485f0d2014c076f2cbfe0b340d1f2046952 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?=
Date: Wed, 23 Jan 2013 12:34:46 +
Subject: [PATCH] head: with --bytes=-N only allocate memory as needed
* src/head.c (elide_tail_bytes_pipe): Don't use calloc as that
bypasses memory overcommit due to the zeroing requirement.
Also realloc rather than malloc the pointer array to avoid
dependence on overcommit entirely.
* tests/misc/head-c.sh: Add a test case.
Fixes http://bugs.gnu.org/13530
---
src/head.c | 28 ++--
tests/misc/head-c.sh |9 +++--
2 files changed, 29 insertions(+), 8 deletions(-)
diff --git a/src/head.c b/src/head.c
index d79d5f7..00e1be1 100644
--- a/src/head.c
+++ b/src/head.c
@@ -196,7 +196,7 @@ copy_fd (int src_fd, FILE *o_stream, uintmax_t n_bytes)
return COPY_FD_OK;
}
-/* Print all but the last N_ELIDE lines from the input available via
+/* Print all but the last N_ELIDE bytes from the input available via
the non-seekable file descriptor FD. Return true upon success.
Give a diagnostic and return false upon error. */
static bool
@@ -313,18 +313,34 @@ elide_tail_bytes_pipe (const char *filename, int fd, uintmax_t n_elide_0)
size_t n_rea
bug#13530: head: memory exhausted when printing all from stdin but last P/E bytes
Pádraig Brady wrote: ... > I expect to push soon, the attached more complete fix to realloc the array. Thanks! That change looks fine and passed tests here, too.
bug#13530: head: memory exhausted when printing all from stdin but last P/E bytes
On 04/12/2013 09:06 PM, Jim Meyering wrote: > Pádraig Brady wrote: > ... >> I expect to push soon, the attached more complete fix to realloc the array. > > Thanks! That change looks fine and passed tests here, too. +1 Have a nice day, Berny
bug#14024: Test failure in coreutils 8.13
Pádraig, Following your points below about groups, I checked the owner/groups of the various directories where I variously ran and repeated the coreutils tests and did sequences for you like: echo test > a /usr/local/bin/install -Cv -m0644 a b /usr/local/bin/stat a b /usr/local/bin/install -Cv -m0644 a b I now see why my latest ones gave different results from the earlier ones, and why the earlier ones did not seem to match the reported test failures. The earlier sequences were run in my own directory ~/tmp, so that the files a and b were in a temporary directory. However, the latest ones, that responded with: removed 'b' 'a' -> 'b' were run where I had just rerun the "make check TESTS=tests/install/install-C.sh VERBOSE=yes SUBDIRS=." which is /Gnu/coreutils/coreutils-8.21. This dir hierarchy is not g+s to a group other than my own, but does have group values that are not my own. There do not seem to be ACLs on these directories, but some have "extended attributes". I was not able to use getfacl on this system, but ls -e shows ACLs, and if the file or directory has extended security information, the permissions field printed by the -l option is followed by a '+' character (from man ls). My user on this iMac has Administrator privileges, which in practice means that it is in group 'admin', and several other groups. These users have the ability to create "top-level" directories, like /Gnu. This ability seems to arise because '/' has group 'admin', and the created directories do too, but are owned by the creator (the result seems to be the same whether the directory is created by a Unix mkdir, or using the Mac gui Finder application): ls -Ald@ / /Gnu /Gnu/coreutils/ /Gnu/coreutils/coreutils-8.21 drwxrwxr-t 45 root admin 1598 Apr 12 20:40 / drwxrwxr-x 12 admin admin 408 Feb 16 12:12 /Gnu drwxr-xr-x 11 ellisnthomas admin 374 Apr 12 20:42 /Gnu/coreutils/ drwxr-xr-x@ 57 ellisnthomas admin 1938 Apr 11 15:27 /Gnu/coreutils/ coreutils-8.21 com.apple.quarantine 78 (I am not clear what this extended attribute does. It seems related to Mac regarding downloaded files with suspicion - the directory tree was originally extracted from /Gnu/coreutils/coreutils-8.21.tar.gz) In my own directories, the group is staff: ls -Ald ~ drwxr-xr-x+ 53 ellisnthomas staff 1802 Apr 12 20:04 /Users/ ellisnthomas ls -Alde ~/tmp/ drwxr-xr-x 8 ellisnthomas staff 272 Mar 27 20:59 / Users/ellisnthomas/tmp/ To clarify the gids and uids I checked the groups, using dscacheutil (man says "dscacheutil does various operations against the Directory Service cache including gathering statistics, initiating lookups, inspection, cache flush, etc. This tool replaces most of the functionality of the lookupd tool previously available Darwin Jan 14, 2007") One aspect here is that there is both a user 'admin' and a group 'admin'. It seems that user admin created the directory /Gnu, but user ellisnthomas created /Gnu/coreutils/ later. dscacheutil -q group -a name admin name: admin password: * gid: 80 users: root ellisnthomas admin dscacheutil -q group -a gid 20 name: staff password: * gid: 20 users: root ellisnthomas admin susanthomas dscacheutil -q user -a name ellisnthomas name: ellisnthomas password: uid: 501 gid: 20 dir: /Users/ellisnthomas shell: /bin/bash gecos: Ellis Thomas dscacheutil -q user -a name admin name: admin password: uid: 502 gid: 20 dir: /Users/admin shell: /bin/bash gecos: ExtraLeveLInSoftware The usage of getfacl is not possible, but tried ls -e. type -a getfacl bash: type: getfacl: not found ls -Alde /Gnu/coreutils/coreutils-8.21 drwxr-xr-x@ 57 ellisnthomas admin 1938 Apr 11 15:27 /Gnu/coreutils/ coreutils-8.21 ls -Alde /Gnu/ drwxrwxr-x 12 admin admin 408 Feb 16 12:12 /Gnu/ ls -Alde ~/tmp/ drwxr-xr-x 8 ellisnthomas staff 272 Mar 27 20:59 /Users/ ellisnthomas/tmp/ ls -Alde ~ drwxr-xr-x+ 53 ellisnthomas staff 1802 Apr 12 09:01 /Users/ ellisnthomas 0: group:everyone deny delete This seems to be the only one with an ACL. So in summary it seems that install is seeing the two different groups: 'admin' in the existing 'a' file, and expecting to set 'staff' in the new 'b' file as you suggested, but because the parent directory already has 'admin' without either the g+s or ACL involvement. It seems it is now explained, anyway. Ellis On 12 Apr 2013, at 02:37, Pádraig Brady wrote: Great thanks. That shows that the gid of the files is 80, which I presume is separate to your gid. That can happen if you're in a dir hierarchy that's g+s to a group other than your own. Hmm I see the test already detects this case and skips the test with skip_if_setgid_(). Perhaps POSIX default ACLs or something are setting this admin group for you? Can you confirm that the dir isn't setgid
bug#14024: Test failure in coreutils 8.13
On 04/12/2013 10:30 PM, Ellis N. Thomas wrote: > On 12 Apr 2013, at 02:37, Pádraig Brady wrote: > >> Great thanks. >> >> That shows that the gid of the files is 80, which I presume is >> separate to your gid. That can happen if you're in a dir hierarchy >> that's g+s to a group other than your own. >> Hmm I see the test already detects this case and skips the test >> with skip_if_setgid_(). Perhaps POSIX default ACLs or something are >> setting this admin group for you? >> Can you confirm that the dir isn't setgid by showing the output of: >> /usr/local/bin/stat . >> Can you display ACLs with `getfacl .` ? > > Pádraig, > > Following your points below about groups, I checked the owner/groups > of the various directories where I variously ran and repeated the coreutils > tests > and did sequences for you like: > > echo test > a > /usr/local/bin/install -Cv -m0644 a b > /usr/local/bin/stat a b > /usr/local/bin/install -Cv -m0644 a b > > I now see why my latest ones gave different results from the earlier ones, > and why the earlier ones did not seem to match the reported test failures. > > The earlier sequences were run in my own directory ~/tmp, so that the > files > a and b were in a temporary directory. However, the latest ones, that > responded > with: > removed 'b' > 'a' -> 'b' > were run where I had just rerun the > "make check TESTS=tests/install/install-C.sh VERBOSE=yes SUBDIRS=." > which is /Gnu/coreutils/coreutils-8.21. This dir hierarchy is not g+s to a > group other than > my own, but does have group values that are not my own. There do not seem to > be ACLs > on these directories, but some have "extended attributes". > > I was not able to use getfacl on this system, but ls -e shows ACLs, and > if the file or > directory has extended security information, the permissions field printed by > the -l option > is followed by a '+' character (from man ls). > > My user on this iMac has Administrator privileges, which in practice > means that > it is in group 'admin', and several other groups. These users have the > ability to create > "top-level" directories, like /Gnu. This ability seems to arise because '/' > has group > 'admin', and the created directories do too, but are owned by the creator > (the result > seems to be the same whether the directory is created by a Unix mkdir, or > using the > Mac gui Finder application): > > ls -Ald@ / /Gnu /Gnu/coreutils/ /Gnu/coreutils/coreutils-8.21 > drwxrwxr-t 45 root admin 1598 Apr 12 20:40 / > drwxrwxr-x 12 admin admin 408 Feb 16 12:12 /Gnu > drwxr-xr-x 11 ellisnthomas admin 374 Apr 12 20:42 /Gnu/coreutils/ > drwxr-xr-x@ 57 ellisnthomas admin 1938 Apr 11 15:27 > /Gnu/coreutils/coreutils-8.21 > com.apple.quarantine 78 > > (I am not clear what this extended attribute does. It seems related to > Mac regarding > downloaded files with suspicion - the directory tree was originally extracted > from > /Gnu/coreutils/coreutils-8.21.tar.gz) > > In my own directories, the group is staff: > ls -Ald ~ > drwxr-xr-x+ 53 ellisnthomas staff 1802 Apr 12 20:04 /Users/ellisnthomas > > ls -Alde ~/tmp/ drwxr-xr-x 8 ellisnthomas staff 272 Mar 27 20:59 > /Users/ellisnthomas/tmp/ > > To clarify the gids and uids I checked the groups, using dscacheutil > (man says "dscacheutil does various operations against the Directory Service > cache > including gathering statistics, initiating lookups, inspection, cache > flush, etc. This tool replaces most of the functionality of the lookupd > tool previously available > Darwin Jan 14, 2007") > > One aspect here is that there is both a user 'admin' and a group 'admin'. > It seems that user admin created the directory /Gnu, but user ellisnthomas > created /Gnu/coreutils/ later. > > dscacheutil -q group -a name admin > name: admin > password: * > gid: 80 > users: root ellisnthomas admin > > dscacheutil -q group -a gid 20 > name: staff > password: * > gid: 20 > users: root ellisnthomas admin susanthomas > > dscacheutil -q user -a name ellisnthomas > name: ellisnthomas > password: > uid: 501 > gid: 20 > dir: /Users/ellisnthomas > shell: /bin/bash > gecos: Ellis Thomas > > dscacheutil -q user -a name admin > name: admin > password: > uid: 502 > gid: 20 > dir: /Users/admin > shell: /bin/bash > gecos: ExtraLeveLInSoftware > > The usage of getfacl is not possible, but tried ls -e. > type -a getfacl > bash: type: getfacl: not found > > ls -Alde /Gnu/coreutils/coreutils-8.21 > drwxr-xr-x@ 57 ellisnthomas admin 1938 Apr 11 15:27 > /Gnu/coreutils/coreutils-8.21 > > ls -Alde /Gnu/ > drwxrwxr-x 12 admin admin 408 Feb 16 12:12 /Gnu/ > > ls -Alde ~/tmp/ > drwxr-xr-x 8 ellisnthomas staff 272 Mar 27 20:59 /Users/ellisnthomas/tmp/ > > ls -Alde ~ > drwxr-xr-x+ 53 ellisnthomas staff 1802 Apr 12 09:01 /Users/ellisnthomas > 0: group:everyone deny delete > > This seems to be the only one with an ACL.
