bug#13172: tail: unrecognized file system type

[Bernhard Voelker]

On 12/13/2012 02:37 PM, coni wrote:
> Hi i've got a warning message in tail command:
> "tail -f /vc/log/apache2/access_log" - report to bug.
>  
> tail (GNU coreutils) 8.20
>  Packaged by Gentoo (8.20 (p1.0))
>  
> - file system type ceph
>  
> message:
>  
> tail: unrecognized file system type 0x00c36400 for 
> '/vc/log/apache2/access_log'.
> please report this to bug-coreutils@gnu.org. reverting to polling
>  
> Regards:
> Conrad.

Thanks for the report.

As a distributed network file system, CEPH seems to be a "remote"
file system, i.e. not supporting inotify.

Conrad, would you mind giving us your full name, so that we can
mention it correctly in THANKS.in?

Here comes the patch.

Have a nice day,
Berny


>From 5997134899c9abd5efd635a446fb1a3975483a82 Mon Sep 17 00:00:00 2001
From: Bernhard Voelker 
Date: Fri, 14 Dec 2012 10:11:51 +0100
Subject: [PATCH] tail,stat: improve support for the ceph file system

Teach tail -f that it must use polling on ceph file systems, and
let stat -f --format=%T report the file system type name, "ceph".
Website: http://ceph.com/

* src/stat.c (human_fstype): Add a case: ceph, 0x00c36400, remote.
* NEWS (Improvements): Mention it.
* THANKS.in: Update.
Reported by Conrad  in http://bugs.gnu.org/13172.
---
 NEWS   |6 ++
 THANKS.in  |1 +
 src/stat.c |2 ++
 3 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/NEWS b/NEWS
index e22d3a9..1ee2c17 100644
--- a/NEWS
+++ b/NEWS
@@ -59,6 +59,12 @@ GNU coreutils NEWS-*- 
outline -*-
   nl no longer supports the --page-increment option which was deprecated
   since coreutils-7.5.  Use --line-increment instead.

+** Improvements
+
+  stat and tail now know about CEPH.  stat -f --format=%T now reports the file
+  system type, and tail -f uses polling for files on CEPH file systems.
+
+
 ** Build-related

   Perl is now more of a prerequisite.  It has long been required in order
diff --git a/THANKS.in b/THANKS.in
index f0ef9b8..802e52d 100644
--- a/THANKS.in
+++ b/THANKS.in
@@ -129,6 +129,7 @@ Clark Morgancmor...@aracnet.com
 Clement Wangclem.w...@overture.com
 Colin Plumb co...@nyx.net
 Collin Rogowski col...@rogowski.de
+Conrad ??   c...@o2.pl
 Cray-Cyber Project  http://www.cray-cyber.org
 Cristian Cadar  cris...@stanford.edu
 Cyril Bouthors  cy...@bouthors.org
diff --git a/src/stat.c b/src/stat.c
index 51f4b90..6283566 100644
--- a/src/stat.c
+++ b/src/stat.c
@@ -266,6 +266,8 @@ human_fstype (STRUCT_STATVFS const *statfsbuf)
   return "binfmt_misc";
 case S_MAGIC_BTRFS: /* 0x9123683E local */
   return "btrfs";
+case S_MAGIC_CEPH: /* 0x00c36400 remote */
+  return "ceph";
 case S_MAGIC_CGROUP: /* 0x0027E0EB local */
   return "cgroupfs";
 case S_MAGIC_CIFS: /* 0xFF534D42 remote */
-- 
1.7.7

bug#13172: tail: unrecognized file system type

[Pádraig Brady]

On 12/14/2012 09:18 AM, Bernhard Voelker wrote:

+case S_MAGIC_CEPH: /* 0x00c36400 remote */
+  return "ceph";


s/00c/00C/ for consistency

thanks for handling this.
Pádraig.

bug#13183: tail -f ignores SIGPIPE

[Ruediger Meier]

Hi,

I want to use tail and grep to follow a file until a particular pattern 
appears. But tail does not exit when grep is finished.

$ echo xxx > /tmp/blabla
$ tail -f /tmp/blabla |grep -m1 --line-buffered "xxx"
xxx

Now tail still tries to read and exits only if I write again 
into /tmp/blabla.

Is this how it's supposed to be?

cu,
Rudi

bug#13183: tail -f ignores SIGPIPE

[Pádraig Brady]

tag 13183 + notabug
close 13183
stop

On 12/14/2012 02:04 PM, Ruediger Meier wrote:

Hi,

I want to use tail and grep to follow a file until a particular pattern
appears. But tail does not exit when grep is finished.

$ echo xxx > /tmp/blabla
$ tail -f /tmp/blabla |grep -m1 --line-buffered "xxx"
xxx

Now tail still tries to read and exits only if I write again
into /tmp/blabla.

Is this how it's supposed to be?


tail does exit on SIGPIPE, however it will
only get the signal on write(), and so you
need to get more data in the file before tail will exit.

thanks,
Pádraig.

bug#13184: Split german help typo

[Gregor]

Hi, whilst using split i stumbled over a typo in the german
command-description from
split --help

ABSCHITTE können sein:
should be
ABSCHNITTE können sein:

Many thanks for the nice program,
Gregor

bug#13184: Split german help typo

[Eric Blake]

On 12/14/2012 06:20 AM, Gregor wrote:
> Hi, whilst using split i stumbled over a typo in the german
> command-description from
> split --help
> 
> ABSCHITTE können sein:
> should be
> ABSCHNITTE können sein:

Such reports should be sent to the translator; I'm cc'ing the address
mentioned in 'LC_ALL=de_DE split --help'.

coreutils.git does not carry the translation, but rather picks up
whatever latest version the translation team provides, so once it is
fixed by the translation team, the next coreutils release will be fixed.
 As such, I'm closing this bug in the coreutils database so it does not
affect statistics of bugs that still need work in coreutils.git; but you
should feel free to continue to reply to this bug as needed to provide
any further information.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

bug#13185: Test case 'misc/timeout-group' failed

[Stefano Lattarini]

On 12/14/2012 06:28 PM, Stefano Lattarini wrote:
> While running the coreutils testsuite on my oldish Debian desktop with
> a somewhat heavy load and several bleeding-edge tools in PATH, I've
> encountered this failure in the 'tests/misc/timeout-group.sh' test.
>
> [SNIP]
>
I've re-run the test few times with the system no longer loaded, and
it passed.  I he re-run it one more time with the system under some
load again, and this time the test was skipped!

timeout-group.sh: skipped test: timeout returned 142. SIGALRM not handled?
SKIP: tests/misc/timeout-group.sh

The detailed log of the skip is here below.

HTH,
  Stefano

-*-*-

++ initial_cwd_=/devel/bleeding/src/coreutils
++ fail=0
+++ testdir_prefix_
+++ printf gt
++ pfx_=gt
+++ mktempd_ /devel/bleeding/src/coreutils gt-timeout-group.sh.
+++ case $# in
+++ destdir_=/devel/bleeding/src/coreutils
+++ template_=gt-timeout-group.sh.
+++ MAX_TRIES_=4
+++ case $destdir_ in
+++ case $template_ in
 unset TMPDIR
+++ d=/devel/bleeding/src/coreutils/gt-timeout-group.sh.6KTf
+++ case $d in
+++ test -d /devel/bleeding/src/coreutils/gt-timeout-group.sh.6KTf
 ls -dgo /devel/bleeding/src/coreutils/gt-timeout-group.sh.6KTf
 tr S -
+++ perms='drwx-- 2 4096 Dec 14 18:30 
/devel/bleeding/src/coreutils/gt-timeout-group.sh.6KTf'
+++ case $perms in
+++ test 0 = 0
+++ echo /devel/bleeding/src/coreutils/gt-timeout-group.sh.6KTf
+++ return
++ test_dir_=/devel/bleeding/src/coreutils/gt-timeout-group.sh.6KTf
++ cd /devel/bleeding/src/coreutils/gt-timeout-group.sh.6KTf
++ gl_init_sh_nl_='
'
++ IFS='
'
++ for sig_ in 1 2 3 13 15
+++ expr 1 + 128
++ eval 'trap '\''Exit 129'\'' 1'
+++ trap 'Exit 129' 1
++ for sig_ in 1 2 3 13 15
+++ expr 2 + 128
++ eval 'trap '\''Exit 130'\'' 2'
+++ trap 'Exit 130' 2
++ for sig_ in 1 2 3 13 15
+++ expr 3 + 128
++ eval 'trap '\''Exit 131'\'' 3'
+++ trap 'Exit 131' 3
++ for sig_ in 1 2 3 13 15
+++ expr 13 + 128
++ eval 'trap '\''Exit 141'\'' 13'
+++ trap 'Exit 141' 13
++ for sig_ in 1 2 3 13 15
+++ expr 15 + 128
++ eval 'trap '\''Exit 143'\'' 15'
+++ trap 'Exit 143' 15
++ trap remove_tmp_ 0
+ path_prepend_ ./src
+ test 1 '!=' 0
+ path_dir_=./src
+ case $path_dir_ in
+ abs_path_dir_=/devel/bleeding/src/coreutils/./src
+ case $abs_path_dir_ in
+ 
PATH=/devel/bleeding/src/coreutils/./src:/devel/bleeding/src/coreutils/src:/usr/local/bleeding/bin:/usr/local/bleeding/sbin:/home/stefano/bin/local:/home/stefano/bin:/usr/local/bin:/opt/bin:/usr/lib/jvm/java-6-sun-1.6.0.26/bin:/usr/games:/usr/bin:/usr/sbin:/bin:/sbin
+ create_exe_shims_ /devel/bleeding/src/coreutils/./src
+ case $EXEEXT in
+ return 0
+ shift
+ test 0 '!=' 0
+ export PATH
+ print_ver_ timeout
+ test yes = yes
+ local i
+ for i in '$*'
+ env timeout --version
timeout (GNU coreutils) 8.20.63-4f62d
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later .
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Padraig Brady.
+ setsid true
+ cat
+ chmod a+x timeout.cmd
+ cat
+ chmod a+x group.sh
+ setsid ./group.sh
+ retry_delay_ check_timeout_cmd_running .1 6
+ local test_func=check_timeout_cmd_running
+ local init_delay=.1
+ local max_n_tries=6
+ local attempt=1
+ local num_sleeps=1
+ local time_fail
+ test 1 -le 6
++ gawk -v n=1 -v s=.1 'BEGIN { print s * n }'
+ local delay=0.1
+ check_timeout_cmd_running 0.1
+ local delay=0.1
+ test -e timeout.running
+ sleep 0.1
+ return 1
+ time_fail=1
++ expr 1 + 1
+ attempt=2
++ expr 1 '*' 2
+ num_sleeps=2
+ test 2 -le 6
++ gawk -v n=2 -v s=.1 'BEGIN { print s * n }'
+ local delay=0.2
+ check_timeout_cmd_running 0.2
+ local delay=0.2
+ test -e timeout.running
+ time_fail=0
+ break
+ test 0 = 0
+ env kill -INT -- -3744
+ wait
+ test -e int.received
+ rm -f int.received timeout.running
++ date +%s
+ start=1355506235
+ pid=3779
+ retry_delay_ check_timeout_cmd_running .1 6
+ local test_func=check_timeout_cmd_running
+ local init_delay=.1
+ local max_n_tries=6
+ local attempt=1
+ local num_sleeps=1
+ local time_fail
+ test 1 -le 6
++ gawk -v n=1 -v s=.1 'BEGIN { print s * n }'
+ timeout -sALRM 30 timeout -sINT 25 ./timeout.cmd 20
+ local delay=0.1
+ check_timeout_cmd_running 0.1
+ local delay=0.1
+ test -e timeout.running
+ time_fail=0
+ break
+ test 0 = 0
+ kill -ALRM 3779
+ wait 3779
+ ret=142
+ test 142 -eq 124
+ skip_ 'timeout returned 142. SIGALRM not handled?'
+ warn_ 'timeout-group.sh: skipped test: timeout returned 142. SIGALRM not 
handled?'
+ case $IFS in
+ printf '%s\n' 'timeout-group.sh: skipped test: timeout returned 142. SIGALRM 
not handled?'
timeout-group.sh: skipped test: timeout returned 142. SIGALRM not handled?
+ test 9 = 2
+ printf '%s\n' 'timeout-group.sh: skipped test: timeout returned 142. SIGALRM 
not handled?'
+ sed 1q
+ Exit 77
+ set +e
+ exit 77
+ exit 77
+ remove_tmp_
+ __st=77
+ cleanup_
+ :
+ cd /devel/bleeding/src/coreutils
+ chmod -R u+rwx /deve

bug#13172: tail: unrecognized file system type

[Bernhard Voelker]

On 12/14/2012 10:28 AM, Pádraig Brady wrote:
> On 12/14/2012 09:18 AM, Bernhard Voelker wrote:
>> +case S_MAGIC_CEPH: /* 0x00c36400 remote */
>> +  return "ceph";
> 
> s/00c/00C/ for consistency
> 
> thanks for handling this.
> Pádraig.

Thanks for the review.
Konrad also confirmed that it works offline.

Pushed:
http://git.sv.gnu.org/cgit/coreutils.git/commit/?id=c321f745

Marking as done.

Have a nice day,
Berny

bug#13135: Loss of data while copying

[Pádraig Brady]

On 12/10/2012 08:43 PM, John Reiser wrote:

On 12/10/2012 10:03 AM, Eric Blake wrote:

On 12/10/2012 10:06 AM, John Reiser wrote:

Yes, because a count was specified,
dd will operate in its default awkward but POSIX specified mode
of counting each read() call, even if it returned less than specified.
This is especially noticeable with pipes:


So this bug report is really about the execrable documentation for 'dd'.
Despite similar complaints appearing yearly [or so],
the text of "info dd" does not contain the string "pipe".  SHAME ON COREUTILS.
Explaining the most common error, and how to avoid it, certainly does
belong in the documentation.  The purpose of documentation is to *FACILITATE*
the correct use of the tool, and not merely to erect the minimal legal defense
of the code.


We've tried really hard to make this issue obvious.
Even going to the effort of auto prompting the user
to use iflag=fullblock.
The full discussion of the awkward auto suggestion logic
can be seen in http://bugs.gnu.org/7362

In more "normal" cases users will get the warning:

$ yes blah | src/dd of=/dev/null bs=11 count=1
dd: warning: partial read (53248 bytes); suggest iflag=fullblock

We didn't prompt in this case because it's
a bit of an edge case in that ibs is specified
rather than bs. So since there is write aggregation
in that case and to support use cases like the following,
we don't warn here:

$ (echo part1; sleep 1; echo part2; sleep 1; echo discard) |
  dd count=2 ibs=4096 obs=1 2>/dev/null


Rather than complaining, how about you submit a patch to improve the
documentation?



diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 21400ad..c2282eb 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -8055,6 +8055,7 @@ OS/360 JCL.
  @item if=@var{file}
  @opindex if
  Read from @var{file} instead of standard input.
+(If the input is a pipe then see @samp{fullblock} below.)


I think I'll move the warning to count=
as it's mostly an issue when that is specified.



  @item of=@var{file}
  @opindex of
@@ -8397,6 +8398,9 @@ may return early if a full block is not available.
  When that happens, continue calling @code{read} to fill the remainder
  of the block.
  This flag can be used only with @code{iflag}.
+If the input is a pipe and argument @samp{count=} also is specified,
+then probably @samp{iflag=fullblock} should be used
+in order to prevent surprises caused by short reads.


How about this instead?

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 5f8fad7..b916a86 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -8117,6 +8117,11 @@ Copy @var{n} @samp{ibs}-byte blocks from the input file,
 of everything until the end of the file.
 if @samp{iflag=count_bytes} is specified, @var{n} is interpreted
 as a byte count rather than a block count.
+Note if the input may return short reads as could be the case
+when reading from a pipe for example, @samp{iflag=fullblock}
+will ensure that @samp{count=} corresponds to complete input blocks
+rather than the traditional POSIX specified behavior of counting
+input read operations.

 @item status=@var{which}
 @opindex status
@@ -8397,6 +8402,10 @@ may return early if a full block is not available.
 When that happens, continue calling @code{read} to fill the remainder
 of the block.
 This flag can be used only with @code{iflag}.
+This flag is useful with pipes for example
+as they may return short reads. I that case,
+this flag is needed to ensure that a @samp{count=} argument is
+interpreted as a block count rather than a count of read operations.

 @item count_bytes
 @opindex count_bytes

bug#13135: Loss of data while copying

[Jim Meyering]

Pádraig Brady wrote:
...
> I think I'll move the warning to count=
> as it's mostly an issue when that is specified.

Good idea.

>>   @item of=@var{file}
>>   @opindex of
>> @@ -8397,6 +8398,9 @@ may return early if a full block is not available.
>>   When that happens, continue calling @code{read} to fill the remainder
>>   of the block.
>>   This flag can be used only with @code{iflag}.
>> +If the input is a pipe and argument @samp{count=} also is specified,
>> +then probably @samp{iflag=fullblock} should be used
>> +in order to prevent surprises caused by short reads.
>
> How about this instead?

Looks good.  Thanks.

> diff --git a/doc/coreutils.texi b/doc/coreutils.texi
...
> @@ -8397,6 +8402,10 @@ may return early if a full block is not available.
>  When that happens, continue calling @code{read} to fill the remainder
>  of the block.
>  This flag can be used only with @code{iflag}.
> +This flag is useful with pipes for example
> +as they may return short reads. I that case,

s/I/In/

...