Hi all,
I was thinking how to improve the situation where users complain about
"unrecognized file systems" errors in "tail -f".
(as in the recent
https://lists.gnu.org/archive/html/coreutils/2017-11/msg00043.html ).
This is not their "fault", as they simply have an older version
installed, and it is not our "fault", as the file system is likely
already supported in newer versions.
It is simply a matter of making it easier for users to understand what's
going on, and easier for us to not have to answer the same thing again
and again.
I suggest the following:
1. a script that can detect which file systems are supported -
there's already one working in 'src/extract-magic'.
2. improve the script to also detect which in which version
the file system is supported (POC attached).
3. Generate an HTML page listing each filesystem, the magic value,
and the minimal version it is supported in.
something like:
Hello,
If you are seeing the error message:
unrecognized file system type [[0x794c7630]]
This error is reported when reading files on [[OVERLAYFS]] file
system. Support for [OVERLAYFS] was added in Coreutils version
[v8.28].
Upgrade coreutils version to avoid this warning.
To see which version you are using, run "tail --version".
This warning is harmless - 'tail' will still operate correctly
on your filesystem, although not as efficient as possible.
If you have any further questions, please contact [email protected]
5. We modify 'tail' to point the user to the web page instead
of reporting a bug.
6. when new filesystems are added, just update the webpage.
What do you think ?
Attached is a proof-of-concept script implementing step 2.
Usage like so:
$ perl prepare-fs-support src/stat.c | head
magic-value c-constant availble-in-version human-name
0xADF5 S_MAGIC_ADFS v8.15 adfs
0xADFF S_MAGIC_AFFS v8.15 affs
0x5346414F S_MAGIC_AFS v8.15 afs
0x09041934 S_MAGIC_ANON_INODE_FS v8.15 anon-inode FS
0x0187 S_MAGIC_AUTOFS v8.15 autofs
0x42465331 S_MAGIC_BEFS v8.15 befs
0x1BADFACE S_MAGIC_BFS v8.15 bfs
0x9123683E S_MAGIC_BTRFS v8.15 btrfs
0x0027E0EB S_MAGIC_CGROUP v8.15 cgroupfs
Of course, it doesn't even have to be an automated process with a perl
script, we can just update a web page manually when a new filesystem is
added.
Comments welcomed,
-assaf
#!/usr/bin/perl -w
# Derive #define directives from specially formatted 'case ...:' statements.
# Copyright (C) 2003-2017 Free Software Foundation, Inc.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
use strict;
use Data::Dumper;
use Getopt::Long;
(my $VERSION = '$Revision: 1.6 $ ') =~ tr/[0-9].//cd;
(my $ME = $0) =~ s|.*/||;
END
{
# Nobody ever checks the status of print()s. That's okay, because
# if any do fail, we're guaranteed to get an indicator when we close()
# the filehandle.
#
# Close stdout now, and if there were no errors, return happy status.
# If stdout has already been closed by the script, though, do nothing.
defined fileno STDOUT
or return;
close STDOUT
and return;
# Errors closing stdout. Indicate that, and hope stderr is OK.
warn "$ME: closing standard output: $!\n";
# Don't be so arrogant as to assume that we're the first END handler
# defined, and thus the last one invoked. There may be others yet
# to come. $? will be passed on to them, and to the final _exit().
#
# If it isn't already an error, make it one (and if it _is_ an error,
# preserve the value: it might be important).
$? ||= 1;
}
sub usage ($)
{
my ($exit_code) = @_;
my $STREAM = ($exit_code == 0 ? *STDOUT : *STDERR);
if ($exit_code != 0)
{
print $STREAM "Try '$ME --help' for more information.\n";
}
else
{
print $STREAM <<EOF;
Usage: $ME [OPTIONS] FILE
This script parses src/stat.c file (from GNU coreutils),
and produces a table of filesystem magic values/names,
together with the released coreutils version in which
the filesystem is available.
Typical usage:
$ME src/stat.c
NOTE:
The version is the most recent version were the filesystem
magic marger was updated. If S_MAGIC_FOO was added in commit
v8.20-11, the output will show v8.21 (assuming v8.21 was released).
If same magic value was later updated in commit v8.24-99,
the shown version will be v8.25 .
OPTIONS:
--no-header omit header line
--full show revision/date/commit in output
--debug show debug/progress information
--help display this help and exit
--version output version information and exit
EOF
}
exit $exit_code;
}
{
my $debug = 0;
my $show_full = 0;
my $header = 1;
GetOptions
(
debug => sub { $debug = 1 },
full => sub { $show_full = 1 },
'no-header' => sub { $header = 0 },
help => sub { usage 0 },
version => sub { print "$ME version $VERSION\n"; exit },
) or usage 1;
my $fail = 0;
@ARGV < 1
and (warn "$ME: missing FILE arguments\n"), $fail = 1;
1 < @ARGV
and (warn "$ME: too many arguments\n"), $fail = 1;
$fail
and usage 1;
my $file = $ARGV[0];
open FH, "git blame -s -e -l -t '$file'|"
or die "$ME: can't git-blame '$file': $!\n";
##
## Parse src/stat.c , extract S_MAGIC_XXX definitions.
## For each one, keep the most recent git revision.
## Modeled after 'extract-magic', but additionally
## tracks the following "return" statement (which is expected
## to contain the human-readable name of the file system).
my ($c_name, $fs_rev, $magic, $local, $name);
my @fs;
while (defined (my $line = <FH>))
{
$line =~ /^([A-Fa-f0-9]{40})\s+(\d+)\)\s+(.*)$/
or warn "failed to parse git-blame line:\n $line\n";
my ($rev, $linenum, $linetext) = ($1, $2, $3);
if ( $linetext =~ /^case S_MAGIC_/ ) {
warn "return statement not detected after magic '$c_name' - will be
skipped\n"
if $c_name;
$linetext =~
m!^case (S_MAGIC_\w+): /\* (0x[0-9A-Fa-f]+) (local|remote) \*/!
or (warn "$ME:$file:$.: malformed case S_MAGIC_... line"),
$fail = 1, next;
$c_name = $1;
$magic = $2;
$local = $3 eq 'local' ? 1 : 0;
$fs_rev = $rev;
#DEBUG
#print "got cname '$c_name'\n";
}
elsif ( $c_name && $linetext =~ /^\s*return\s+"([^"]*)"\s*;/ ) {
$name = $1 ;
push @fs, { c_name => $c_name,
name => $name,
magic => $magic,
rev => $fs_rev };
#DEBUG
#print "got name '$name'\n";
undef $c_name ;
undef $name ;
}
}
close FH;
if ($debug) {
print STDERR "\n\nDetected filesystem magic constant/names:\n";
print STDERR Dumper(\@fs),"\n";
}
### Get list of official released versions
my @tags = `git tag -l`
or die "failed to run 'git tag -l': $!";
my %released_versions = map { chomp ; $_ => 1 } @tags;
if ($debug) {
print STDERR "\n\nDetected official released versions:\n";
print Dumper(\%released_versions);
}
### Get the released version where each filesystem is available
my %query_revs = map { $_->{rev} => 1 } @fs;
my %revs;
foreach my $r (keys %query_revs) {
# Get the exact commit where this file system was added
my $commit = `git describe '$r'`
or die "failed to run 'git describe $r': $!";
chomp $commit;
# Extract the major/minor version
my ($major,$minor) = $commit =~ /^v(\d+)\.(\d+)/;
# Find the released version in which this filesystem is supported
my $available;
my $avail_ver1 = "v" . $major . "." . ($minor+1);
my $avail_ver2 = "v" . ($major+1) . ".0";
if (exists $released_versions{$avail_ver1}) {
# Next minor version
$available = $avail_ver1;
} elsif (exists $released_versions{$avail_ver2}) {
# Next major version
$available = $avail_ver2;
} else {
# Not available yet
$available = undef ;
}
$revs{$r} = { commit => $commit,
available => $available
};
}
if ($debug) {
print STDERR "\n\nDetected version for each revision:\n";
print Dumper(\%revs);
}
# Add version info for each file system
foreach my $f (@fs) {
my $rev = $f->{rev};
my $commit = $revs{$rev}->{commit};
my $available = $revs{$rev}->{available};
$f->{commit} = $commit;
$f->{available} = ( $available or "not-released-yet" );
}
#TODO: version sort
@fs = sort { $a->{available} cmp $b->{available} } @fs ;
if ($debug) {
print STDERR "\n\nFinal record for each filesystem record:\n";
print Dumper(\@fs),"\n";
}
if ($header) {
if ($show_full) {
printf "%40s %12s %-23s %12s %-20s %s\n",
"git-revision",
"magic-value",
"c-constant",
"availble-in-version",
"commit",
"human-name";
} else {
printf "%12s %-23s %-20s %s\n",
"magic-value",
"c-constant",
"availble-in-version",
"human-name";
}
}
# print as a table
foreach my $f (@fs) {
if ($show_full) {
printf "%40s %12s %-23s %12s %-20s %s\n",
$f->{rev},
$f->{magic},
$f->{c_name},
$f->{available},
$f->{commit},
$f->{name};
} else {
printf "%12s %-23s %-20s %s\n",
$f->{magic},
$f->{c_name},
$f->{available},
$f->{name};
}
}
exit $fail;
}
