On
- GNU/Hurd x86_64 from 2024,
- GNU/Hurd i386 from 2023,
I see a test hang: hash-collision-perf.
On GNU/Hurd x86_64:
When I interrupted the build, the file 'in' has 5120000 lines, and
find attached the log file of this test. As you can see, the value of
small_ms stays 0 even for larger files.
By running
$ date; LC_ALL=C ../../src/grep --file=in empty; date
I can see that the execution times grow like this:
640000 0.3 sec
1280000 0.9 sec
2560000 1.5 sec
5120000 > 60 sec
On GNU/Hurd i386, it's similar. Here it's when the file 'in' has
40960000 lines, that the grep execution hangs. Find attached the
last stack trace I was able to obtain before it hung.
Regardless how much RAM I give to the machine, there will always
be a point where "grep --file=in empty" will take more RAM than
available, and (since Hurd does not have an OOM killer) the machine
then hangs.
IMO, the correct behaviour would be that 'grep' exits via xalloc_die(),
not that it hangs.
Whereas on GNU/Linux (in a machine that has the same amount of RAM as
the GNU/Hurd machine):
$ : > empty
$ seq 640000 > in; LC_ALL=C time ./src/grep --file=in empty
real 0.44s
$ seq 1280000 > in; LC_ALL=C time ./src/grep --file=in empty
real 0.99s
$ seq 2560000 > in; LC_ALL=C time ./src/grep --file=in empty
real 2.22s
$ seq 5120000 > in; LC_ALL=C time ./src/grep --file=in empty
real 4.84s
$ seq 10240000 > in; LC_ALL=C time ./src/grep --file=in empty
real 24.19s
$ seq 20480000 > in; LC_ALL=C time ./src/grep --file=in empty
Killed
real 24.40s
Here it was the OOM killer that saved the machine from hanging.
So, IMO, there are two bugs:
1) When the allocation of the kwset takes more memory than available,
'grep' should exit via xalloc_die(), instead of waiting to be killed
by the OOM killer.
2) In the 'hash-collision-perf' unit test: The use of a perl primitive
for measuring the execution time of a child process, that is not
properly ported to GNU/Hurd.
Bruno
++ initial_cwd_=/home/bruno/grep-3.11.69-a4628/build/tests
+++ testdir_prefix_
+++ printf gt
++ pfx_=gt
+++ mktempd_ /home/bruno/grep-3.11.69-a4628/build/tests gt-hash-collision-perf.XXXX
+++ case $# in
+++ destdir_=/home/bruno/grep-3.11.69-a4628/build/tests
+++ template_=gt-hash-collision-perf.XXXX
+++ MAX_TRIES_=4
+++ case $destdir_ in
+++ destdir_slash_=/home/bruno/grep-3.11.69-a4628/build/tests/
+++ case $template_ in
++++ unset TMPDIR
+++ d=/home/bruno/grep-3.11.69-a4628/build/tests/gt-hash-collision-perf.bCSE
+++ case $d in
+++ :
+++ test -d /home/bruno/grep-3.11.69-a4628/build/tests/gt-hash-collision-perf.bCSE
++++ ls -dgo /home/bruno/grep-3.11.69-a4628/build/tests/gt-hash-collision-perf.bCSE
+++ perms='drwx------ 2 4096 Apr 7 15:55 /home/bruno/grep-3.11.69-a4628/build/tests/gt-hash-collision-perf.bCSE'
+++ case $perms in
+++ :
+++ echo /home/bruno/grep-3.11.69-a4628/build/tests/gt-hash-collision-perf.bCSE
+++ return
++ test_dir_=/home/bruno/grep-3.11.69-a4628/build/tests/gt-hash-collision-perf.bCSE
++ cd /home/bruno/grep-3.11.69-a4628/build/tests/gt-hash-collision-perf.bCSE
++ case $srcdir in
++ srcdir=../../../tests
++ builddir=..
++ export srcdir builddir
++ gl_init_sh_nl_='
'
++ IFS='
'
++ for sig_ in 1 2 3 13 15
+++ expr 1 + 128
++ eval 'trap '\''Exit 129'\'' 1'
+++ trap 'Exit 129' 1
++ for sig_ in 1 2 3 13 15
+++ expr 2 + 128
++ eval 'trap '\''Exit 130'\'' 2'
+++ trap 'Exit 130' 2
++ for sig_ in 1 2 3 13 15
+++ expr 3 + 128
++ eval 'trap '\''Exit 131'\'' 3'
+++ trap 'Exit 131' 3
++ for sig_ in 1 2 3 13 15
+++ expr 13 + 128
++ eval 'trap '\''Exit 141'\'' 13'
+++ trap 'Exit 141' 13
++ for sig_ in 1 2 3 13 15
+++ expr 15 + 128
++ eval 'trap '\''Exit 143'\'' 15'
+++ trap 'Exit 143' 15
++ saved_IFS='
'
++ IFS=:
++ new_PATH=
++ for dir in $PATH
++ IFS='
'
++ case "$dir" in
++ test -d /home/bruno/grep-3.11.69-a4628/build/src/.
++ new_PATH=/home/bruno/grep-3.11.69-a4628/build/src
++ for dir in $PATH
++ IFS='
'
++ case "$dir" in
++ for dir in $PATH
++ IFS='
'
++ case "$dir" in
++ test -d /usr/local/bin/.
++ new_PATH=/home/bruno/grep-3.11.69-a4628/build/src:/usr/local/bin
++ for dir in $PATH
++ IFS='
'
++ case "$dir" in
++ test -d /usr/bin/.
++ new_PATH=/home/bruno/grep-3.11.69-a4628/build/src:/usr/local/bin:/usr/bin
++ for dir in $PATH
++ IFS='
'
++ case "$dir" in
++ test -d /bin/.
++ new_PATH=/home/bruno/grep-3.11.69-a4628/build/src:/usr/local/bin:/usr/bin:/bin
++ for dir in $PATH
++ IFS='
'
++ case "$dir" in
++ test -d /usr/local/games/.
++ new_PATH=/home/bruno/grep-3.11.69-a4628/build/src:/usr/local/bin:/usr/bin:/bin:/usr/local/games
++ for dir in $PATH
++ IFS='
'
++ case "$dir" in
++ test -d /usr/games/.
++ new_PATH=/home/bruno/grep-3.11.69-a4628/build/src:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
++ IFS='
'
++ PATH=/home/bruno/grep-3.11.69-a4628/build/src:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
++ export PATH
++ trap remove_tmp_ EXIT
+ path_prepend_ ../src
+ test 1 '!=' 0
+ path_dir_=../src
+ case $path_dir_ in
+ abs_path_dir_=/home/bruno/grep-3.11.69-a4628/build/tests/../src
+ case $abs_path_dir_ in
+ PATH=/home/bruno/grep-3.11.69-a4628/build/tests/../src:/home/bruno/grep-3.11.69-a4628/build/src:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ create_exe_shims_ /home/bruno/grep-3.11.69-a4628/build/tests/../src
+ case $EXEEXT in
+ return 0
+ shift
+ test 0 '!=' 0
+ export PATH
+ fail=0
+ require_perl_
+ test perl
+ perl -e 'use warnings'
+ :
+ n_pat=40000
+ :
+ seq 40000
++ LC_ALL=C
++ user_time_ 1 grep --file=in empty
++ perl -le '
my $expected_exit_status = $ARGV[0];
shift @ARGV;
system (@ARGV);
my ($user, $system, $child_user, $child_system) = times;
my $me = q(hash-collision-perf);
$? == -1
and die qq($me: failed to exec ") . join (" ", @ARGV) . qq(": $!\n);
my $rc = $?;
my $sig = ($rc & 127);
$sig and die "$me: child died with signal $sig\n";
$rc >>= 8;
$rc == $expected_exit_status
or die "$me: bad exit status: expected $expected_exit_status; got $rc\n";
# Print milliseconds of child user time.
$child_user *= 1000;
print int ($child_user + 0.5)' 1 grep --file=in empty
+ small_ms=0
+ test 0 -ge 200
++ expr 40000 '*' 2
+ n_pat=80000
+ :
+ seq 80000
++ LC_ALL=C
++ user_time_ 1 grep --file=in empty
++ perl -le '
my $expected_exit_status = $ARGV[0];
shift @ARGV;
system (@ARGV);
my ($user, $system, $child_user, $child_system) = times;
my $me = q(hash-collision-perf);
$? == -1
and die qq($me: failed to exec ") . join (" ", @ARGV) . qq(": $!\n);
my $rc = $?;
my $sig = ($rc & 127);
$sig and die "$me: child died with signal $sig\n";
$rc >>= 8;
$rc == $expected_exit_status
or die "$me: bad exit status: expected $expected_exit_status; got $rc\n";
# Print milliseconds of child user time.
$child_user *= 1000;
print int ($child_user + 0.5)' 1 grep --file=in empty
+ small_ms=0
+ test 0 -ge 200
++ expr 80000 '*' 2
+ n_pat=160000
+ :
+ seq 160000
++ LC_ALL=C
++ user_time_ 1 grep --file=in empty
++ perl -le '
my $expected_exit_status = $ARGV[0];
shift @ARGV;
system (@ARGV);
my ($user, $system, $child_user, $child_system) = times;
my $me = q(hash-collision-perf);
$? == -1
and die qq($me: failed to exec ") . join (" ", @ARGV) . qq(": $!\n);
my $rc = $?;
my $sig = ($rc & 127);
$sig and die "$me: child died with signal $sig\n";
$rc >>= 8;
$rc == $expected_exit_status
or die "$me: bad exit status: expected $expected_exit_status; got $rc\n";
# Print milliseconds of child user time.
$child_user *= 1000;
print int ($child_user + 0.5)' 1 grep --file=in empty
+ small_ms=0
+ test 0 -ge 200
++ expr 160000 '*' 2
+ n_pat=320000
+ :
+ seq 320000
++ LC_ALL=C
++ user_time_ 1 grep --file=in empty
++ perl -le '
my $expected_exit_status = $ARGV[0];
shift @ARGV;
system (@ARGV);
my ($user, $system, $child_user, $child_system) = times;
my $me = q(hash-collision-perf);
$? == -1
and die qq($me: failed to exec ") . join (" ", @ARGV) . qq(": $!\n);
my $rc = $?;
my $sig = ($rc & 127);
$sig and die "$me: child died with signal $sig\n";
$rc >>= 8;
$rc == $expected_exit_status
or die "$me: bad exit status: expected $expected_exit_status; got $rc\n";
# Print milliseconds of child user time.
$child_user *= 1000;
print int ($child_user + 0.5)' 1 grep --file=in empty
+ small_ms=0
+ test 0 -ge 200
++ expr 320000 '*' 2
+ n_pat=640000
+ :
+ seq 640000
++ LC_ALL=C
++ user_time_ 1 grep --file=in empty
++ perl -le '
my $expected_exit_status = $ARGV[0];
shift @ARGV;
system (@ARGV);
my ($user, $system, $child_user, $child_system) = times;
my $me = q(hash-collision-perf);
$? == -1
and die qq($me: failed to exec ") . join (" ", @ARGV) . qq(": $!\n);
my $rc = $?;
my $sig = ($rc & 127);
$sig and die "$me: child died with signal $sig\n";
$rc >>= 8;
$rc == $expected_exit_status
or die "$me: bad exit status: expected $expected_exit_status; got $rc\n";
# Print milliseconds of child user time.
$child_user *= 1000;
print int ($child_user + 0.5)' 1 grep --file=in empty
+ small_ms=0
+ test 0 -ge 200
++ expr 640000 '*' 2
+ n_pat=1280000
+ :
+ seq 1280000
++ LC_ALL=C
++ user_time_ 1 grep --file=in empty
++ perl -le '
my $expected_exit_status = $ARGV[0];
shift @ARGV;
system (@ARGV);
my ($user, $system, $child_user, $child_system) = times;
my $me = q(hash-collision-perf);
$? == -1
and die qq($me: failed to exec ") . join (" ", @ARGV) . qq(": $!\n);
my $rc = $?;
my $sig = ($rc & 127);
$sig and die "$me: child died with signal $sig\n";
$rc >>= 8;
$rc == $expected_exit_status
or die "$me: bad exit status: expected $expected_exit_status; got $rc\n";
# Print milliseconds of child user time.
$child_user *= 1000;
print int ($child_user + 0.5)' 1 grep --file=in empty
+ small_ms=0
+ test 0 -ge 200
++ expr 1280000 '*' 2
+ n_pat=2560000
+ :
+ seq 2560000
++ LC_ALL=C
++ user_time_ 1 grep --file=in empty
++ perl -le '
my $expected_exit_status = $ARGV[0];
shift @ARGV;
system (@ARGV);
my ($user, $system, $child_user, $child_system) = times;
my $me = q(hash-collision-perf);
$? == -1
and die qq($me: failed to exec ") . join (" ", @ARGV) . qq(": $!\n);
my $rc = $?;
my $sig = ($rc & 127);
$sig and die "$me: child died with signal $sig\n";
$rc >>= 8;
$rc == $expected_exit_status
or die "$me: bad exit status: expected $expected_exit_status; got $rc\n";
# Print milliseconds of child user time.
$child_user *= 1000;
print int ($child_user + 0.5)' 1 grep --file=in empty
+ small_ms=0
+ test 0 -ge 200
++ expr 2560000 '*' 2
+ n_pat=5120000
+ :
+ seq 5120000
