bug#12339: Bug: rm -fr . doesn't dir depth first deletion yet it is documented to do so.
Paul Eggert wrote: > On 09/03/2012 10:20 PM, Bernhard Voelker wrote: >> the question is what rm should remove: >> the symlink or the target? > > The convention in POSIX is that if a symlink is > followed by '/', it's dereferenced. So this does > appear to be a bug in coreutils 'rm'. > > I confirmed that the bug does not occur with > Solaris 11 'rm': > > $ mkdir d > $ touch d/file > $ ln -s d dl > $ ls -al * > lrwxrwxrwx 1 eggert faculty1 Sep 3 22:26 dl -> d Same for FreeBSD 9.0-RELEASE-p3: freebsd$ mkdir d; ln -s d s; /bin/rm -r s/; ls s@ But not with busybox 1.20.0-6: $ mkdir d; ln -s d s; busybox rm -r s/; ls d/ I have mixed feelings, but am leaning towards the Solaris 11/FreeBSD behavior.
bug#12339: Bug: rm -fr . doesn't dir depth first deletion yet it is documented to do so.
On 09/04/2012 12:42 PM, Jim Meyering wrote: > I have mixed feelings, but am leaning towards the > Solaris 11/FreeBSD behavior. They must have fixed it for Solaris, here's an older one: $ uname -sr ; mkdir d; ln -s d s; /bin/rm -r s/; ls SunOS 5.9 d Have a nice day, Berny
bug#12339: Bug: rm -fr . doesn't dir depth first deletion yet it is documented to do so.
Paul Eggert wrote:
> I was about to suggest the following, on GNU/Linux:
>
> # Don't do this unless you know what you're doing!
> rm -fr /proc/self/cwd/
>
> Except it doesn't work! Not even if I append '.':
>
> $ mkdir /tmp/victim
> $ cd /tmp/victim
> $ touch foo
> $ rm -fr /proc/self/cwd/
> rm: cannot remove `/proc/self/cwd': Too many levels of symbolic links
> $ rm -fr /proc/self/cwd/.
> rm: cannot remove directory: `/proc/self/cwd/.'
> $ ls
> foo
>
> Aren't these bugs, at least?
With the following patch, I see new behavior.
It's an improvement, but we're still not there:
$ mkdir -p d/e/f; ln -s d s; rm -r s/
rm: cannot remove 's/': Not a directory
[Exit 1]
$ find
.
./s
./d
Notice how it did traverse s/ into d/, and removed d/e and d/e/f.
The only problem is that when it attempted to remove the command-line
specified "s/", unlinkat (AT_FDCWD, "s/", AT_REMOVEDIR) failed:
unlinkat(4, "d", 0) = 0
unlinkat(5, "f", AT_REMOVEDIR) = 0
unlinkat(4, "e", AT_REMOVEDIR) = 0
unlinkat(AT_FDCWD, "s/", AT_REMOVEDIR) = -1 ENOTDIR (Not a directory)
rm: cannot remove 's/': Not a directory
+++ exited with 1 +++
Now, this looks like a problem with unlinkat.
Debating whether to add a test.
Just writing the above, I know the answer. I will add a test.
==
>From 958c553a84d37bdd8669ae68bfc8c4d4c5e7d2e1 Mon Sep 17 00:00:00 2001
From: Jim Meyering
Date: Tue, 4 Sep 2012 14:40:11 +0200
Subject: [PATCH] rm: avoid bogus diagnostic for a slash-decorated
symlink-to-dir
These commands would evoke an invalid diagnostic:
$ mkdir d && ln -s d s && env rm -r s/
rm: cannot remove 's': Too many levels of symbolic links
remove.c was stripping trailing slashes from "s/" before passing
the name to "rm". But a trailing slash may change the semantics,
and thus should not be stripped.
* src/remove.c (rm_fts): Do not strip trailing slashes.
* tests/rm/v-slash.sh: Adapt to new expected output.
* gnulib: Update to latest, for an improved fts.c that merely
normalizes trailing slashes.
Reported by Paul Eggert in discussion of http://bugs.gnu.org/12339
---
gnulib | 2 +-
src/remove.c| 3 ---
tests/rm/v-slash.sh | 2 +-
3 files changed, 2 insertions(+), 5 deletions(-)
diff --git a/gnulib b/gnulib
index 68f693f..3a9002d 16
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit 68f693ff1db33bf24695f0f42c62e7801966fd06
+Subproject commit 3a9002d3cc63da7110f133b1040d2d2b0aad8305
diff --git a/src/remove.c b/src/remove.c
index 69faae6..847a5cc 100644
--- a/src/remove.c
+++ b/src/remove.c
@@ -433,9 +433,6 @@ rm_fts (FTS *fts, FTSENT *ent, struct rm_options const *x)
/* Perform checks that can apply only for command-line arguments. */
if (ent->fts_level == FTS_ROOTLEVEL)
{
- if (strip_trailing_slashes (ent->fts_path))
-ent->fts_pathlen = strlen (ent->fts_path);
-
/* If the basename of a command line argument is "." or "..",
diagnose it and do nothing more with that argument. */
if (dot_or_dotdot (last_component (ent->fts_accpath)))
diff --git a/tests/rm/v-slash.sh b/tests/rm/v-slash.sh
index 504f4ff..ec77bd0 100755
--- a/tests/rm/v-slash.sh
+++ b/tests/rm/v-slash.sh
@@ -26,7 +26,7 @@ touch a/x || framework_failure_
rm --verbose -r a/// > out || fail=1
cat <<\EOF > exp || fail=1
removed 'a/x'
-removed directory: 'a'
+removed directory: 'a/'
EOF
compare exp out || fail=1
--
1.7.12.176.g3fc0e4c
bug#12350: Composites identified as primes in factor.c (when HAVE_GMP)
The very old factoring code cut from an now obsolete version GMP does not pass proper arguments to the mpz_probab_prime_p function. It ask for 3 Miller-Rabin tests only, which is not sufficient. I am afraid the original poor code was wrritten by me, where this particular problem was introduced with 3.0's demos/factorize.c. A Miller-Rabin test will detect composites with at least a probability of 3/4. For a uniform random composite, the probability will actually by much higher. Or put another way, of the N-3 possible Miller-Rabin tests for checking the composite N, there is no number N for which more than (N-3)/4 of the tests will fail to detect the number as a composite. For most numbers N the number of "false witnesses" will be much, much lower. Problem numbers are of the for N=pq, p,q prime and (p-1)/(q-1) = s, where s is a small integer. (There are other problem forms too, incvolving 3 or more prime factors.) When s = 2, we get the 3/4 factor. It is easy to find numbers of that form that causes coreutils factor to fail: 465658903 2242724851 6635692801 17709149503 17754345703 20889169003 42743470771 54890944111 72047131003 85862644003 98275842811 114654168091 117225546301 ... There are 9008992 composites of the form with s=2 below 2^64. With 3 Miller-Rabin test, one would expect about 9008992/4^64 = 140766 to be invalidly recognised as primes in that range. Here is a simple patch: diff Description: Binary data I and Niels Möller have written a suggested replacement for coreutils factor.c. It fixes a number of issues with the current code: (1) Much faster trial division code (> 10x) based on a small table of prime inverses. Still, the new code doesn't perform lots of trial dividing. (2) Pollard rho code using Montgomery representation for numbers < 2^64. (We consider extending this to 128 bits.) Not dependent on GMP. (3) Lucas prime proving code instead of probablitic Miller-Rabin primes testing. (4) SQUFOF code, which might be included depending on performance issues. (5) Replacement GMP code (#if HAVE_GMP) that also includes Lucas proving code. The new code is faster than the current code: Old: seq `pexpr 2^64-1000` `pexpr 2^64-1` | time factor >/dev/null 524.57 user New: seq `pexpr 2^64-1000` `pexpr 2^64-1` | time ./factor >/dev/null 0.05 user For smaller number ranges, the improvements are currently much more modest, as little as 2x in some cases. The code should be ready within a few weeks. -- Torbjörn
bug#12339: Bug: rm -fr . doesn't dir depth first deletion yet it is documented to do so.
On 09/04/2012 02:55 PM, Jim Meyering wrote: > With the following patch, I see new behavior. > It's an improvement, but we're still not there: > > $ mkdir -p d/e/f; ln -s d s; rm -r s/ > rm: cannot remove 's/': Not a directory > [Exit 1] > $ find > . > ./s > ./d > > Notice how it did traverse s/ into d/, and removed d/e and d/e/f. > The only problem is that when it attempted to remove the command-line > specified "s/", unlinkat (AT_FDCWD, "s/", AT_REMOVEDIR) failed: > > unlinkat(4, "d", 0) = 0 > unlinkat(5, "f", AT_REMOVEDIR) = 0 > unlinkat(4, "e", AT_REMOVEDIR) = 0 > unlinkat(AT_FDCWD, "s/", AT_REMOVEDIR) = -1 ENOTDIR (Not a directory) > rm: cannot remove 's/': Not a directory > +++ exited with 1 +++ > > Now, this looks like a problem with unlinkat. Hmm, what about dereferencing command line symlinks with trailing slashes _before_ entering the FTS loop instead? Have a nice day, Berny
bug#12350: Composites identified as primes in factor.c (when HAVE_GMP)
Torbjorn Granlund wrote:
> The very old factoring code cut from an now obsolete version GMP does
> not pass proper arguments to the mpz_probab_prime_p function. It ask
> for 3 Miller-Rabin tests only, which is not sufficient.
Hi Torbjorn
Thank you for the patch and explanation.
I've converted that into the commit below in your name.
Please proofread it and let me know if you'd like to change anything.
I tweaked the patch to change MR_REPS from a #define to an enum
and to add the comment just preceding.
I'll add NEWS and tests separately.
>From ea6dd126e6452504f9fa1d6708d25473e2c27e67 Mon Sep 17 00:00:00 2001
From: Torbjorn Granlund
Date: Tue, 4 Sep 2012 16:22:47 +0200
Subject: [PATCH] factor: don't ever declare composites to be prime
The multiple-precision factoring code (with HAVE_GMP) was copied from
a now-obsolete version of GMP that did not pass proper arguments to
the mpz_probab_prime_p function. It makes that code perform no more
than 3 Miller-Rabin tests only, which is not sufficient.
A Miller-Rabin test will detect composites with at least a probability
of 3/4. For a uniform random composite, the probability will actually
by much higher.
Or put another way, of the N-3 possible Miller-Rabin tests for checking
the composite N, there is no number N for which more than (N-3)/4 of the
tests will fail to detect the number as a composite. For most numbers N
the number of "false witnesses" will be much, much lower.
Problem numbers are of the for N=pq, p,q prime and (p-1)/(q-1) = s,
where s is a small integer. (There are other problem forms too,
incvolving 3 or more prime factors.) When s = 2, we get the 3/4 factor.
It is easy to find numbers of that form that cause coreutils' factor to
fail:
465658903
2242724851
6635692801
17709149503
17754345703
20889169003
42743470771
54890944111
72047131003
85862644003
98275842811
114654168091
117225546301
...
There are 9008992 composites of the form with s=2 below 2^64. With 3
Miller-Rabin test, one would expect about 9008992/4^64 = 140766 to be
invalidly recognised as primes in that range.
* src/factor.c (MR_REPS): Define to 25.
(factor_using_pollard_rho): Use MR_REPS, not 3.
(print_factors_multi): Likewise.
---
src/factor.c | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/src/factor.c b/src/factor.c
index 1d55805..e63e0e0 100644
--- a/src/factor.c
+++ b/src/factor.c
@@ -153,6 +153,9 @@ factor_using_division (mpz_t t, unsigned int limit)
mpz_clear (r);
}
+/* The number of Miller-Rabin tests we require. */
+enum { MR_REPS = 25 };
+
static void
factor_using_pollard_rho (mpz_t n, int a_int)
{
@@ -222,7 +225,7 @@ S4:
mpz_div (n, n, g); /* divide by g, before g is overwritten */
- if (!mpz_probab_prime_p (g, 3))
+ if (!mpz_probab_prime_p (g, MR_REPS))
{
do
{
@@ -242,7 +245,7 @@ S4:
mpz_mod (x, x, n);
mpz_mod (x1, x1, n);
mpz_mod (y, y, n);
- if (mpz_probab_prime_p (n, 3))
+ if (mpz_probab_prime_p (n, MR_REPS))
{
emit_factor (n);
break;
@@ -411,7 +414,7 @@ print_factors_multi (mpz_t t)
if (mpz_cmp_ui (t, 1) != 0)
{
debug ("[is number prime?] ");
- if (mpz_probab_prime_p (t, 3))
+ if (mpz_probab_prime_p (t, MR_REPS))
emit_factor (t);
else
factor_using_pollard_rho (t, 1);
--
1.7.12.176.g3fc0e4c
bug#12350: Composites identified as primes in factor.c (when HAVE_GMP)
On 09/04/2012 04:46 PM, Jim Meyering wrote: > incvolving 3 or more prime factors.) When s = 2, we get the 3/4 factor. s/incvolving/involving/ Have a nice day, Berny
bug#12339: Bug: rm -fr . doesn't dir depth first deletion yet it is documented to do so.
Bernhard Voelker wrote: > On 09/04/2012 02:55 PM, Jim Meyering wrote: >> With the following patch, I see new behavior. >> It's an improvement, but we're still not there: >> >> $ mkdir -p d/e/f; ln -s d s; rm -r s/ >> rm: cannot remove 's/': Not a directory >> [Exit 1] >> $ find >> . >> ./s >> ./d >> >> Notice how it did traverse s/ into d/, and removed d/e and d/e/f. >> The only problem is that when it attempted to remove the command-line >> specified "s/", unlinkat (AT_FDCWD, "s/", AT_REMOVEDIR) failed: >> >> unlinkat(4, "d", 0) = 0 >> unlinkat(5, "f", AT_REMOVEDIR) = 0 >> unlinkat(4, "e", AT_REMOVEDIR) = 0 >> unlinkat(AT_FDCWD, "s/", AT_REMOVEDIR) = -1 ENOTDIR (Not a directory) >> rm: cannot remove 's/': Not a directory >> +++ exited with 1 +++ >> >> Now, this looks like a problem with unlinkat. > > Hmm, what about dereferencing command line symlinks with trailing > slashes _before_ entering the FTS loop instead? Thanks, but I'm pretty sure that would be too much work to perform on any slash-decorated operand. Besides, I'm not 100% sold on the idea that that final unlinkat call should be creating a dangling symlink (i.e., by removing the directory to which "s" points).
bug#12350: Composites identified as primes in factor.c (when HAVE_GMP)
Bernhard Voelker wrote: > On 09/04/2012 04:46 PM, Jim Meyering wrote: >> incvolving 3 or more prime factors.) When s = 2, we get the 3/4 factor. > > s/incvolving/involving/ Pádraig Brady wrote: > Miller-Rabin test, one would expect about 9008992/4^64 = 140766 to be s/4^64/64/ ? Fixed both. Thanks. I've also changed s/s/z/ in recognised.
bug#12350: Composites identified as primes in factor.c (when HAVE_GMP)
Jim Meyering wrote:
> Torbjorn Granlund wrote:
>> The very old factoring code cut from an now obsolete version GMP does
>> not pass proper arguments to the mpz_probab_prime_p function. It ask
>> for 3 Miller-Rabin tests only, which is not sufficient.
>
> Hi Torbjorn
>
> Thank you for the patch and explanation.
> I've converted that into the commit below in your name.
> Please proofread it and let me know if you'd like to change anything.
> I tweaked the patch to change MR_REPS from a #define to an enum
> and to add the comment just preceding.
>
> I'll add NEWS and tests separately.
...
> From: Torbjorn Granlund
> Date: Tue, 4 Sep 2012 16:22:47 +0200
> Subject: [PATCH] factor: don't ever declare composites to be prime
Torbjörn, I've just noticed that I misspelled your name above.
Here's the NEWS/tests addition.
Following is an adjusted commit that spells your name properly.
>From e561ff991b74dc19f6728aa1e6e61d1927055ac1 Mon Sep 17 00:00:00 2001
From: Jim Meyering
Date: Tue, 4 Sep 2012 18:26:25 +0200
Subject: [PATCH] factor: doc and tests
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
* NEWS (Bug fixes): Mention it.
* tests/misc/factor.pl: Add five of Torbjörn's tests.
---
NEWS | 3 +++
tests/misc/factor.pl | 5 +
2 files changed, 8 insertions(+)
diff --git a/NEWS b/NEWS
index f3874fd..ffa7939 100644
--- a/NEWS
+++ b/NEWS
@@ -15,6 +15,9 @@ GNU coreutils NEWS-*-
outline -*-
it detects this precise type of cycle, diagnoses it as such and
eventually exits nonzero.
+ factor (when using gmp) would mistakenly declare some composite numbers
+ to be prime, e.g., 465658903, 2242724851, 6635692801.
+
rm -i -d now prompts the user then removes an empty directory, rather
than ignoring the -d option and failing with an 'Is a directory' error.
[bug introduced in coreutils-8.19, with the addition of --dir (-d)]
diff --git a/tests/misc/factor.pl b/tests/misc/factor.pl
index 47f9343..38a5037 100755
--- a/tests/misc/factor.pl
+++ b/tests/misc/factor.pl
@@ -67,6 +67,11 @@ my @Tests =
{OUT => "4: 2 2\n"},
{ERR => "$prog: 'a' is not a valid positive integer\n"},
{EXIT => 1}],
+ ['bug-2012-a', '465658903', {OUT => '15259 30517'}],
+ ['bug-2012-b', '2242724851', {OUT => '33487 66973'}],
+ ['bug-2012-c', '6635692801', {OUT => '57601 115201'}],
+ ['bug-2012-d', '17709149503', {OUT => '94099 188197'}],
+ ['bug-2012-e', '17754345703', {OUT => '94219 188437'}],
);
# Prepend the command line argument and append a newline to end
--
1.7.12.176.g3fc0e4c
>From 4c21a96443ee26eb0d4da31526ce4cf180ac7a4e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Torbj=C3=B6rn=20Granlund?=
Date: Tue, 4 Sep 2012 18:38:29 +0200
Subject: [PATCH] factor: don't ever declare composites to be prime
The multiple-precision factoring code (with HAVE_GMP) was copied from
a now-obsolete version of GMP that did not pass proper arguments to
the mpz_probab_prime_p function. It makes that code perform no more
than 3 Miller-Rabin tests only, which is not sufficient.
A Miller-Rabin test will detect composites with at least a probability
of 3/4. For a uniform random composite, the probability will actually
by much higher.
Or put another way, of the N-3 possible Miller-Rabin tests for checking
the composite N, there is no number N for which more than (N-3)/4 of the
tests will fail to detect the number as a composite. For most numbers N
the number of "false witnesses" will be much, much lower.
Problem numbers are of the for N=pq, p,q prime and (p-1)/(q-1) = s,
where s is a small integer. (There are other problem forms too,
involving 3 or more prime factors.) When s = 2, we get the 3/4 factor.
It is easy to find numbers of that form that cause coreutils' factor to
fail:
465658903
2242724851
6635692801
17709149503
17754345703
20889169003
42743470771
54890944111
72047131003
85862644003
98275842811
114654168091
117225546301
...
There are 9008992 composites of the form with s=2 below 2^64. With 3
Miller-Rabin test, one would expect about 9008992/64 = 140766 to be
invalidly recognized as primes in that range.
* src/factor.c (MR_REPS): Define to 25.
(factor_using_pollard_rho): Use MR_REPS, not 3.
(print_factors_multi): Likewise.
* THANKS.in: Remove my name, now that it will be automatically
included in the generated THANKS file.
---
THANKS.in| 1 -
src/factor.c | 9 ++---
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/THANKS.in b/THANKS.in
index 1580151..2c3f83c 100644
--- a/THANKS.in
+++ b/THANKS.in
@@ -608,7 +608,6 @@ Tony Leneis t...@plaza.ds.adp.com
Tony Robinson a...@eng.cam.ac.uk
Toomas Soometoomas.so...@elion.ee
Toralf Förster toralf.foers...@gmx.de
-Torbjorn Granlund t...@nada.kth.se
Torbjorn Lindgren t...@funcom
bug#12350: Composites identified as primes in factor.c (when HAVE_GMP)
On 09/04/2012 10:42 AM, Jim Meyering wrote: > Jim Meyering wrote: > >> > Torbjorn Granlund wrote: > > Problem numbers are of the for N=pq, p,q prime and (p-1)/(q-1) = s, s/for/form/ -- Eric Blake ebl...@redhat.com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
bug#12339: [PATCH] rm: avoid bogus diagnostic for a slash-decorated symlink-to-dir
Jim Meyering wrote: These commands would evoke an invalid diagnostic: $ mkdir d && ln -s d s && env rm -r s/ rm: cannot remove 's': Too many levels of symbolic links remove.c was stripping trailing slashes from "s/" before passing the name to "rm". But a trailing slash may change the semantics, and thus should not be stripped. I would assert that the trailing "." shouldn't be stripped either. If the directory is a mount point -- then "." would indicate the contents of the mount point -- being after the "/", while the "/" would be the mount point itself. In my case, I wanted to delete everything in the mount point. Removal of the mount point itself would ideally unmount the 'fs', but I won't hold my breath on that one (and leave the root inode of the file system in the unmounted file system -- thus I've effectively removed "." from mountpoint/. Which is different from the "." of a directory entry -- NO I'm not asking for this as a feature, I'm just saying it would be a valid semantic paradigm"). I WOULD want rm dir foobar/. to remove all contents in the directory before failing on ".". The original version of posix didn't have this restriction/problem. POSIX is portability standard -- it should not be regarded as a restrictive standard, unless not doing so would hinder portability. In effect, they are disallowing OS's to make extensions beyond what they allow in the newer standards... That goes WAY beyond a portability standard...
bug#12339: [PATCH] rm: avoid bogus diagnostic for a slash-decorated symlink-to-dir
On 09/04/2012 11:46 AM, Linda A. Walsh wrote: > POSIX is portability standard -- it should not > be regarded as a restrictive standard, unless not doing so would hinder > portability. In effect, they are disallowing OS's to make extensions > beyond > what they allow in the newer standards... That's where you are wrong. POSIX _does_ allow OS's to make extensions beyond the standard. -- Eric Blake ebl...@redhat.com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
bug#12339: [PATCH] rm: avoid bogus diagnostic for a slash-decorated symlink-to-dir
On 09/04/2012 10:46 AM, Linda A. Walsh wrote:
> I would assert that the trailing "." shouldn't be stripped either.
If we were designing anew, I'd be inclined to agree with you.
But there are probably people expecting the standard behavior now,
and there's an argument for not departing from the standard
unless there's a good reason.
Here, if we can provide a command "rm -fr foo/" to do what you
want, then I hope it's merely a minor glitch that "rm -fr foo/."
doesn't do what you want. After all, the version without the
"." is easier to type, and it's more convenient with shells
that autocomplete, so the fact that POSIX requires failure for
the version with the "." shouldn't be that big a deal.
("rm -fr foo/" doesn't work right either, but we're
looking into that)
bug#12339: Bug: rm -fr . doesn't dir depth first deletion yet it is documented to do so.
On 09/03/2012 10:29 AM, Linda Walsh wrote: >> Coreutils has already implemented an exception for the root >> directory: --no-preserve-root (whatever this would be useful >> for except investigating what'd happen in a VM ...). > --- > And by doing so has violated posix, since it doesn't say that if > a switch is present, you can override the 2nd paragraph. Unfortunately, you are wrong on your interpretation of POSIX. POSIX specifically allows implementations to add extensions, such as --no-preserve-root, and that such extensions may explicitly override the required behavior for the application when no extensions are used. After all, that's what extensions are for. In other words, coreutils is _not_ violating POSIX by adding --no-preserve-root as an extension. > If we > are going to be stricly posix, it says if the arg is the root > directory -- the util must have nothing more to do with the operand. POSIX does indeed say that, but it applies only when you use 'rm' in a standards-compliant invocation; the moment you add --no-preserve-root to your invocation, you are no longer using a standards-compliant invocation, so all bets are off as far as POSIX goes. > > If you are are going to rigidly adhere to a broken standard that is > becoming more broken and restricted over time, you can't put in > pet exceptions while using POSIX compat as a justification for > not allowing other, more user friendly, options. If you are going to continually gripe about POSIX rather than constructively contribute towards improving it, then we have no reason to pay attention to your gripes. You can become a member of the Austin Group at no monetary cost, and contribute bug reports to places where you think the standard has got things wrong. But griping to the coreutils list is not the way to get POSIX fixed. If, on the other hand, you are arguing for more long-option extensions to coreutils, then please contribute patches rather than gripes. -- Eric Blake ebl...@redhat.com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
bug#12350: Composites identified as primes in factor.c (when HAVE_GMP)
Jim Meyering writes: Jim Meyering wrote: > Torbjorn Granlund wrote: >> The very old factoring code cut from an now obsolete version GMP does >> not pass proper arguments to the mpz_probab_prime_p function. It ask >> for 3 Miller-Rabin tests only, which is not sufficient. > > Hi Torbjorn > > Thank you for the patch and explanation. > I've converted that into the commit below in your name. > Please proofread it and let me know if you'd like to change anything. > I tweaked the patch to change MR_REPS from a #define to an enum > and to add the comment just preceding. > > I'll add NEWS and tests separately. ... > From: Torbjorn Granlund > Date: Tue, 4 Sep 2012 16:22:47 +0200 > Subject: [PATCH] factor: don't ever declare composites to be prime Torbjörn, I've just noticed that I misspelled your name above. Did you? Well, you misspell recognise too, but then again, most people on the other side of the pond misspell lots of English words. :-) -- Torbjörn
bug#12339: Bug: rm -fr . doesn't dir depth first deletion yet it is documented to do so.
Eric Blake wrote: POSIX does indeed say that, but it applies only when you use 'rm' in a standards-compliant invocation; the moment you add --no-preserve-root to your invocation, you are no longer using a standards-compliant invocation, so all bets are off as far as POSIX goes. -- Which under GNU tools has mean when POSIX_CORRECTLY=1 is set in the environment. It is not. There is nothing preventing tools functioning in a more userfriendly manner than POSIX allows in the absence of a request for POSIX_CORRECTLY. Unless you are telling me that all GNU utils have removed that and always run in POSIX mode by default. POSIX was designed to be a lowest common denominator -- not a highest level allowed -- that's why under Gnu/Linux it's only under a specific request for POSIX compat that lowest level functionality is enforced.
bug#12350: Composites identified as primes in factor.c (when HAVE_GMP)
Pádraig Brady writes: On 09/04/2012 03:46 PM, Jim Meyering wrote: > There are 9008992 composites of the form with s=2 below 2^64. With 3 > Miller-Rabin test, one would expect about 9008992/4^64 = 140766 to be s/4^64/64/ ? For what it's worth I checked the million primes in the range 452,930,477 to 472,882,027 and they're now identified correctly (465658903 was included previously). Note processing time has increased with the patch. On my 2.1GHz i3-2310M, running over the above range used to take 14m, but now takes 18m. It sometimes takes more time to do things correctly. As I mentioned in the original post, we will replace the current code with code that is many times faster. Your example above will run at less than a minute on your system. -- Torbjörn
bug#12350: Composites identified as primes in factor.c (when HAVE_GMP)
On 09/04/2012 07:10 PM, Torbjorn Granlund wrote: Pádraig Brady writes: On 09/04/2012 03:46 PM, Jim Meyering wrote: > There are 9008992 composites of the form with s=2 below 2^64. With 3 > Miller-Rabin test, one would expect about 9008992/4^64 = 140766 to be s/4^64/64/ ? For what it's worth I checked the million primes in the range 452,930,477 to 472,882,027 and they're now identified correctly (465658903 was included previously). Note processing time has increased with the patch. On my 2.1GHz i3-2310M, running over the above range used to take 14m, but now takes 18m. It sometimes takes more time to do things correctly. Sure. I was just quantifying the performance change, for others who may be referencing or noticing patches. (Actually, I'd add a note to the commit message that, this increases calculations by about 25%). As I mentioned in the original post, we will replace the current code with code that is many times faster. Your example above will run at less than a minute on your system. I'd left my test files in place in anticipation ;) thanks, Pádraig.
bug#12350: Composites identified as primes in factor.c (when HAVE_GMP)
Pádraig Brady writes: Sure. I was just quantifying the performance change, for others who may be referencing or noticing patches. (Actually, I'd add a note to the commit message that, this increases calculations by about 25%). And surely mode for certain cases. We spend 25/3 or about 8 times more effort in Miller Rabin. > As I mentioned in the original post, we will replace the current code > with code that is many times faster. Your example above will run at > less than a minute on your system. I'd left my test files in place in anticipation ;) Please do, and let me and Niels know if it takes more than 45s. Your test case takes 28s on my 3.3 GHz Sandy bridge system with our current code. I'm a little disappointed the code doesn't beat the old code more for small factorisations. -- Torbjörn
bug#12339: [PATCH] rm: avoid bogus diagnostic for a slash-decorated symlink-to-dir
Paul Eggert wrote: On 09/04/2012 10:46 AM, Linda A. Walsh wrote: I would assert that the trailing "." shouldn't be stripped either. If we were designing anew, I'd be inclined to agree with you. But there are probably people expecting the standard behavior now, i.e. expecting it to do nothing useful other than issue an error? Who would type that in, expecting that? What useful purpose does it serve? and there's an argument for not departing from the standard unless there's a good reason. Usability, Expressiveness. Here, if we can provide a command "rm -fr foo/" to do what you want, then I hope it's merely a minor glitch that "rm -fr foo/." Ok, will foo/ remove the contents only and not the directory? If so, then it's a replacement -- that sounds like what is being talked about .. yes? It seems so fragile -- because some utils require the /. to make sure you don't ALSO remove the directory. if I remove foo/. I know, unambigously, that the directory "foo won't be removed. With "foo/", it will depend on the program. That means other programs will have to change to be compat with the new standard -- when they were all compat before when foo/. did a recursive remove and gave an error at the end about can't remove "." That was a behavior I've relied on...(i.e. if it really did remove ".", that would defeat the purpose of the of the safety invocation. However, adding foo/ -- does that imply that ./ will also work? so now rm -fr ./ will work? That's awfully close to "rm -fr /." just as rm -fr ./* is close to rm -fr /.* I've never relied on rm to protect me from rm -fr /, which is why I have tended to use relative paths with "." and ending in "." if I wanted to keep the dir, or not -- as there is ALOT of code out there that removes trailing "slashes"... (like the code under discussion -- perfect example)it is by far not the only one to treat '/' as option at the end of a path.
bug#12336: test bug
Hi Bob, thanks for u r reply. See i will explain u neatly. I am using bash shell. When I tried to do this i getting this.I am getting this error. [balamup2@cl-flor-dvvm026 ~]$ echo $SHELL */bin/bash* [balamup2@cl-flor-dvvm026 ~]$ ls *test1.txt test3.txt testdir test.txt* [balamup2@cl-flor-dvvm026 ~]$ test -f test*.txt *-bash: test: too many arguments* But my friend is using ksh shell. He is fine with that command . I will past this O/P also. $ echo $SHELL */usr/bin/ksh* $ ls *123.txt a1.txt abc.txt a.txt dead.letter nohup.out ww1.txt* $ test -f a*.txt $ echo $? *0* Please take a deep look and resolve this problem
bug#12336: test bug
Please keep the mailing list in the reply. I have set Reply-To accordingly. Bala Murugan wrote: > I am using bash shell. When I tried to do this i getting this.I am getting > this error. > > [balamup2@cl-flor-dvvm026 ~]$ echo $SHELL > */bin/bash* I am sure that those '*' characters are not there. You must have added them for emphasis. > [balamup2@cl-flor-dvvm026 ~]$ ls > *test1.txt test3.txt testdir test.txt* So you have three files that match test*.txt in that directory. > [balamup2@cl-flor-dvvm026 ~]$ test -f test*.txt > *-bash: test: too many arguments* That is correct. You are passing in too many arguments. The test -f operator requires one argument. You have passed *three* arguments. That is two too many. You can verify this by using 'echo'. Try this: $ echo test -f test*.txt Using that you will see that the test*.txt is matching and being expanded by the shell into three arguments. It is almost always incorrect to pass a '*' or other shell file glob metacharacter unquoted on the command line to test. Your usage is very problematic. Instead you may want to loop over the files. for file in test*.txt; do test -f "$file" && echo file: "$file" || echo nofile: "$file" done > But my friend is using ksh shell. He is fine with that command . But does your friend have three files that match that file glob in his directory? Have you friend do this: $ touch test1.txt test2.txt test3.txt $ echo test -f test*.txt $ test -f test*.txt By creating the extra files with touch that should ensure that multiple files are matched. > I will past this O/P also. > > $ echo $SHELL > */usr/bin/ksh* > > $ ls > *123.txt a1.txt abc.txt a.txt dead.letter nohup.out ww1.txt* > > $ test -f a*.txt > $ echo $? > *0* I cannot reproduce that result. I tried this: $ mkdir /tmp/testdir $ cd /tmp/testdir /testdir$ touch 123.txt a1.txt abc.txt a.txt dead.letter nohup.out ww1.txt /testdir$ ls a*.txt a.txt a1.txt abc.txt /testdir$ test -f a*.txt bash: test: too many arguments That is correct behavior because a*.txt matches several files. > Please take a deep look and resolve this problem I am sorry but I think this is simply incorrect usage. Bob
bug#12339: [PATCH] rm: avoid bogus diagnostic for a slash-decorated symlink-to-dir
On 09/04/2012 04:21 PM, Linda A. Walsh wrote:> Paul Eggert wrote: > expecting it to do nothing useful other than issue an error? Sure. People might expect the utility to complain about what they consider to be obvious typos, rather than to remove files they don't expect to be removed. For example: rm -fr .* safely removes all file names beginning with "." in the working directory, without inadvertently removing file names that do not begin with ".", or files in the parent directory. I can see people being used to this sort of thing, and being annoyed greatly if we change it. > Ok, will foo/ remove the contents only and not the directory? No, "rm -fr foo/" removes the directory too. > It seems so fragile -- because some utils require the /. to make > sure you don't ALSO remove the directory. Which utilities are those? The standard POSIX style is that foo/ names the directory. > However, adding foo/ -- does that imply that ./ will also work? No, "./" is treated as ".". Sorry.
bug#12339: [PATCH] rm: avoid bogus diagnostic for a slash-decorated symlink-to-dir
Paul Eggert wrote: On 09/04/2012 04:21 PM, Linda A. Walsh wrote:> Paul Eggert wrote: expecting it to do nothing useful other than issue an error? Sure. People might expect the utility to complain about what they consider to be obvious typos, rather than to remove files they don't expect to be removed. For example: rm -fr .* safely removes all file names beginning with "." in the working directory, without inadvertently removing file names that do not begin with ".", or files in the parent directory. I can see people being used to this sort of thing, and being annoyed greatly if we change it. i wouldn't regard that as safe. I always use rm -fr .[^.]* Why is that interfaces must sync to the lowest level. unix was NOT supposed to be that way. Ok, will foo/ remove the contents only and not the directory? No, "rm -fr foo/" removes the directory too. So how do delete all files in the directory without wild cards? (as you don't know how or if they will be expanded). (only VERY recently has bash included "." in "*" -- it used to be ignored and still is if you have flags set to ignore it. Shells don't consistently expand wildcards, and the OS doesn't expand them at all (i.e. when passed in 'exec')... This is what I mean about useful features being removed in order to dumb down the interface -- as dir/" also tries to remove the dir... which isn't likely to work on a mount point.
bug#12339: [PATCH] rm: avoid bogus diagnostic for a slash-decorated symlink-to-dir
On 09/04/2012 06:55 PM, Linda A. Walsh wrote: > So how do delete all files in the directory without wild cards? Why tie your hands behind your back? Use wild cards: cd DIRECTORY && rm -fr * .[!.] .??* If you do this a lot, put it into a shell script. It's not like the need is all that common, for most people, but if you need it that's what scripts are for. If you fiddle with your shell settings about how wildcards are expanded, make sure your script does wildcards in the standard way.
bug#12339: [PATCH] rm: avoid bogus diagnostic for a slash-decorated symlink-to-dir
On 09/05/2012 08:25 AM, Paul Eggert wrote: > On 09/04/2012 06:55 PM, Linda A. Walsh wrote: > >> So how do delete all files in the directory without wild cards? > > Why tie your hands behind your back? Use wild cards: > > cd DIRECTORY && rm -fr * .[!.] .??* > > If you do this a lot, put it into a shell script. > It's not like the need is all that common, for most people, > but if you need it that's what scripts are for. > > If you fiddle with your shell settings about > how wildcards are expanded, make > sure your script does wildcards in the standard way. Exactly, we're now at the point where globbing is the topic, much way away from coreutils and rm. Bash knows dotglob: $ mkdir d; cd d $ touch .a b ; mkdir c $ ( shopt -s dotglob ; rm -rv * ) removed `.a' removed `b' removed directory: `c' Have a nice day, Berny
bug#12350: Composites identified as primes in factor.c (when HAVE_GMP)
Eric Blake wrote: > On 09/04/2012 10:42 AM, Jim Meyering wrote: >> Jim Meyering wrote: >> >>> > Torbjorn Granlund wrote: >> >> Problem numbers are of the for N=pq, p,q prime and (p-1)/(q-1) = s, > > s/for/form/ Fixed. Thanks.
bug#12350: Composites identified as primes in factor.c (when HAVE_GMP)
Torbjorn Granlund wrote: > Jim Meyering writes: > > Jim Meyering wrote: > > > Torbjorn Granlund wrote: > >> The very old factoring code cut from an now obsolete version GMP does > >> not pass proper arguments to the mpz_probab_prime_p function. It ask > >> for 3 Miller-Rabin tests only, which is not sufficient. > > > > Hi Torbjorn > > > > Thank you for the patch and explanation. > > I've converted that into the commit below in your name. > > Please proofread it and let me know if you'd like to change anything. > > I tweaked the patch to change MR_REPS from a #define to an enum > > and to add the comment just preceding. > > > > I'll add NEWS and tests separately. > ... > > From: Torbjorn Granlund > > Date: Tue, 4 Sep 2012 16:22:47 +0200 > > Subject: [PATCH] factor: don't ever declare composites to be prime > > Torbjörn, I've just noticed that I misspelled your name above. > > Did you? I meant that I used Torbjorn rather than Torbjörn. > Well, you misspell recognise too, but then again, most people > on the other side of the pond misspell lots of English words. :-) Yes, the dichotomy is unfortunate. Over the years, it has even caused interface problems, i.e., with --colours vs --colors and $LS_COLOURS vs LS_COLORS. I wanted to settle on one, and US spelling is more common than British so I settled on that.
bug#12339: [PATCH] rm: avoid bogus diagnostic for a slash-decorated symlink-to-dir
Paul Eggert wrote: On 09/04/2012 06:55 PM, Linda A. Walsh wrote: So how do delete all files in the directory without wild cards? Why tie your hands behind your back? Use wild cards: cd DIRECTORY && rm -fr * .[!.] .??* If you do this a lot, put it into a shell script. It's not like the need is all that common, for most people, but if you need it that's what scripts are for. If you fiddle with your shell settings about how wildcards are expanded, make sure your script does wildcards in the standard way. --- I'm not talking for interactive use... I'm talking for use in a script that might run on systems that are not mine -- so I can't rely on shell settings.
