Hi, The changes from yesterday, that make gzip use 'quotearg' in the "C" locale, have a major drawback: For normal users that use UTF-8 file names, the diagnostics emitted by gzip make an unreasonable use of escape sequences.
UTF-8 file names and directory names are commonplace nowadays. For instance,
the "special" folders in the users home directory, created by a Kubuntu 26.04
installation, have the following names:
English French German
Desktop Desktop Desktop
Documents Documents Dokumente
Downloads Téléchargements Downloads
Music Musique Musik
Pictures Images Bilder
Public Public Öffentlich
Templates Modèles Vorlagen
Videos Vidéos Videos
Now, imagine simply a user wanting to compress a file in their
~/Téléchargements/ directory, and using the wrong option.
Find attached the test data.
This is what the user's experience looks like on various
operating systems (a GNU system, musl libc, macOS, Haiku):
On a GNU system (Ubuntu 24.04):
Before:
$ gzip -d ~/Téléchargements/Jörg.png
gzip: /home/bruno/Téléchargements/Jörg.png: unknown suffix -- ignored
After:
$ ./gzip -d ~/Téléchargements/Jörg.png
gzip:
'/home/bruno/T'$'\303\251''l'$'\303\251''chargements/J'$'\303\266''rg.png':
unknown suffix -- ignored
On musl libc (Alpine Linux 3.22):
Before:
$ gzip -d ~/Téléchargements/Jörg.png
gzip: /home/bruno/Téléchargements/Jörg.png: unknown suffix - ignored
After:
$ ./gzip -d ~/Téléchargements/Jörg.png
gzip:
'/home/bruno/T'$'\303\251''l'$'\303\251''chargements/J'$'\303\266''rg.png':
unknown suffix -- ignored
On macOS (cfarm104.cfarm.net):
Before:
$ gzip -d ~/Téléchargements/Jörg.png
gzip: /Users/haible/Téléchargements/Jörg.png: unknown suffix -- ignored
After:
$ ./gzip -d ~/Téléchargements/Jörg.png
gzip:
'/Users/haible/T'$'\303\251''l'$'\303\251''chargements/J'$'\303\266''rg.png':
unknown suffix -- ignored
On Haiku:
Before:
$ /bin/gzip -d ~/Téléchargements/Jörg.png
gzip: /boot/home/Téléchargements/Jörg.png: unknown suffix -- ignored
After:
$ ./gzip -d ~/Téléchargements/Jörg.png
gzip:
'/boot/home/T'$'\303\251''l'$'\303\251''chargements/J'$'\303\266''rg.png':
unknown suffix -- ignored
What's a normal user's reaction to such output?
- Some users might think their file system is broken. Then they might
check with "ls". Or they might delete the file and download it again.
Or they might fiddle with backups. Etc.
- Other users might recognize that these are octal escape sequences.
And immediately wonder why 'gzip' behaves like a program from the
1970ies or 1980ies by
- treating non-ASCII characters as invalid (many programs did so
in the 1980ies), and
- using octal escape sequence, where the world switched to hexadecimal
escape sequences for the most part, a long long time ago.
Is it good if 'gzip' gives the impression of being *that* old?
What is the purpose of 'quotearg' in the first place? As far as I understand
it, it is to avoid outputting terminal escape sequences that originate from
file names (or command-line arguments) and that would possibly fake some
output in the terminal.
But if the user is in a UTF-8 locale — which is frequent nowadays —, UTF-8
encoded characters are *not* dangerous escape sequences that must be
backslash-escaped.
So, the entire idea of using quotearg in the "C" locale is not useful.
I really mean that: It's a show-stopper. You can't make a release with
a diagnostics behaviour like that.
What are the possible fixes for the problem?
A) tryA.diff (attached) leads to
$ ./gzip -d ~/Téléchargements/Jörg.png
gzip: /home/bruno/Téléchargements/Jörg.png: unknown suffix -- ignored
It adds quotes if the file name contains spaces.
But maybe this is less that what one wants: It does not protect against
dangerous escape sequences.
B) tryB.diff (attached) leads to
$ ./gzip -d ~/Téléchargements/Jörg.png
gzip: /home/bruno/Téléchargements/Jörg.png: unknown suffix -- ignored
$ ./gzip -d ~/Téléchargements/B*
gzip: '/home/bruno/Téléchargements/Böse Bübchen': unknown suffix -- ignored
$ ./gzip -d ~/Téléchargements/f*
gzip: '/home/bruno/Téléchargements/foo\003bar': unknown suffix -- ignored
This is reasonable behaviour: It uses the current locale to know which
characters it can output literally, and it uses backslash-escape sequences
in the (rare, but dangerous) case that a file name contains dangerous
escape sequences.
I would suggest to adopt tryB.diff.
Note that tryB.diff does not make use of the Gnulib module 'setlocale'.
You would need that module for decent behaviour on native Windows;
I understand that native Windows is of secondary importance for 'gzip'.
Bruno
testdata.tar.gz
Description: application/compressed-tar
diff --git a/util.c b/util.c
index eca9fbe..e647e86 100644
--- a/util.c
+++ b/util.c
@@ -430,5 +430,5 @@ quotef (char const *arg)
char *
quotef_n (int n, char const *arg)
{
- return quotearg_n_style_colon (n, shell_escape_quoting_style, arg);
+ return quotearg_n_style_colon (n, shell_quoting_style, arg);
}
diff --git a/bootstrap.conf b/bootstrap.conf
index ef0b96d..7a763a5 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -74,17 +74,13 @@ yesno
# Additional gnulib-tool options to use. Use "\newline" to break lines.
# We don't need the getline and rpmatch, since yesno doesn't do i18n.
gnulib_tool_option_extras="\
- --avoid c32isprint\
--avoid getline\
--avoid gnulib-i18n\
- --avoid mbrtoc32\
- --avoid mbsinit\
--avoid threadlib\
--avoid rpmatch\
--avoid windows-once\
--avoid windows-spin\
--avoid windows-tls\
- --avoid uchar-h\
--makefile-name=gnulib.mk\
"
diff --git a/configure.ac b/configure.ac
index f8ce122..ca3126e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -30,9 +30,6 @@ AM_INIT_AUTOMAKE([1.11 subdir-objects dist-xz dist-zip
color-tests parallel-tests])
AM_SILENT_RULES([yes]) # make --enable-silent-rules the default.
-dnl gzip does not use setlocale.
-AC_DEFINE([USE_C_LOCALE], [1], [Define to 1 if only the C locale is used.])
-
dnl POSIXCHECK is worthwhile for maintainers, but adds several seconds
dnl (more than 10% execution time) to ./configure, with no benefit for
dnl most users. Using it to look for bugs requires:
diff --git a/gzip.c b/gzip.c
index 302efe9..a8e8dab 100644
--- a/gzip.c
+++ b/gzip.c
@@ -77,6 +77,7 @@ static char const license_msg[] =
#include <getopt.h>
#include <inttypes.h>
#include <limits.h>
+#include <locale.h>
#include <signal.h>
#include <stdcountof.h>
#include <stddef.h>
@@ -422,6 +423,15 @@ int main (int argc, char **argv)
EXPAND(argc, argv); /* wild card expansion if necessary */
+ /* On POSIX-like systems, where the locale is specified by
+ environment variables such as LANG, LC_CTYPE, LC_ALL,
+ call setlocale, to obey these environment variables.
+ In this program, we don't use the LC_MESSAGES category;
+ only LC_CTYPE is needed (for quotearg). */
+#if !(defined _WIN32 && !defined __CYGWIN__)
+ setlocale (LC_CTYPE, "");
+#endif
+
program_name = gzip_base_name (argv[0]);
proglen = strlen (program_name);
