sed issue? [was: Subject: [GNU Autoconf 2.67] testsuite: 233 failed]

[Eric Blake]

[adding bug-gnu-utils, for what may be a sed bug, or at least a usage bug in how autoconf is using sed]

On 09/20/2010 04:32 PM, Eric Blake wrote:

I managed to get on a Darwin machine (MacOS X 10.5.5), and reproduced
your failure:

+allowed-chars Zardoz differ: char 168, line 2
./torture.at:937: exit code was 1, expected 0

I'm now looking into it further, first by trying to see if it is still
present in the latest autoconf.git.

On my Fedora box, with sed 4.2.1, I see different behavior for:

$ LC_ALL=en_US.UTF-8 sed -f file.sed file.awk > out1
$ LC_ALL=C sed -f file.sed file.awk > out2





--
Eric Blake   ebl...@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

Yes, the problem is still present in the latest autoconf.git. It appears to be that we are triggering some sort of locale-specific bug in GNU sed, and that for some reason, using LC_ALL=C on Darwin is not enough to avoid that bug. I'm attaching some files pruned from the test, and hoping that they don't get corrupted during mailing since they include some 8-bit bytes. Where the behavior of out2 is what configure must use in order to generate a valid config.status for the funky contents of the shell variable $zardoz. But on Darwin, even with LC_ALL=C, I'm getting the out1 behavior, which leads to a broken config.status. Maybe the sed script in file.sed is non-portable? It's certainly more complex than the normal run-of-the-mill sed script. Or maybe it is that the regex '.' has problems matching non-characters, and the definition of the various locales determine whether 8-bit bytes are characters or not. Is there any portable way to guarantee a single-byte locale where '.' matches all possible 8-bit bytes?

file.awk
Description: Binary data

h
s/^/S["/; s/!.*/"]=/
p
g
s/^[^!]*!//
:repl
t repl
s/%!_!# $//
t delim
:nl
h
s/\(.\{148\}\)..*/\1/
t more1
s/["\\]/\\&/g; s/^/"/; s/$/\\n"\\/
p
n
b repl
:more1
s/["\\]/\\&/g; s/^/"/; s/$/"\\/
p
g
s/.\{148\}//
t nl
:delim
h
s/\(.\{148\}\)..*/\1/
t more2
s/["\\]/\\&/g; s/^/"/; s/$/"/
p
b
:more2
s/["\\]/\\&/g; s/^/"/; s/$/"\\/
p
g
s/.\{148\}//
t delim

out1
Description: Binary data

out2
Description: Binary data