poll(2) emulation doesn't work well on a file descriptor
Hi,
When I tried a tiny program which uses gnulib's poll(2) emulation on
MacOS X 10.4, I found a bug. gnulib's poll(2) uses recv(2) with
MSG_PEEK to support POLLHUP. However, recv(2) is only applicable to a
socket, not to a file descriptor.
Though I don't know how to fix it, I think it can be by-passed if
(pfd[i].events & POLLHUP) == 0? Here is the patch
Index: poll.c
===
RCS file: /sources/gnulib/gnulib/lib/poll.c,v
retrieving revision 1.4
diff -u -r1.4 poll.c
--- poll.c 19 Sep 2005 17:28:14 - 1.4
+++ poll.c 11 Aug 2006 07:32:03 -
@@ -155,7 +155,8 @@
{
/* support for POLLHUP. An hung up descriptor does not
increase the return value! */
- if (recv (pfd[i].fd, data, 64, MSG_PEEK) == -1)
+ if (pfd[i].events & POLLHUP &&
+ recv (pfd[i].fd, data, 64, MSG_PEEK) == -1)
{
if (errno == ESHUTDOWN || errno == ECONNRESET
|| errno == ECONNABORTED || errno == ENETRESET)
Regards,
--
Daiki Ueno
Re: poll(2) emulation doesn't work well on a file descriptor
>>>>> In <[EMAIL PROTECTED]> >>>>> Paolo Bonzini <[EMAIL PROTECTED]> wrote: > Bruno Haible wrote: > > Daiki Ueno wrote: > > > >> When I tried a tiny program which uses gnulib's poll(2) emulation on > >> MacOS X 10.4, I found a bug. gnulib's poll(2) uses recv(2) with > >> MSG_PEEK to support POLLHUP. However, recv(2) is only applicable to a > >> socket, not to a file descriptor. > >> > It's much worse than that. On some other kinds of descriptor, recv > works, but ignores MSG_PEEK! > I have a similar patch using FIONREAD instead of recv for Mac OS only, > but the bottom line is that you cannot rely on POLLHUP for Mac OS :-( > I'll polish the patch and send it soon. A month passed. Any news? Regards, -- Daiki Ueno
Re: 'inline' changes pushed
Hi, Paul Eggert writes: > No comments about the 'inline' fixes I posted a couple of > weeks ago, so I just now pushed them into the gnulib master. I'm using gcc 4.7 with gcc -std=gnu99. After commit 618fd97d, list functions are now defined as "extern inline" (previously "static inline"). This causes "multiple definition of `foo'" errors when linking together multiple object files[1]. For my case, bootstrapping gettext from the git master fails with: .libs/gl_xlist.o: In function `gl_list_nx_create_empty':/home/ueno/devel/gettext/gettext-tools/gnulib-lib/gl_list.h:557: multiple definition of `gl_list_nx_create_empty' .libs/gl_list.o:/home/ueno/devel/gettext/gettext-tools/gnulib-lib/gl_list.h:557: first defined here ... Can it be fixed by caller side? Footnotes: [1] http://gcc.gnu.org/gcc-4.3/porting_to.html Regards, -- Daiki Ueno
Re: [bug-gettext] [PATCH] m4: use AC_PROG_MKDIR_P, not the deprecated AM_PROG_MKDIR_P
Hi Stefano,
Stefano Lattarini writes:
> AM_PROG_MKDIR_P has been deprecated in Automake 1.12.1, and will
> be removed in Automake 1.13.
[...]
> gettext-runtime/m4/intl.m4 | 6 +++---
> gettext-runtime/m4/po.m4 | 6 +++---
Since these files also reside in gnulib/m4, maybe it is good to update
gnulib as well?
(I still don't fully understand the relationship between gnulib and
gettext, so perhaps I'm missing the point, but gettext-tools actually
seems to pick gnulib's {po,intl}.m4 through bison-i18n.)
Regards,
--
Daiki Ueno
Re: [bug-gettext] [PATCH] m4: use AC_PROG_MKDIR_P, not the deprecated AM_PROG_MKDIR_P
Paul Eggert writes: > On 12/06/12 12:10, Stefano Lattarini wrote: >> Not sure whether Gnulib simply syncs those files from Gettext, in > > It used to, and the mechanism is still there, > but it's not being used now. It'd be easy to > start it up again, if you just give us a list > of files to copy (from and to locations). Please find below the list (taken from the output of "make -n distcheck-hook" in gettext). Note that some files need adjustment of license headers (marked as *), since the most part of gettext-runtime is LGPLv2+ while those files from gnulib are under different licenses. m4/codeset.m4 -> gettext-runtime/m4/codeset.m4 m4/fcntl-o.m4 -> gettext-runtime/m4/fcntl-o.m4 m4/gettext.m4 -> gettext-runtime/m4/gettext.m4 m4/glibc2.m4 -> gettext-runtime/m4/glibc2.m4 m4/glibc21.m4 -> gettext-runtime/m4/glibc21.m4 m4/iconv.m4 -> gettext-runtime/m4/iconv.m4 m4/intdiv0.m4 -> gettext-runtime/m4/intdiv0.m4 m4/intl.m4 -> gettext-runtime/m4/intl.m4 m4/intldir.m4 -> gettext-runtime/m4/intldir.m4 m4/intlmacosx.m4 -> gettext-runtime/m4/intlmacosx.m4 m4/intmax.m4 -> gettext-runtime/m4/intmax.m4 m4/inttypes-pri.m4 -> gettext-runtime/m4/inttypes-pri.m4 m4/inttypes_h.m4 -> gettext-runtime/m4/inttypes_h.m4 m4/lcmessage.m4 -> gettext-runtime/m4/lcmessage.m4 m4/lock.m4 -> gettext-runtime/m4/lock.m4 m4/longlong.m4 -> gettext-runtime/m4/longlong.m4 m4/nls.m4 -> gettext-runtime/m4/nls.m4 m4/po.m4 -> gettext-runtime/m4/po.m4 m4/gettext.m4 -> gettext-runtime/m4/gettext.m4 m4/printf-posix.m4 -> gettext-runtime/m4/printf-posix.m4 m4/progtest.m4 -> gettext-runtime/m4/progtest.m4 m4/size_max.m4 -> gettext-runtime/m4/size_max.m4 m4/stdint_h.m4 -> gettext-runtime/m4/stdint_h.m4 m4/threadlib.m4 -> gettext-runtime/m4/threadlib.m4 m4/uintmax_t.m4 -> gettext-runtime/m4/uintmax_t.m4 m4/visibility.m4 -> gettext-runtime/m4/visibility.m4 m4/wchar_t.m4 -> gettext-runtime/m4/wchar_t.m4 m4/wint_t.m4 -> gettext-runtime/m4/wint_t.m4 m4/xsize.m4 -> gettext-runtime/m4/xsize.m4 m4/intmax_t.m4 -> gettext-runtime/libasprintf/gnulib-m4/intmax_t.m4 lib/config.charset -> gettext-runtime/intl/config.charset (*) lib/localcharset.h -> gettext-runtime/intl/localcharset.h (*) lib/localcharset.c -> gettext-runtime/intl/localcharset.c (*) lib/localename.c -> gettext-runtime/intl/localename.c (*) lib/glthread/lock.h -> gettext-runtime/intl/lock.h (*) lib/glthread/lock.c -> gettext-runtime/intl/lock.c (*) lib/verify.h -> gettext-runtime/intl/verify.h (*) lib/ref-add.sin -> gettext-runtime/intl/ref-add.sin (*) lib/ref-del.sin -> gettext-runtime/intl/ref-del.sin (*) lib/relocatable.h -> gettext-runtime/intl/relocatable.h (*) lib/relocatable.c -> gettext-runtime/intl/relocatable.c (*) lib/vasnprintf.c -> gettext-runtime/intl/vasnprintf.c (*) lib/xsize.h -> gettext-runtime/intl/xsize.h (*) Regards, -- Daiki Ueno
Re: [bug-gettext] gettext-tools test failures on Darwin
Daiki Ueno writes: > After the following change (in gnulib), locale_charset() returns "ASCII" > on Mac OS X, even if thread's locale is set to "de_DE.UTF-8", say. So > the solution would be either to make sure MB_CUR_MAX > 1 somehow in the > test, or to make locale_charset() thread-safe. Just tried the latter. I've tested the attached patch on Mac OS X and it still returns "ASCII" until the first uselocale() or setlocale(). --- ChangeLog | 6 ++ lib/localcharset.c | 7 ++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/ChangeLog b/ChangeLog index 413e435..87e2339 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,9 @@ +2012-12-20 Daiki Ueno + + localecharset: make locale_charset thread-safe on Mac OS X + * lib/localcharset.c (locale_charset) [DARWIN7]: Use MB_CUR_MAX_L + instead of MB_CUR_MAX. + 2012-12-08 Stefano Lattarini maint.mk: avoid extra forks diff --git a/lib/localcharset.c b/lib/localcharset.c index 1a94042..1ad03d7 100644 --- a/lib/localcharset.c +++ b/lib/localcharset.c @@ -65,6 +65,11 @@ # include #endif +/* For MB_CUR_MAX_L */ +#if defined DARWIN7 +# include +#endif + #if ENABLE_RELOCATABLE # include "relocatable.h" #else @@ -545,7 +550,7 @@ locale_charset (void) #ifdef DARWIN7 /* Mac OS X sets MB_CUR_MAX to 1 when LC_ALL=C, and "UTF-8" (the default codeset) does not work when MB_CUR_MAX is 1. */ - if (strcmp (codeset, "UTF-8") == 0 && MB_CUR_MAX <= 1) + if (strcmp (codeset, "UTF-8") == 0 && MB_CUR_MAX_L (uselocale (NULL)) <= 1) codeset = "ASCII"; #endif -- 1.7.11.7 Regards, -- Daiki Ueno
Re: [bug-gettext] gnulib configure.ac and autopoint?
Paul Eggert writes:
> We should alter gnulib-tool to not indent that line,
> in the short term. In the longer term autopoint should be fixed
> to work even if the AM_GNU_GETTEXT_VERSION line is indented,
> so I'll CC: this message to bug-gnu-gettext.
>
> By the way, I'm puzzled that this problem was discovered just now.
> Isn't there a similar problem with autopoint 0.18.1?
>
> Anyway, here's the patch I installed into gnulib.
Installed the following fix into gettext as well.
>From a071c8a3cdff4f6487390ceeb720cc9ae66e1474 Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Mon, 7 Jan 2013 08:09:09 +0900
Subject: [PATCH] Make autopoint AM_GNU_GETTEXT_VERSION handling more robust.
---
gettext-tools/misc/ChangeLog|7 +++
gettext-tools/misc/autopoint.in | 12 +---
2 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/gettext-tools/misc/ChangeLog b/gettext-tools/misc/ChangeLog
index c943598..6351ace 100644
--- a/gettext-tools/misc/ChangeLog
+++ b/gettext-tools/misc/ChangeLog
@@ -1,3 +1,10 @@
+2013-01-07 Daiki Ueno
+
+ * autopoint.in: Extract version number from configure.ac in a more
+ robust way.
+ Reported by Tom G. Christensen in
+ <http://lists.gnu.org/archive/html/bug-gnulib/2013-01/msg00053.html>.
+
2012-12-25 Daiki Ueno
* autopoint.in: Update for 0.18.2.
diff --git a/gettext-tools/misc/autopoint.in b/gettext-tools/misc/autopoint.in
index dd7e203..717b0db 100644
--- a/gettext-tools/misc/autopoint.in
+++ b/gettext-tools/misc/autopoint.in
@@ -267,9 +267,15 @@ fi
# Check whether the -V option and the version number in configure.in match.
# At least one of the two must be given. If both are given, they must agree.
-sed_extract_AM_GNU_GETTEXT_VERSION_argument='s/^AM_GNU_GETTEXT_VERSION(\([^()]*\)).*$/\1/'
-sed_remove_outer_brackets='s/^\[\(.*\)\]$/\1/'
-xver=`cat "$configure_in" | grep '^AM_GNU_GETTEXT_VERSION(' | sed -n -e
"$sed_extract_AM_GNU_GETTEXT_VERSION_argument"p | sed -e
"$sed_remove_outer_brackets" | sed -e 1q`
+sed_extract_AM_GNU_GETTEXT_VERSION_argument='
+s,#.*$,,; s,^dnl .*$,,; s, dnl .*$,,;
+/AM_GNU_GETTEXT_VERSION/ {
+s,^.*AM_GNU_GETTEXT_VERSION(\([^()]*\)).*$,xver=\1,
+s,^xver=\[\(.*\)\]$,xver=\1,
+p
+}
+d'
+eval `sed "$sed_extract_AM_GNU_GETTEXT_VERSION_argument" "$configure_in"`
if test -z "$xver" && test -f intl/VERSION; then
xver=`cat intl/VERSION | LC_ALL=C sed -n -e
's/^.*gettext-\([-+_.0-9A-Za-z]*\).*$/\1/p'`
fi
--
1.7.10.4
Re: [bug-gettext] gettext 0.18.2 fails to compile without optimisations
Hi, Sam Thursfield writes: > I find that if gettext 0.18.2 is configured as follows: > > CFLAGS= ./configure > > The compilation fails later on with the following error: > > ... > Making all in tests > make[3]: Entering directory `/gettext.build/gettext-tools/tests' > /bin/sh ../libtool --tag=CC --mode=link gcc -std=gnu99 \ > -o gettextpo-1-prg gettextpo_1_prg-gettextpo-1-prg.o \ > ../libgettextpo/libgettextpo.la ../intl/libgnuintl.la > libtool: link: gcc -std=gnu99 -o .libs/gettextpo-1-prg \ > gettextpo_1_prg-gettextpo-1-prg.o ../libgettextpo/.libs \ > /libgettextpo.so ../intl/.libs/libgnuintl.so -lc > ../intl/.libs/libgnuintl.so: undefined reference to `xsum' > ../intl/.libs/libgnuintl.so: undefined reference to `xmax' > ../intl/.libs/libgnuintl.so: undefined reference to `xsum4' > collect2: ld returned 1 exit status > > The compiler is GCC 4.6.2 on x86_64. Compilation succeeds if the > default CFLAGS are used, so I think the cause of the problem is that > without inlining, these functions do not work somehow. I cannot reproduce this on x86_64 GNU/Linux with GCC 4.6.3 and 4.7.2, but can on mingw64 cross build[1]. It seems that xsize.o needs to be linked to the library, after recent gnulib's "extern inline" changes. Could you try the attached patch? $ tar xf gettext-0.18.2.tar.gz $ cd getteext-0.18.2 $ patch -p1 < patch $ pushd gettext-runtime/libasprintf $ autoreconf -f $ popd $ CFLAGS= ./configure && make I'm not 100% sure if this is the right fix, so Cc'ed bug-gnulib. Footnotes: [1] https://savannah.gnu.org/bugs/?37981 Regards, -- Daiki Ueno >From 4c197b383b5922cd277e53cc6bd5d05eb56083eb Mon Sep 17 00:00:00 2001 From: Daiki Ueno Date: Wed, 16 Jan 2013 17:08:28 +0900 Subject: [PATCH] Make sure to link libraries with xsize.o. After recent C99-style extern inline changes in gnulib, functions defined in xsize.h are now declared as "extern inline". That means the libraries using those functions need to be linked to xsize.o. --- gettext-runtime/intl/Makefile.in| 6 +- gettext-runtime/intl/xsize.c| 3 +++ gettext-runtime/libasprintf/Makefile.am | 2 +- gettext-runtime/libasprintf/xsize.c | 3 +++ 4 files changed, 12 insertions(+), 2 deletions(-) create mode 100644 gettext-runtime/intl/xsize.c create mode 100644 gettext-runtime/libasprintf/xsize.c diff --git a/gettext-runtime/intl/Makefile.in b/gettext-runtime/intl/Makefile.in index 006d3b0..449fdb1 100644 --- a/gettext-runtime/intl/Makefile.in +++ b/gettext-runtime/intl/Makefile.in @@ -158,6 +158,7 @@ SOURCES = \ printf.c \ setlocale.c \ version.c \ + xsize.c \ osdep.c \ os2compat.c \ intl-exports.c \ @@ -190,6 +191,7 @@ OBJECTS = \ printf.$lo \ setlocale.$lo \ version.$lo \ + xsize.$lo \ osdep.$lo \ intl-compat.$lo OBJECTS_RES_yes = libintl.res.$lo @@ -298,6 +300,8 @@ setlocale.lo: $(srcdir)/setlocale.c $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC --mode=compile $(COMPILE) $(srcdir)/setlocale.c version.lo: $(srcdir)/version.c $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC --mode=compile $(COMPILE) $(srcdir)/version.c +xsize.lo: $(srcdir)/xsize.c + $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC --mode=compile $(COMPILE) $(srcdir)/xsize.c osdep.lo: $(srcdir)/osdep.c $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC --mode=compile $(COMPILE) $(srcdir)/osdep.c intl-compat.lo: $(srcdir)/intl-compat.c @@ -585,7 +589,7 @@ dcigettext.$lo: $(srcdir)/eval-plural.h localcharset.$lo: $(srcdir)/localcharset.h bindtextdom.$lo dcigettext.$lo finddomain.$lo loadmsgcat.$lo localealias.$lo lock.$lo log.$lo: $(srcdir)/lock.h localealias.$lo localcharset.$lo relocatable.$lo: $(srcdir)/relocatable.h -printf.$lo: $(srcdir)/printf-args.h $(srcdir)/printf-args.c $(srcdir)/printf-parse.h $(srcdir)/wprintf-parse.h $(srcdir)/xsize.h $(srcdir)/printf-parse.c $(srcdir)/vasnprintf.h $(srcdir)/vasnwprintf.h $(srcdir)/vasnprintf.c +printf.$lo: $(srcdir)/printf-args.h $(srcdir)/printf-args.c $(srcdir)/printf-parse.h $(srcdir)/wprintf-parse.h $(srcdir)/xsize.h $(srcdir)/xsize.c $(srcdir)/printf-parse.c $(srcdir)/vasnprintf.h $(srcdir)/vasnwprintf.h $(srcdir)/vasnprintf.c # A bison-2.1 generated plural.c includes if ENABLE_NLS. PLURAL_DEPS_yes = libintl.h diff --git a/gettext-runtime/intl/xsize.c b/gettext-runtime/intl/xsize.c new file mode 100644 index 000..4b4914c --- /dev/null +++ b/gettext-runtime/intl/xsize.c @@ -0,0 +1,3 @@ +#include +#define XSIZE_INLINE _GL_EXTERN_INLINE +#include "xsize.h" diff --git a/gettext-runtime/libasprintf/Makefile.am b/gettext-runtime/libasprintf/Makefile.am index 2aebbed..343ae4b 100644 --- a/gettext-runtime/libasprintf/Makefile.am +++ b/gettext-runtime/libasprintf/Makefile.am @@ -49,13
Re: [bug-gettext] gettext 0.18.2 fails to compile without optimisations
Paul Eggert writes: > On 01/16/13 01:08, Daiki Ueno wrote: >> I'm not 100% sure if this is the right fix, so Cc'ed bug-gnulib. > > This patch looks good to me. Might there be other files > with the same problem? Maybe gettext should be changed to > use gnulib-tool to import modules from gnulib, as that would > help avoid similar issues in the future. Thanks. I've modified the patch to use gnulib-tool for libasprintf, and pushed it to the master. For libintl, perhaps we may want to consider doing the auto-sync between gettext <-> gnulib. Regards, -- Daiki Ueno
sigaction test failure on FreeBSD 9.1 when pthread is used
Hi, On FreeBSD 9.1, I got an assertion failure in test-sigaction.c when the test case is compiled with: $ gnulib-tool --create-testdir --dir=test-sa-openmp sigaction openmp $ cd test-sa-openmp $ ./configure --enable-openmp $ make $ make check ... test-sigaction.c:69: assertion failed Abort trap (core dumped) FAIL: test-sigaction It seems -fopenmp flag causes the error. It can also be reproduced without openmp, but with CFLAGS="-O2 -pthread". Regards, -- Daiki Ueno
Re: sigaction test failure on FreeBSD 9.1 when pthread is used
Paul Eggert writes:
> On 01/17/13 23:04, Daiki Ueno wrote:
>> It seems -fopenmp flag causes the error. It can also be reproduced
>> without openmp, but with CFLAGS="-O2 -pthread".
>
> I can't reproduce it with Fedora 17, but I guess that's not
> too surprising.
Yes.
> Can you send the output of this shell command?
>
> truss ./test-sigaction
Here it is:
mmap(0x0,32768,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 34366128128
(0x800618000)
issetugid(0x800817f80,0x7fffefa4,0x40,0x0,0x800800818fdc,0x0) = 0 (0x0)
lstat("/etc",{ mode=drwxr-xr-x ,inode=481536,size=2048,blksize=32768 }) = 0
(0x0)
lstat("/etc/libmap.conf",0x7fffc610) ERR#2 'No such file or
directory'
open("/var/run/ld-elf.so.hints",O_RDONLY,057)= 3 (0x3)
read(3,"Ehnt\^A\0\0\0\M^@\0\0\0-\0\0\0\0"...,128) = 128 (0x80)
lseek(3,0x80,SEEK_SET) = 128 (0x80)
read(3,"/lib:/usr/lib:/usr/lib/compat:/u"...,45) = 45 (0x2d)
close(3) = 0 (0x0)
access("/lib/libgomp.so.1",0)ERR#2 'No such file or
directory'
access("/usr/lib/libgomp.so.1",0)= 0 (0x0)
open("/usr/lib/libgomp.so.1",O_RDONLY,040305300) = 3 (0x3)
fstat(3,{ mode=-r--r--r-- ,inode=1693510,size=31248,blksize=32768 }) = 0 (0x0)
mmap(0x0,4096,PROT_READ,MAP_PRIVATE|0x4,3,0x0) = 34366160896 (0x80062)
mmap(0x0,2125824,PROT_NONE,MAP_PRIVATE|MAP_ANON|MAP_NOCORE,-1,0x0) =
34368229376 (0x800819000)
mmap(0x800819000,28672,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|0x4,3,0x0)
= 34368229376 (0x800819000)
mmap(0x800a1f000,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|0x4,3,0x6000)
= 34370351104 (0x800a1f000)
munmap(0x80062,4096) = 0 (0x0)
close(3) = 0 (0x0)
access("/lib/libthr.so.3",0) = 0 (0x0)
open("/lib/libthr.so.3",O_RDONLY,040305300) = 3 (0x3)
fstat(3,{ mode=-r--r--r-- ,inode=1364383,size=101048,blksize=32768 }) = 0 (0x0)
mmap(0x0,4096,PROT_READ,MAP_PRIVATE|0x4,3,0x0) = 34366160896 (0x80062)
mmap(0x0,2236416,PROT_NONE,MAP_PRIVATE|MAP_ANON|MAP_NOCORE,-1,0x0) =
34370355200 (0x800a2)
mmap(0x800a2,94208,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|0x4,3,0x0)
= 34370355200 (0x800a2)
mmap(0x800c36000,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|0x4,3,0x16000)
= 34372542464 (0x800c36000)
mmap(0x800c38000,40960,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_ANON,-1,0x0)
= 34372550656 (0x800c38000)
munmap(0x80062,4096) = 0 (0x0)
close(3) = 0 (0x0)
access("/lib/libc.so.7",0) = 0 (0x0)
open("/lib/libc.so.7",O_RDONLY,040305300)= 3 (0x3)
fstat(3,{ mode=-r--r--r-- ,inode=1364354,size=1369520,blksize=32768 }) = 0 (0x0)
mmap(0x0,4096,PROT_READ,MAP_PRIVATE|0x4,3,0x0) = 34366160896 (0x80062)
mmap(0x0,3485696,PROT_NONE,MAP_PRIVATE|MAP_ANON|MAP_NOCORE,-1,0x0) =
34372591616 (0x800c42000)
mmap(0x800c42000,1236992,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|0x4,3,0x0)
= 34372591616 (0x800c42000)
mmap(0x800f6f000,45056,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|0x4,3,0x12d000)
= 34375921664 (0x800f6f000)
mmap(0x800f7a000,110592,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_ANON,-1,0x0)
= 34375966720 (0x800f7a000)
munmap(0x80062,4096) = 0 (0x0)
close(3) = 0 (0x0)
munmap(0x80061f000,4096) = 0 (0x0)
mmap(0x0,40960,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 34366156800
(0x80061f000)
munmap(0x800622000,28672)= 0 (0x0)
mmap(0x0,102400,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 34366169088
(0x800622000)
sysarch(0x81,0x7fffd380,0x80061d1c8,0x0,0xff6c4e70,0x8080808080808080)
= 0 (0x0)
sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2,0x0)
= 0 (0x0)
sigprocmask(SIG_SETMASK,0x0,0x0) = 0 (0x0)
sigprocmask(SIG_BLOCK,SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2,0x0)
= 0 (0x0)
sigprocmask(SIG_SETMASK,0x0,0x0) = 0 (0x0)
getpid() = 849 (0x351)
__sysctl(0x7fffd2e0,0x2,0x800c41f80,0x7fffd2e8,0x0,0x0) = 0 (0x0)
__sysctl(0x7fffd210,0x2,0x7fffd240,0x7fffd2a8,0x800a33a30,0xd) = 0
(0x0)
__sysctl(0x7fffd240,0x3,0x800c40e68,0x7fffd2e8,0x0,0x0) = 0 (0x0)
readlink("/etc/malloc.conf",0x7ff
Re: sigaction test failure on FreeBSD 9.1 when pthread is used
Sorry for the delay. Paul Eggert writes: > Always reporting SA_SIGINFO violates POSIX but I doubt whether > anybody really cares, so does the following patch fix things for you? > It simply adjusts the test so that it doesn't check for this bug. Yes, but from the gettext Hydra build log[1], FreeBSD 8.x does not seem to have that behavior. Perhaps it might be a bug in the latest FreeBSD kernel. Footnotes: [1] http://hydra.nixos.org/build/3910833/nixlog/1 Regards, -- Daiki Ueno
[PATCH] wctype-h: fix rpl_towupper/rpl_towlower link error on mingw
See <https://lists.gnu.org/archive/html/bug-gettext/2013-03/msg00086.html>. With the recent inline change, rpl_towlower/rpl_towupper are defined as C99 extern inline. If there are declarations of those functions, GCC reports "multiple definition" error. On mingw (not mingw-w64, but http://mingw.org), towlower/towuppwer are also declared in as well as , as follows: /* Older MS docs uses wchar_t for arg and return type, while newer online MS docs say arg is wint_t and return is int. ISO C uses wint_t for both. */ _CRTIMP wint_t __cdecl __MINGW_NOTHROW towlower (wint_t); _CRTIMP wint_t __cdecl __MINGW_NOTHROW towupper (wint_t); When is included after Gnulib's , they will be expanded to: _CRTIMP wint_t __cdecl __MINGW_NOTHROW rpl_towlower (wint_t); _CRTIMP wint_t __cdecl __MINGW_NOTHROW rpl_towupper (wint_t); and causes "multiple definition" error. I'm attaching a patch. Regards, -- Daiki Ueno >From e23bbce242682524d7b52a8637c92482d9a8c2fd Mon Sep 17 00:00:00 2001 From: Daiki Ueno Date: Wed, 27 Mar 2013 17:16:46 +0900 Subject: [PATCH] wctype-h: fix gettext link error on mingw Reported by Josue Andrade Gomes and Takayuki Tsunakawa in <https://lists.gnu.org/archive/html/bug-gettext/2013-03/msg00086.html>. * lib/wctype.in.h [__MINGW32__]: Include before defining rpl_towupper and rpl_towupper. --- ChangeLog | 8 lib/wctype.in.h | 7 +++ 2 files changed, 15 insertions(+) diff --git a/ChangeLog b/ChangeLog index 3843c33..d8b1ac1 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,11 @@ +2013-03-27 Daiki Ueno + + wctype-h: fix gettext link error on mingw + Reported by Josue Andrade Gomes and Takayuki Tsunakawa in + <https://lists.gnu.org/archive/html/bug-gettext/2013-03/msg00086.html>. + * lib/wctype.in.h [__MINGW32__]: Include before defining + rpl_towupper and rpl_towupper. + 2013-03-21 Paul Eggert sys_select, sys_time: port 2013-01-30 Solaris 2.6 fix to Cygwin diff --git a/lib/wctype.in.h b/lib/wctype.in.h index a465af2..660f23d 100644 --- a/lib/wctype.in.h +++ b/lib/wctype.in.h @@ -44,6 +44,13 @@ # include #endif +/* mingw has declarations of towupper and towlower in as + well . Include in advance to avoid rpl_ prefix + being added to the declarations. */ +#if __MINGW32__ +# include +#endif + /* Include the original if it exists. BeOS 5 has the functions but no . */ /* The include_next requires a split double-inclusion guard. */ -- 1.8.1.4
Re: sigaction test failure on FreeBSD 9.1 when pthread is used
Ed Maste writes: > Tested on 8.4-PRERELEASE, 9.1-STABLE (the 9.x branch after 9.1 > released), and 10.0-CURRENT, all returned 0. Perhaps Daiki Ueno can > test in the environment that originally demonstrated the problem. I can still reproduce this on 9.1-RELEASE. Perhaps you are missing -pthread cflag when compiling? Regards, -- Daiki Ueno
Re: [PATCH] qacl: new module, broken out from the acl module
Hi, After the change, 'acl' module fails to build: $ gnulib-tool --create-testdir --dir=test-acl acl $ cd test-acl $ ./configure && make ... copy-acl.c: In function 'copy_acl': copy-acl.c:47:17: error: 'errno' undeclared (first use in this function) copy-acl.c:47:17: note: each undeclared identifier is reported only once for each function it appears in Paul Eggert writes: > * lib/copy-acl.c, lib/set-acl.c: Include errno,h, not acl-internal.h. [...] > diff --git a/lib/copy-acl.c b/lib/copy-acl.c > index 58ff54a..bcc86b6 100644 > --- a/lib/copy-acl.c > +++ b/lib/copy-acl.c > @@ -21,8 +21,7 @@ > > #include "acl.h" > > -#include "acl-internal.h" > - > +#include "error.h" Seems #include is also needed (same for set-acl.c). Regards, -- Daiki Ueno
Re: bug#14253: Testsuite failure in gnulib-tests: test programs cannot be built
Paul Eggert writes: >> Ping on this? I'm still have to create a dummy 'intl' directory myself >> if I want to be able to bootstrap coreutils from the git repository ... > > Since it's causing you problems I installed the patch into gnulib. > We can always change it later if Daiki or Bruno prefers that. FWIW, the change looks reasonable to me. Given that modules/gettext adds AM_GNU_GETTEXT([external]) to configure.ac, the AM_CPPFLAGS setting seems to be nothing more than example. Regards, -- Daiki Ueno
[PATCH] lock: work around pthread recursive mutexes bug in Mac OS X 10.6
Hi, We've got a couple of bug reports saying that gettext's libintl doesn't work when it was compiled on Mac OS X >= 10.7 targetting 10.6, because of missing PTHREAD_RECURSIVE_MUTEX_INITIALIZER definition: <http://savannah.gnu.org/bugs/?37844> <http://lists.gnu.org/archive/html/bug-gettext/2013-05/msg7.html> According to the report, other projects seem to have added a work around for this, perhaps good to add the same check to Gnulib. Patch attached. It compiles fine on 10.8, but not actually tested on 10.6. It would be nice if anyone could test the compiled binary on 10.6. Regards, -- Daiki Ueno >From 19685cf6de36e6f09423edc059d7ba24c8b4e45a Mon Sep 17 00:00:00 2001 From: Daiki Ueno Date: Sat, 11 May 2013 10:40:55 +0900 Subject: [PATCH] lock: work around pthread recursive mutexes bug in Mac OS X 10.6 * m4/lock.m4: Don't define HAVE_PTHREAD_MUTEX_RECURSIVE if the compilation target is Mac OS X 10.6. Problem reported by parafin and Andoni Morales in <http://savannah.gnu.org/bugs/?37844> and <http://lists.gnu.org/archive/html/bug-gettext/2013-05/msg7.html>. --- ChangeLog |9 + m4/lock.m4 |2 ++ 2 files changed, 11 insertions(+) diff --git a/ChangeLog b/ChangeLog index 13dafaa..b65057e 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,12 @@ +2013-05-11 Daiki Ueno + + lock: work around pthread mutexes bug in Mac OS X 10.6 + * m4/lock.m4: Don't define HAVE_PTHREAD_MUTEX_RECURSIVE if the + compilation target is Mac OS X 10.6. + Problem reported by parafin and Andoni Morales in + <http://savannah.gnu.org/bugs/?37844> and + <http://lists.gnu.org/archive/html/bug-gettext/2013-05/msg7.html>. + 2012-05-10 Stefano Lattarini deps: require Automake >= 1.9.6 in generated Makefile fragments diff --git a/m4/lock.m4 b/m4/lock.m4 index d3fc1ef..78a72ef 100644 --- a/m4/lock.m4 +++ b/m4/lock.m4 @@ -24,6 +24,8 @@ AC_DEFUN([gl_LOCK], [[ #if __FreeBSD__ == 4 error "No, in FreeBSD 4.0 recursive mutexes actually don't work." +#elif defined __APPLE__ && MAC_OS_X_VERSION_MIN_REQUIRED == 1060 +error "No, in Mac OS X 10.6 recursive mutexes actually don't work." #else int x = (int)PTHREAD_MUTEX_RECURSIVE; return !x; -- 1.7.10.4
Re: [PATCH] lock: work around pthread recursive mutexes bug in Mac OS X 10.6
Daiki Ueno writes: > Patch attached. Sorry, wrong patch. Please check the attached one instead. I've confirmed that HAVE_PTHREAD_MUTEX_RECURSIVE is defined by default on 10.8, but not with CFLAGS=-mmacosx-version-min=10.6. Regards, -- Daiki Ueno >From eb742ec35bfd68288bfc23aae47e3d27dcaff4de Mon Sep 17 00:00:00 2001 From: Daiki Ueno Date: Sat, 11 May 2013 10:40:55 +0900 Subject: [PATCH] lock: work around pthread recursive mutexes bug in Mac OS X 10.6 * m4/lock.m4: Don't define HAVE_PTHREAD_MUTEX_RECURSIVE if the compilation target is Mac OS X 10.6. Problem reported by parafin and Andoni Morales in <http://savannah.gnu.org/bugs/?37844> and <http://lists.gnu.org/archive/html/bug-gettext/2013-05/msg7.html>. --- ChangeLog |9 + m4/lock.m4 |2 ++ 2 files changed, 11 insertions(+) diff --git a/ChangeLog b/ChangeLog index 13dafaa..2590e4a 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,12 @@ +2013-05-11 Daiki Ueno + + lock: work around pthread recursive mutexes bug in Mac OS X 10.6 + * m4/lock.m4: Don't define HAVE_PTHREAD_MUTEX_RECURSIVE if the + compilation target is Mac OS X 10.6. + Problem reported by parafin and Andoni Morales in + <http://savannah.gnu.org/bugs/?37844> and + <http://lists.gnu.org/archive/html/bug-gettext/2013-05/msg7.html>. + 2012-05-10 Stefano Lattarini deps: require Automake >= 1.9.6 in generated Makefile fragments diff --git a/m4/lock.m4 b/m4/lock.m4 index d3fc1ef..e7aedec 100644 --- a/m4/lock.m4 +++ b/m4/lock.m4 @@ -24,6 +24,8 @@ AC_DEFUN([gl_LOCK], [[ #if __FreeBSD__ == 4 error "No, in FreeBSD 4.0 recursive mutexes actually don't work." +#elif defined __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ && __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ < 1070 +error "No, in Mac OS X < 10.7 recursive mutexes actually don't work." #else int x = (int)PTHREAD_MUTEX_RECURSIVE; return !x; -- 1.7.10.4
Re: [PATCH] tests: don't assume getdtablesize () <= 10000000
Hi,
Paul Eggert writes:
> + * tests/test-posix_spawn_file_actions_addclose.c:
> + * tests/test-posix_spawn_file_actions_adddup2.c:
> + * tests/test-posix_spawn_file_actions_addopen.c:
> + Don't assume getdtablesize () <= 1000.
Those changes are failing on Mac OS X:
FAIL: test-posix_spawn_file_actions_addclose
test-posix_spawn_file_actions_addclose.c:45: assertion failed
FAIL: test-posix_spawn_file_actions_adddup2
===
test-posix_spawn_file_actions_adddup2.c:45: assertion failed
FAIL: test-posix_spawn_file_actions_addopen
===
test-posix_spawn_file_actions_addopen.c:50: assertion failed
> -ASSERT (posix_spawn_file_actions_addopen (&actions, 1000,
> +ASSERT (posix_spawn_file_actions_addopen (&actions, getdtablesize (),
>"foo", 0, O_RDONLY)
> == EBADF);
It seems Mac OS X's posix_spawn_file_actions_add{open,dup2,close} do not
check the runtime limits. Here:
* getdtablesize () == 256
* sysconfig (_SC_OPEN_MAX) == 256
* OPEN_MAX == 10240
If I replace getdtablesize () with OPEN_MAX in the above call, the test
passes, while it still fails with OPEN_MAX - 1.
Regards,
--
Daiki Ueno
Re: [PATCH] tests: don't assume getdtablesize () <= 10000000
Paul Eggert writes: > Thanks, I pushed the following to try to fix that. > Does it work for you now? Yes, all those 3 tests pass now. Thanks for the quick fix. Regards, -- Daiki Ueno
[PATCH] test-open: avoid compilation error with -D_FORTIFY_SOURCE=2
I got the following error when compiling test-open.c with
-D_FORTIFY_SOURCE=2.
$ gcc -DHAVE_CONFIG_H -I. -DGNULIB_STRICT_CHECKING=1 -I. -I. -I.. -I./..
-I../gllib -I./../gllib -Wall -O1 -D_FORTIFY_SOURCE=2 -g -MT test-open.o -MD
-MP -MF .deps/test-open.Tpo -c -o test-open.o test-open.c
In file included from /usr/include/fcntl.h:296:0,
from ../gllib/fcntl.h:61,
from test-open.c:21:
test-open.h: In function 'main':
/usr/include/bits/fcntl2.h:41:1: error: inlining failed in call to
always_inline 'open': indirect function call with a yet undetermined callee
open (const char *__path, int __oflag, ...)
^
It seems that open cannot be directly passed to test_open as a function
pointer, since it is marked as always_inline. I'm attaching a patch.
Regards,
--
Daiki Ueno
>From f0ab4954d777a041c88964ce667a9b01e858b395 Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Fri, 21 Jun 2013 12:37:20 +0900
Subject: [PATCH] test-open: avoid compilation error with -D_FORTIFY_SOURCE=2
* tests/test-open.c (do_open): New wrapper around the system open.
---
ChangeLog | 5 +
tests/test-open.c | 24 +++-
2 files changed, 28 insertions(+), 1 deletion(-)
diff --git a/ChangeLog b/ChangeLog
index 5e4f02f..90140fd 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,8 @@
+2013-06-21 Daiki Ueno
+
+ test-open: avoid compilation error with -D_FORTIFY_SOURCE=2
+ * tests/test-open.c (do_open): New wrapper around the system open.
+
2013-06-18 Paul Eggert
doc: document extern-inline
diff --git a/tests/test-open.c b/tests/test-open.c
index f04144f..c21b21c 100644
--- a/tests/test-open.c
+++ b/tests/test-open.c
@@ -34,8 +34,30 @@ SIGNATURE_CHECK (open, int, (char const *, int, ...));
#include "test-open.h"
+/* Wrapper around open to test it with test_open. When
+ -D_FORTIFY_SOURCE=2, open may be inlined and cannot be passed as a
+ function pointer. */
+static int
+do_open (char const *name, int flags, ...)
+{
+ if (flags & O_CREAT)
+{
+ mode_t mode = 0;
+ va_list arg;
+ va_start (arg, flags);
+
+ /* We have to use PROMOTED_MODE_T instead of mode_t, otherwise GCC 4
+ creates crashing code when 'mode_t' is smaller than 'int'. */
+ mode = va_arg (arg, PROMOTED_MODE_T);
+
+ va_end (arg);
+ return open (name, flags, mode);
+}
+ return open (name, flags);
+}
+
int
main (void)
{
- return test_open (open, true);
+ return test_open (do_open, true);
}
--
1.8.2.1
update from gettext 0.18.3
Bruno Haible writes: > The mechanism is to update the .m4 files from gettext to gnulib once, shortly > after a gettext release. Which I'm doing now. Thanks. So, here is the update from 0.18.3 released yesterday. >From c868fda8ac1d5e8be64c3a157c1a8c540e02b3e8 Mon Sep 17 00:00:00 2001 From: Daiki Ueno Date: Mon, 8 Jul 2013 09:39:05 +0900 Subject: [PATCH] gettext: Update to version 0.18.3. * m4/intl.m4, m4/po.m4: Update from gettext-0.18.3. In particular, require AC_PROG_SED to allow user to specify custom sed command when generating en@quot PO file. --- ChangeLog | 7 +++ m4/intl.m4 | 37 - m4/po.m4 | 3 ++- 3 files changed, 13 insertions(+), 34 deletions(-) diff --git a/ChangeLog b/ChangeLog index 7698099..f54c773 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,10 @@ +2013-07-08 Daiki Ueno + +gettext: Update to version 0.18.3. +* m4/intl.m4, m4/po.m4: Update from gettext-0.18.3. In particular, +require AC_PROG_SED to allow user to specify custom sed command when +generating en@quot PO file. + 2013-07-06 Paul Eggert fnmatch: don't goto over declaration diff --git a/m4/intl.m4 b/m4/intl.m4 index 486b5cc..959bd04 100644 --- a/m4/intl.m4 +++ b/m4/intl.m4 @@ -1,4 +1,4 @@ -# intl.m4 serial 22 (gettext-0.18.2) +# intl.m4 serial 23 (gettext-0.18.3) dnl Copyright (C) 1995-2013 Free Software Foundation, Inc. dnl This file is free software; the Free Software Foundation dnl gives unlimited permission to copy and/or distribute it, @@ -61,15 +61,13 @@ AC_DEFUN([AM_INTL_SUBDIR], dnl Use the _snprintf function only if it is declared (because on NetBSD it dnl is defined as a weak alias of snprintf; we prefer to use the latter). - gt_CHECK_DECL(_snprintf, [#include ]) - gt_CHECK_DECL(_snwprintf, [#include ]) + AC_CHECK_DECLS([_snprintf _snwprintf], , , [#include ]) dnl Use the *_unlocked functions only if they are declared. dnl (because some of them were defined without being declared in Solaris dnl 2.5.1 but were removed in Solaris 2.6, whereas we want binaries built dnl on Solaris 2.5.1 to run on Solaris 2.6). - dnl Don't use AC_CHECK_DECLS because it isn't supported in autoconf-2.13. - gt_CHECK_DECL(getc_unlocked, [#include ]) + AC_CHECK_DECLS([getc_unlocked], , , [#include ]) case $gt_cv_func_printf_posix in *yes) HAVE_POSIX_PRINTF=1 ;; @@ -236,9 +234,7 @@ AC_DEFUN([gt_INTL_SUBDIR_CORE], dnl (because some of them were defined without being declared in Solaris dnl 2.5.1 but were removed in Solaris 2.6, whereas we want binaries built dnl on Solaris 2.5.1 to run on Solaris 2.6). - dnl Don't use AC_CHECK_DECLS because it isn't supported in autoconf-2.13. - gt_CHECK_DECL([feof_unlocked], [#include ]) - gt_CHECK_DECL([fgets_unlocked], [#include ]) + AC_CHECK_DECLS([feof_unlocked fgets_unlocked], , , [#include ]) AM_ICONV @@ -273,28 +269,3 @@ changequote([,])dnl INTLBISON=: fi ]) - - -dnl gt_CHECK_DECL(FUNC, INCLUDES) -dnl Check whether a function is declared. -AC_DEFUN([gt_CHECK_DECL], -[ - AC_CACHE_CHECK([whether $1 is declared], [ac_cv_have_decl_$1], -[AC_COMPILE_IFELSE( - [AC_LANG_PROGRAM( - [[$2]], - [[ -#ifndef $1 - char *p = (char *) $1; -#endif - ]])], - [ac_cv_have_decl_$1=yes], - [ac_cv_have_decl_$1=no])]) - if test $ac_cv_have_decl_$1 = yes; then -gt_value=1 - else -gt_value=0 - fi - AC_DEFINE_UNQUOTED([HAVE_DECL_]m4_translit($1, [a-z], [A-Z]), [$gt_value], -[Define to 1 if you have the declaration of '$1', and to 0 if you don't.]) -]) diff --git a/m4/po.m4 b/m4/po.m4 index f395723..1c70b6c 100644 --- a/m4/po.m4 +++ b/m4/po.m4 @@ -1,4 +1,4 @@ -# po.m4 serial 20 (gettext-0.18.2) +# po.m4 serial 21 (gettext-0.18.3) dnl Copyright (C) 1995-2013 Free Software Foundation, Inc. dnl This file is free software; the Free Software Foundation dnl gives unlimited permission to copy and/or distribute it, @@ -25,6 +25,7 @@ AC_DEFUN([AM_PO_SUBDIRS], AC_REQUIRE([AC_PROG_MAKE_SET])dnl AC_REQUIRE([AC_PROG_INSTALL])dnl AC_REQUIRE([AC_PROG_MKDIR_P])dnl + AC_REQUIRE([AC_PROG_SED])dnl AC_REQUIRE([AM_NLS])dnl dnl Release version of the gettext macros. This is used to ensure that -- 1.8.3.2 Regards, -- Daiki Ueno
[PATCH] announce-gen: avoid failure when Digest::SHA is installed
Hi,
The 'announce-gen' script fails when Digest::SHA (not Digest::SHA1) is
installed:
Can't locate object method "new" via package "Digest::SHA1" (perhaps you
forgot to load "Digest::SHA1"?) at ../gnulib/build-aux/announce-gen line 165.
zsh: exit 25./gnulib/build-aux/announce-gen --bootstrap-tools=automake
This is because of the following code for import:
eval { require Digest::SHA; }
or eval 'use Digest::SHA1';
and the usage in print_checksums:
my $dig =
($meth eq 'md5'
? Digest::MD5->new->addfile(*IN)->hexdigest
: Digest::SHA1->new->addfile(*IN)->hexdigest);
Here, Digest::SHA1 package may be undefined. I'm attaching a patch.
As I'm new to Perl, comments (and a better fix) would be appreciated :)
>From 8726f48a1a263b528040f19a3323e2c558dba492 Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Tue, 9 Jul 2013 15:54:48 +0900
Subject: [PATCH] announce-gen: avoid failure when Digest::SHA is installed
When Digest::SHA is available, Digest::SHA1 is not loaded and thus
Digest::SHA1->new in print_checksums fails.
* build-aux/announce-gen (digest_classes): New associative array
for available message digest implementations.
(print_locations): Use it.
---
ChangeLog | 9 +
build-aux/announce-gen | 17 +
2 files changed, 18 insertions(+), 8 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 4247ec8..05644ae 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,12 @@
+2013-07-09 Daiki Ueno
+
+ announce-gen: avoid failure when Digest::SHA is installed
+ When Digest::SHA is available, Digest::SHA1 is not loaded and thus
+ Digest::SHA1->new in print_checksums fails.
+ * build-aux/announce-gen (digest_classes): New associative array
+ for available message digest implementations.
+ (print_locations): Use it.
+
2013-07-07 Paul Eggert
stdalign, verify: port to FreeBSD 9.1, to C11, and to C++11
diff --git a/build-aux/announce-gen b/build-aux/announce-gen
index 3a64ec6..21afd0b 100755
--- a/build-aux/announce-gen
+++ b/build-aux/announce-gen
@@ -3,7 +3,7 @@ eval '(exit $?0)' && eval 'exec perl -wS "$0" ${1+"$@"}'
if 0;
# Generate a release announcement message.
-my $VERSION = '2012-06-08 06:53'; # UTC
+my $VERSION = '2013-07-09 06:39'; # UTC
# The definition above must lie within the first 8 lines in order
# for the Emacs time-stamp write hook (at end) to update it.
# If you change this file with Emacs, please let the write hook
@@ -29,15 +29,18 @@ my $VERSION = '2012-06-08 06:53'; # UTC
use strict;
use Getopt::Long;
-use Digest::MD5;
-eval { require Digest::SHA; }
- or eval 'use Digest::SHA1';
use POSIX qw(strftime);
(my $ME = $0) =~ s|.*/||;
my %valid_release_types = map {$_ => 1} qw (alpha beta stable);
my @archive_suffixes = ('tar.gz', 'tar.bz2', 'tar.lzma', 'tar.xz');
+my %digest_classes =
+ (
+ 'md5' => (eval { require Digest::MD5; } and 'Digest::MD5'),
+ 'sha1' => ((eval { require Digest::SHA; } and 'Digest::SHA')
+ or (eval { require Digest::SHA1; } and 'Digest::SHA1'))
+ );
my $srcdir = '.';
sub usage ($)
@@ -157,15 +160,13 @@ sub print_checksums (@)
foreach my $meth (qw (md5 sha1))
{
+ my $class = $digest_classes{$meth} or next;
foreach my $f (@file)
{
open IN, '<', $f
or die "$ME: $f: cannot open for reading: $!\n";
binmode IN;
- my $dig =
-($meth eq 'md5'
- ? Digest::MD5->new->addfile(*IN)->hexdigest
- : Digest::SHA1->new->addfile(*IN)->hexdigest);
+ my $dig = $class->new->addfile(*IN)->hexdigest;
close IN;
print "$dig $f\n";
}
--
1.8.3.1
Regards,
--
Daiki Ueno
Re: update from gettext 0.18.3
Paul Eggert writes: > On 07/19/2013 03:52 PM, Karl Berry wrote: >> Can some gnulib-ber with more git-fu than me (which is all of you, I'm >> sure) install it, please? > > Done, as commit a7f78b10889cd4bad2a56f110cf4177446eef3bb. Thanks. By the way, speaking of gettext-related old patch, any chance of this getting committed? 2012-12-20 Daiki Ueno localecharset: make locale_charset thread-safe on Mac OS X * lib/localcharset.c (locale_charset) [DARWIN7]: Use MB_CUR_MAX_L instead of MB_CUR_MAX. http://lists.gnu.org/archive/html/bug-gnulib/2012-12/msg00091.html We've shipped gettext 0.18.2 and 0.18.3 with this as a local patch, since otherwise there would have been a regression (the patch was not necessary with the gnulib checkout used to bootstrap 0.18.1). No issues regarding this change have been reported so far. Regards, -- Daiki Ueno
[PATCH] gettext: update to version 0.18.3.1
* m4/intl.m4: Update from gettext-0.18.3.1, which fixes a misuse of AC_CHECK_DECLS. --- ChangeLog | 5 + m4/intl.m4 | 6 +++--- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/ChangeLog b/ChangeLog index 7894c2d..994c6d7 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,8 @@ +2013-08-24 Daiki Ueno + + * m4/intl.m4: Update from gettext-0.18.3.1, which fixes a misuse + of AC_CHECK_DECLS. + 2013-08-23 Paul Eggert selinux-at: omit unnecessary include diff --git a/m4/intl.m4 b/m4/intl.m4 index 959bd04..dffcd88 100644 --- a/m4/intl.m4 +++ b/m4/intl.m4 @@ -1,4 +1,4 @@ -# intl.m4 serial 23 (gettext-0.18.3) +# intl.m4 serial 24 (gettext-0.18.3) dnl Copyright (C) 1995-2013 Free Software Foundation, Inc. dnl This file is free software; the Free Software Foundation dnl gives unlimited permission to copy and/or distribute it, @@ -61,7 +61,7 @@ AC_DEFUN([AM_INTL_SUBDIR], dnl Use the _snprintf function only if it is declared (because on NetBSD it dnl is defined as a weak alias of snprintf; we prefer to use the latter). - AC_CHECK_DECLS([_snprintf _snwprintf], , , [#include ]) + AC_CHECK_DECLS([_snprintf, _snwprintf], , , [#include ]) dnl Use the *_unlocked functions only if they are declared. dnl (because some of them were defined without being declared in Solaris @@ -234,7 +234,7 @@ AC_DEFUN([gt_INTL_SUBDIR_CORE], dnl (because some of them were defined without being declared in Solaris dnl 2.5.1 but were removed in Solaris 2.6, whereas we want binaries built dnl on Solaris 2.5.1 to run on Solaris 2.6). - AC_CHECK_DECLS([feof_unlocked fgets_unlocked], , , [#include ]) + AC_CHECK_DECLS([feof_unlocked, fgets_unlocked], , , [#include ]) AM_ICONV -- 1.8.3.1
Re: portability of fopen and 'e' (O_CLOEXEC) flag
Hello Bruno,
Bruno Haible writes:
> Additionally, gnulib is the right place to do this because - as Eric said -
> the 'e' flag is already scheduled for being added to the next POSIX version:
> https://www.austingroupbugs.net/view.php?id=411
>
> My evaluation of current platform support was incorrect: Current versions of
> FreeBSD, NetBSD, OpenBSD, Solaris, Cygwin, and even Minix already support it.
Thank you for this; would it make sense to use it in the modules that do
one-shot fopen/fclose, such as read-file?
>From 5de739d03a7da71127b771cc213c873cd711ce51 Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Tue, 26 May 2020 07:56:13 +0200
Subject: [PATCH] read-file: make use of fopen-gnu
* lib/read-file.c (read_file): Pass an 'e' flag to fopen.
(read_binary_file): Likewise.
* modules/read-file (Depends-on): Add fopen-gnu.
---
ChangeLog | 7 +++
lib/read-file.c | 4 ++--
modules/read-file | 1 +
3 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index b1acb99ca..07d4d5124 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2020-05-26 Daiki Ueno
+
+ read-file: make use of fopen-gnu
+ * lib/read-file.c (read_file): Pass an 'e' flag to fopen.
+ (read_binary_file): Likewise.
+ * modules/read-file (Depends-on): Add fopen-gnu.
+
2020-05-25 Paul Eggert
getentropy, getrandom: new modules
diff --git a/lib/read-file.c b/lib/read-file.c
index c6f230178..293bc3e8a 100644
--- a/lib/read-file.c
+++ b/lib/read-file.c
@@ -171,7 +171,7 @@ internal_read_file (const char *filename, size_t *length, const char *mode)
char *
read_file (const char *filename, size_t *length)
{
- return internal_read_file (filename, length, "r");
+ return internal_read_file (filename, length, "re");
}
/* Open (on non-POSIX systems, in binary mode) and read the contents
@@ -184,5 +184,5 @@ read_file (const char *filename, size_t *length)
char *
read_binary_file (const char *filename, size_t *length)
{
- return internal_read_file (filename, length, "rb");
+ return internal_read_file (filename, length, "rbe");
}
diff --git a/modules/read-file b/modules/read-file
index 506e88f0a..a6e7faf0a 100644
--- a/modules/read-file
+++ b/modules/read-file
@@ -7,6 +7,7 @@ lib/read-file.c
m4/read-file.m4
Depends-on:
+fopen-gnu
fstat
ftello
malloc-posix
--
2.26.2
Regards,
--
Daiki Ueno
[PATCH] read-file: add variants that clear internal memory
Hello,
The functions provided by the read-file module are handy, but they are
suboptimal for reading sensitive materials, because they do not clear
the allocated memory blocks upon failure. The attached patch adds
a set of variants that deal with that.
It's tempting to make this behavior enabled by default, but I worry that
it may cause any performance drawback.
Regards,
--
Daiki Ueno
>From d2fc964fa1dd87f5970f28c22349fb6983ff379e Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Tue, 26 May 2020 10:22:37 +0200
Subject: [PATCH] read-file: add variants that clear internal memory
* lib/read-file.h: Add declarations for fread_file_clear,
read_file_clear, and read_binary_file_clear.
* lib/read-file.c (clear_free, clear_realloc, fast_free)
(fast_realloc): New internal functions.
(internal_fread_file): Take free and realloc substitutes.
(internal_read_file): Take fread_file substitute.
(fread_file_clear, read_file_clear, read_binary_file_clear): New
functions.
* modules/read-file (Depends-on): Add explicit_bzero.
This adds a variant of those functions to explicitly
clear the internal memory block when it becomes unused.
---
ChangeLog | 15 ++
lib/read-file.c | 111 +++-
lib/read-file.h | 6 +++
modules/read-file | 1 +
modules/read-file-tests | 1 +
tests/test-read-file.c | 15 --
6 files changed, 132 insertions(+), 17 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 07d4d5124..c451ff057 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,18 @@
+2020-05-26 Daiki Ueno
+
+ read-file: add variants that clear internal memory
+ * lib/read-file.h: Add declarations for fread_file_clear,
+ read_file_clear, and read_binary_file_clear.
+ * lib/read-file.c (clear_free, clear_realloc, fast_free)
+ (fast_realloc): New internal functions.
+ (internal_fread_file): Take free and realloc substitutes.
+ (internal_read_file): Take fread_file substitute.
+ (fread_file_clear, read_file_clear, read_binary_file_clear): New
+ functions.
+ * modules/read-file (Depends-on): Add explicit_bzero.
+ This adds a variant of those functions to explicitly
+ clear the internal memory block when it becomes unused.
+
2020-05-26 Daiki Ueno
read-file: make use of fopen-gnu
diff --git a/lib/read-file.c b/lib/read-file.c
index 293bc3e8a..3fd94b046 100644
--- a/lib/read-file.c
+++ b/lib/read-file.c
@@ -31,16 +31,62 @@
/* Get malloc, realloc, free. */
#include
+/* Get explicit_bzero, memcpy. */
+#include
+
/* Get errno. */
#include
-/* Read a STREAM and return a newly allocated string with the content,
- and set *LENGTH to the length of the string. The string is
- zero-terminated, but the terminating zero byte is not counted in
- *LENGTH. On errors, *LENGTH is undefined, errno preserves the
- values set by system functions (if any), and NULL is returned. */
-char *
-fread_file (FILE *stream, size_t *length)
+/* Get assert. */
+#include
+
+static void
+clear_free (void *ptr, size_t old_size)
+{
+ if (ptr)
+{
+ explicit_bzero (ptr, old_size);
+ free (ptr);
+}
+}
+
+static void *
+clear_realloc (void *ptr, size_t old_size, size_t new_size)
+{
+ void *new_ptr;
+
+ assert (ptr);
+ assert (new_size);
+
+ if (new_size < old_size)
+{
+ explicit_bzero (ptr + new_size, old_size - new_size);
+ return ptr;
+}
+
+ new_ptr = malloc (new_size);
+ memcpy (new_ptr, ptr, old_size);
+ clear_free (ptr, old_size);
+ return new_ptr;
+}
+
+static void
+fast_free (void *ptr, size_t old_size _GL_UNUSED)
+{
+ if (ptr)
+free (ptr);
+}
+
+static void *
+fast_realloc (void *ptr, size_t old_size _GL_UNUSED, size_t new_size)
+{
+ return realloc (ptr, new_size);
+}
+
+static char *
+internal_fread_file (FILE *stream, size_t *length,
+ void (*free_func)(void *, size_t),
+ void *(*realloc_func)(void *, size_t, size_t))
{
char *buf = NULL;
size_t alloc = BUFSIZ;
@@ -94,7 +140,7 @@ fread_file (FILE *stream, size_t *length)
/* Shrink the allocated memory if possible. */
if (size < alloc - 1)
{
-char *smaller_buf = realloc (buf, size + 1);
+char *smaller_buf = realloc_func (buf, alloc, size + 1);
if (smaller_buf != NULL)
buf = smaller_buf;
}
@@ -106,6 +152,7 @@ fread_file (FILE *stream, size_t *length)
{
char *new_buf;
+ size_t save_alloc = alloc;
if (alloc == PTRDIFF_MAX)
{
@@ -118,7 +165,7 @@ fread_file (FILE *stream, size_t *length)
else
alloc = PTRDIFF_MAX;
- if (!(new_buf = realloc (buf, alloc)))
+ if (!(new_buf = realloc_func (buf, save_alloc, alloc)))
{
save_errno = errno;
break;
@@ -128,14 +175,34 @@ fread_file (FILE *stream, size_t *length)
}
}
-free (buf);
+free_func (buf, allo
Re: [PATCH] read-file: add variants that clear internal memory
Bruno Haible writes:
> If you agree, I'd like to see two commits:
> 1. the introduction of the flags and RF_BINARY,
> 2. the RF_SENSITIVE flag.
>
> Do you want me to code the first commit, or do you want to do it?
Sure, that would make things much simpler. I'm attaching a patch along
these lines.
Regards,
--
Daiki Ueno
>From 5a40a392b3b551cf92d4be022b4efd900f339e39 Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Tue, 26 May 2020 10:22:37 +0200
Subject: [PATCH] read-file: add flags to modify reading behavior
* lib/read-file.h (RF_BINARY, RF_SENSITIVE): New define.
(fread_file, read_file): Take FLAGS argument.
(read_binary_file): Remove.
* lib/read-file.c (internal_fread_file): Take into account of
RF_SENSITIVE flag.
(internal_read_file): Take into account of RF_BINARY and
RF_SENSITIVE flags.
* modules/read-file (Depends-on): Add explicit_bzero.
This adds an alternative behavior of those functions to explicitly
clear the internal memory block when it becomes unused. This is
useful for reading sensitive information from a file.
---
ChangeLog | 15
lib/read-file.c | 85 -
lib/read-file.h | 10 +++--
modules/read-file | 1 +
modules/read-file-tests | 1 +
tests/test-read-file.c | 17 +++--
6 files changed, 97 insertions(+), 32 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 07d4d5124..d1ee01a54 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,18 @@
+2020-05-26 Daiki Ueno
+
+ read-file: add flags to modify reading behavior
+ * lib/read-file.h (RF_BINARY, RF_SENSITIVE): New define.
+ (fread_file, read_file): Take FLAGS argument.
+ (read_binary_file): Remove.
+ * lib/read-file.c (internal_fread_file): Take into account of
+ RF_SENSITIVE flag.
+ (internal_read_file): Take into account of RF_BINARY and
+ RF_SENSITIVE flags.
+ * modules/read-file (Depends-on): Add explicit_bzero.
+ This adds an alternative behavior of those functions to explicitly
+ clear the internal memory block when it becomes unused. This is
+ useful for reading sensitive information from a file.
+
2020-05-26 Daiki Ueno
read-file: make use of fopen-gnu
diff --git a/lib/read-file.c b/lib/read-file.c
index 293bc3e8a..d95aae168 100644
--- a/lib/read-file.c
+++ b/lib/read-file.c
@@ -31,16 +31,17 @@
/* Get malloc, realloc, free. */
#include
+/* Get explicit_bzero, memcpy. */
+#include
+
/* Get errno. */
#include
-/* Read a STREAM and return a newly allocated string with the content,
- and set *LENGTH to the length of the string. The string is
- zero-terminated, but the terminating zero byte is not counted in
- *LENGTH. On errors, *LENGTH is undefined, errno preserves the
- values set by system functions (if any), and NULL is returned. */
-char *
-fread_file (FILE *stream, size_t *length)
+/* Get assert. */
+#include
+
+static char *
+internal_fread_file (FILE *stream, int flags, size_t *length)
{
char *buf = NULL;
size_t alloc = BUFSIZ;
@@ -94,7 +95,12 @@ fread_file (FILE *stream, size_t *length)
/* Shrink the allocated memory if possible. */
if (size < alloc - 1)
{
-char *smaller_buf = realloc (buf, size + 1);
+char *smaller_buf;
+
+if (flags & RF_SENSITIVE)
+ explicit_bzero (buf + size, alloc - size);
+
+smaller_buf = realloc (buf, size + 1);
if (smaller_buf != NULL)
buf = smaller_buf;
}
@@ -106,6 +112,7 @@ fread_file (FILE *stream, size_t *length)
{
char *new_buf;
+ size_t save_alloc = alloc;
if (alloc == PTRDIFF_MAX)
{
@@ -118,7 +125,21 @@ fread_file (FILE *stream, size_t *length)
else
alloc = PTRDIFF_MAX;
- if (!(new_buf = realloc (buf, alloc)))
+ if (flags & RF_SENSITIVE)
+{
+ new_buf = malloc (alloc);
+ if (!new_buf)
+{
+ /* BUF should be cleared below after the loop. */
+ save_errno = errno;
+ break;
+}
+ memcpy (new_buf, buf, save_alloc);
+ explicit_bzero (buf, save_alloc);
+ free (buf);
+ buf = new_buf;
+}
+ else if (!(new_buf = realloc (buf, alloc)))
{
save_errno = errno;
break;
@@ -128,14 +149,32 @@ fread_file (FILE *stream, size_t *length)
}
}
+if (flags & RF_SENSITIVE)
+ explicit_bzero (buf, alloc);
+
free (buf);
errno = save_errno;
return NULL;
}
}
+/* Read a STREAM and return a newly allocated string with the content,
+ and set *LENGTH to the length of the string. The string is
+ zero-terminated, but the terminating zero byte is not counted in
+ *LENGTH. On errors, *LENGTH is undef
Re: [PATCH] read-file: add variants that clear internal memory
Bruno Haible writes:
> It would be useful to first concentrate on the first part, the refactoring
> that introduces flags and RF_BINARY. This would provide a patch that is easier
> to review and does not have the following problems:
> - internal_fread_file still exists, although fread_file is a no-op wrapper
> around it.
> - In tests/test-read-file.c, please terminate the main() function with a
> return statement. We assume C99 only in modules that explicitly list 'c99'
> as a dependency. If it's trivial to avoid this dependency, let's do it.
> - The NEWS file needs an entry.
Indeed, I missed the last paragraph of your suggestion in the previous
mail; sorry about that. Here are the two separate commits for this.
>From 60608590e2b106708dd74fd31331567af5166d2e Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Wed, 27 May 2020 08:14:44 +0200
Subject: [PATCH 1/2] read-file: add flags to modify reading behavior
* lib/read-file.h (RF_BINARY): New define.
(fread_file, read_file): Take FLAGS argument.
(read_binary_file): Remove.
* lib/read-file.c (internal_read_file): Merge into ...
(read_file): ... here.
* modules/read-file-tests (Files): Add "tests/macros.h".
* tests/test-read-file.c (main): Refactor using ASSERT macro.
* NEWS: Mention this change.
---
ChangeLog | 12
NEWS| 5 +
lib/read-file.c | 43 ++---
lib/read-file.h | 7 ---
modules/read-file-tests | 1 +
tests/test-read-file.c | 17 +---
6 files changed, 50 insertions(+), 35 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 07d4d5124..94faf6984 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,15 @@
+2020-05-27 Daiki Ueno
+
+ read-file: add flags to modify reading behavior
+ * lib/read-file.h (RF_BINARY): New define.
+ (fread_file, read_file): Take FLAGS argument.
+ (read_binary_file): Remove.
+ * lib/read-file.c (internal_read_file): Merge into ...
+ (read_file): ... here.
+ * modules/read-file-tests (Files): Add "tests/macros.h".
+ * tests/test-read-file.c (main): Refactor using ASSERT macro.
+ * NEWS: Mention this change.
+
2020-05-26 Daiki Ueno
read-file: make use of fopen-gnu
diff --git a/NEWS b/NEWS
index 99973c5c3..c559a65e9 100644
--- a/NEWS
+++ b/NEWS
@@ -3,6 +3,11 @@ Important general notes
DateModules Changes
+2020-05-27 read-file The functions provided by this module now take an
+'int flags' argument to modify the file reading
+behavior. The read_binary_file function has been
+removed as it is no longer necessary.
+
2019-12-11 Support for These modules are now supported in C++ mode as well.
ISO C or POSIX This means, while the autoconfiguration uses the C
functions compiler, the resulting header files and function
diff --git a/lib/read-file.c b/lib/read-file.c
index 293bc3e8a..904f1c901 100644
--- a/lib/read-file.c
+++ b/lib/read-file.c
@@ -40,7 +40,7 @@
*LENGTH. On errors, *LENGTH is undefined, errno preserves the
values set by system functions (if any), and NULL is returned. */
char *
-fread_file (FILE *stream, size_t *length)
+fread_file (FILE *stream, int flags _GL_UNUSED, size_t *length)
{
char *buf = NULL;
size_t alloc = BUFSIZ;
@@ -134,9 +134,19 @@ fread_file (FILE *stream, size_t *length)
}
}
-static char *
-internal_read_file (const char *filename, size_t *length, const char *mode)
+/* Open and read the contents of FILENAME, and return a newly
+ allocated string with the content, and set *LENGTH to the length of
+ the string. The string is zero-terminated, but the terminating
+ zero byte is not counted in *LENGTH. On errors, *LENGTH is
+ undefined, errno preserves the values set by system functions (if
+ any), and NULL is returned.
+
+ If the RF_BINARY flag is set in FLAGS, the file is opened in binary
+ mode. */
+char *
+read_file (const char *filename, int flags, size_t *length)
{
+ const char *mode = (flags & RF_BINARY) ? "rbe" : "re";
FILE *stream = fopen (filename, mode);
char *out;
int save_errno;
@@ -144,7 +154,7 @@ internal_read_file (const char *filename, size_t *length, const char *mode)
if (!stream)
return NULL;
- out = fread_file (stream, length);
+ out = fread_file (stream, flags, length);
save_errno = errno;
@@ -161,28 +171,3 @@ internal_read_file (const char *filename, size_t *length, const char *mode)
return out;
}
-
-/* Open and read the contents of FILENAME, and return a newly
- allocated string with the content, and set *LENGTH to the length of
- the string. The string is zero-terminated, but the terminating
- zero byte is not counted in *LENGTH. On errors, *LENGTH is
- undefined, errno preserves the values set by s
Re: [PATCH] read-file: add variants that clear internal memory
Bruno Haible writes: > The first one is nearly perfect. Only the NEWS entry should go into section > "User visible incompatible changes", not into section "Important general > notes". > > The second one: > - In fread_file, around line 100, you use a realloc workaround that appears > to be not as reliable as the malloc+free based workaround around line 135. > - In read_file, please don't use tabs for indentation. We've switched to > spaces-only indentation long ago. > > Once this is fixed, please push. Both has been fixed and pushed. Thank you for the review! Regards, -- Daiki Ueno
Re: portability of fopen and 'e' (O_CLOEXEC) flag
Bruno Haible writes:
>> Here are proposed patches for other modules. Does this look right?
>
> There were no objections. I pushed the changes.
Thank you for this. I have rebased GnuTLS on top of it, but noticed a
strange test failures on Windows CI, which involve reading binary files
(OCSP response):
https://gitlab.com/gnutls/gnutls/-/jobs/569815031
It seems that the fopen module ignores a 'b' flag. The attached patch
fixes the failures.
Regards,
--
Daiki Ueno
>From 17fbb2560a05e3006125f8793c8e814ef5baa847 Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Thu, 28 May 2020 11:40:49 +0200
Subject: [PATCH] fopen-gnu: make 'b' flag can be used with 'e' on Windows
* lib/fopen.c (rpl_fopen): Pass O_BINARY to open, if a 'b' flag is
specified on Windows.
* tests/test-fopen-gnu.c (DATA): New define.
(main): Add test for reading binary files with an 'e' flag.
---
ChangeLog | 8
lib/fopen.c| 9 +++--
tests/test-fopen-gnu.c | 17 +
3 files changed, 32 insertions(+), 2 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index c17b76b72..ea2716b2f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2020-05-28 Daiki Ueno
+
+ fopen-gnu: make 'b' flag can be used with 'e' on Windows
+ * lib/fopen.c (rpl_fopen): Pass O_BINARY to open, if a 'b' flag is
+ specified on Windows.
+ * tests/test-fopen-gnu.c (DATA): New define.
+ (main): Add test for reading binary files with an 'e' flag.
+
2020-05-27 Bruno Haible
Don't assume that UNICODE is not defined.
diff --git a/lib/fopen.c b/lib/fopen.c
index 20065e4c6..f60c51f95 100644
--- a/lib/fopen.c
+++ b/lib/fopen.c
@@ -51,6 +51,7 @@ rpl_fopen (const char *filename, const char *mode)
int open_flags_standard;
#if GNULIB_FOPEN_GNU
int open_flags_gnu;
+ int open_flags_other;
# define BUF_SIZE 80
char fdopen_mode_buf[BUF_SIZE + 1];
#endif
@@ -66,6 +67,7 @@ rpl_fopen (const char *filename, const char *mode)
open_flags_standard = 0;
#if GNULIB_FOPEN_GNU
open_flags_gnu = 0;
+ open_flags_other = 0;
#endif
{
const char *p = mode;
@@ -101,6 +103,9 @@ rpl_fopen (const char *filename, const char *mode)
#endif
continue;
case 'b':
+#if defined _WIN32 && ! defined __CYGWIN__
+ open_flags_other |= O_BINARY;
+#endif
#if GNULIB_FOPEN_GNU
if (q < fdopen_mode_buf + BUF_SIZE)
*q++ = *p;
@@ -142,9 +147,9 @@ rpl_fopen (const char *filename, const char *mode)
#endif
}
#if GNULIB_FOPEN_GNU
- open_flags = open_flags_standard | open_flags_gnu;
+ open_flags = open_flags_standard | open_flags_other | open_flags_gnu;
#else
- open_flags = open_flags_standard;
+ open_flags = open_flags_standard | open_flags_other;
#endif
#if FOPEN_TRAILING_SLASH_BUG
diff --git a/tests/test-fopen-gnu.c b/tests/test-fopen-gnu.c
index cae40421a..eeb1712c7 100644
--- a/tests/test-fopen-gnu.c
+++ b/tests/test-fopen-gnu.c
@@ -29,15 +29,20 @@
#define BASE "test-fopen-gnu.t"
+/* 0x1a is an EOF on Windows. */
+#define DATA "abc\x1adef"
+
int
main (void)
{
FILE *f;
int fd;
int flags;
+ char buf[16];
/* Remove anything from prior partial run. */
unlink (BASE "file");
+ unlink (BASE "binary");
/* Create the file. */
f = fopen (BASE "file", "w");
@@ -64,8 +69,20 @@ main (void)
ASSERT (f == NULL);
ASSERT (errno == EEXIST);
+ /* Open a binary file and check that the 'e' mode doesn't interfere. */
+ f = fopen (BASE "binary", "wbe");
+ ASSERT (f);
+ ASSERT (fwrite (DATA, 1, sizeof(DATA)-1, f) == sizeof(DATA)-1);
+ ASSERT (fclose (f) == 0);
+
+ f = fopen (BASE "binary", "rbe");
+ ASSERT (f);
+ ASSERT (fread (buf, 1, sizeof(buf), f) == sizeof(DATA)-1);
+ ASSERT (fclose (f) == 0);
+
/* Cleanup. */
ASSERT (unlink (BASE "file") == 0);
+ ASSERT (unlink (BASE "binary") == 0);
return 0;
}
--
2.26.2
Re: portability of fopen and 'e' (O_CLOEXEC) flag
Daiki Ueno writes:
> Bruno Haible writes:
>
>>> Here are proposed patches for other modules. Does this look right?
>>
>> There were no objections. I pushed the changes.
>
> Thank you for this. I have rebased GnuTLS on top of it, but noticed a
> strange test failures on Windows CI, which involve reading binary files
> (OCSP response):
> https://gitlab.com/gnutls/gnutls/-/jobs/569815031
>
> It seems that the fopen module ignores a 'b' flag. The attached patch
> fixes the failures.
Sorry, attached an old patch; this would be simpler (and also supports
other platforms that need O_BINARY).
>From 81695244eb467603009a2777c3a8438f1a707954 Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Thu, 28 May 2020 11:40:49 +0200
Subject: [PATCH] fopen-gnu: make 'b' flag can be used with 'e' on Windows
* lib/fopen.c (rpl_fopen): Pass O_BINARY to open, if a 'b' flag is
specified on Windows.
* tests/test-fopen-gnu.c (DATA): New define.
(main): Add test for reading binary files with an 'e' flag.
---
ChangeLog | 8
lib/fopen.c| 4
tests/test-fopen-gnu.c | 17 +
3 files changed, 29 insertions(+)
diff --git a/ChangeLog b/ChangeLog
index c17b76b72..ea2716b2f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2020-05-28 Daiki Ueno
+
+ fopen-gnu: make 'b' flag can be used with 'e' on Windows
+ * lib/fopen.c (rpl_fopen): Pass O_BINARY to open, if a 'b' flag is
+ specified on Windows.
+ * tests/test-fopen-gnu.c (DATA): New define.
+ (main): Add test for reading binary files with an 'e' flag.
+
2020-05-27 Bruno Haible
Don't assume that UNICODE is not defined.
diff --git a/lib/fopen.c b/lib/fopen.c
index 20065e4c6..47d7f194d 100644
--- a/lib/fopen.c
+++ b/lib/fopen.c
@@ -101,6 +101,10 @@ rpl_fopen (const char *filename, const char *mode)
#endif
continue;
case 'b':
+/* While it is non-standard, O_BINARY is guaranteed by
+ gnulib . We can also assume that orig_fopen
+ supports the 'b' flag. */
+open_flags_standard |= O_BINARY;
#if GNULIB_FOPEN_GNU
if (q < fdopen_mode_buf + BUF_SIZE)
*q++ = *p;
diff --git a/tests/test-fopen-gnu.c b/tests/test-fopen-gnu.c
index cae40421a..eeb1712c7 100644
--- a/tests/test-fopen-gnu.c
+++ b/tests/test-fopen-gnu.c
@@ -29,15 +29,20 @@
#define BASE "test-fopen-gnu.t"
+/* 0x1a is an EOF on Windows. */
+#define DATA "abc\x1adef"
+
int
main (void)
{
FILE *f;
int fd;
int flags;
+ char buf[16];
/* Remove anything from prior partial run. */
unlink (BASE "file");
+ unlink (BASE "binary");
/* Create the file. */
f = fopen (BASE "file", "w");
@@ -64,8 +69,20 @@ main (void)
ASSERT (f == NULL);
ASSERT (errno == EEXIST);
+ /* Open a binary file and check that the 'e' mode doesn't interfere. */
+ f = fopen (BASE "binary", "wbe");
+ ASSERT (f);
+ ASSERT (fwrite (DATA, 1, sizeof(DATA)-1, f) == sizeof(DATA)-1);
+ ASSERT (fclose (f) == 0);
+
+ f = fopen (BASE "binary", "rbe");
+ ASSERT (f);
+ ASSERT (fread (buf, 1, sizeof(buf), f) == sizeof(DATA)-1);
+ ASSERT (fclose (f) == 0);
+
/* Cleanup. */
ASSERT (unlink (BASE "file") == 0);
+ ASSERT (unlink (BASE "binary") == 0);
return 0;
}
--
2.26.2
Regards,
--
Daiki Ueno
Re: portability of fopen and 'e' (O_CLOEXEC) flag
Bruno Haible writes: >> + ASSERT (fwrite (DATA, 1, sizeof(DATA)-1, f) == sizeof(DATA)-1); > ... >> + ASSERT (fread (buf, 1, sizeof(buf), f) == sizeof(DATA)-1); > > GNU coding style wants a space between 'sizeof' and the opening parenthesis. > Other than that, your patch is perfect. Thank you for the review; fixed and pushed. Afterwards, I realized a compilation error in the test code on FreeBSD, so I've added the follow-up patch attached. Regards, -- Daiki Ueno >From 9326739489050009e3b8834838abc02203c592e5 Mon Sep 17 00:00:00 2001 From: Daiki Ueno Date: Fri, 29 May 2020 04:54:31 +0200 Subject: [PATCH] fopen-gnu-tests: fix "\x" escape usage * tests/test-fopen-gnu.c (DATA): Use safer escape sequence. --- ChangeLog | 5 + tests/test-fopen-gnu.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/ChangeLog b/ChangeLog index c39bfd372..77c637414 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,8 @@ +2020-05-29 Daiki Ueno + + fopen-gnu-tests: fix "\x" escape usage + * tests/test-fopen-gnu.c (DATA): Use safer escape sequence. + 2020-05-28 Bruno Haible Avoid dynamic loading of Windows API functions when possible. diff --git a/tests/test-fopen-gnu.c b/tests/test-fopen-gnu.c index 4d98dcd85..7de45ab8d 100644 --- a/tests/test-fopen-gnu.c +++ b/tests/test-fopen-gnu.c @@ -30,7 +30,7 @@ #define BASE "test-fopen-gnu.t" /* 0x1a is an EOF on Windows. */ -#define DATA "abc\x1adef" +#define DATA "abc\x1axyz" int main (void) -- 2.26.2
Re: [PATCH] read-file: add variants that clear internal memory
Bruno Haible writes: > Let me update the uses of the module 'read-file' in Gnulib. > I think the next weekly CI run would have caught this. Thank you; I completely missed those uses in Gnulib. On a different note, it was suggested to disable stdio buffering if RF_SENSITIVE is set. I am attaching a patch for this. >From 9165e495461db91b8abc42661fc543784d26d0d6 Mon Sep 17 00:00:00 2001 From: Daiki Ueno Date: Fri, 29 May 2020 05:45:40 +0200 Subject: [PATCH] read-file: disable buffering if RF_SENSITIVE is set * lib/read-file.c (read_file): Call setvbuf if RF_SENSITIVE. Suggested by Glenn Strauss. --- ChangeLog | 6 ++ lib/read-file.c | 3 +++ 2 files changed, 9 insertions(+) diff --git a/ChangeLog b/ChangeLog index 77c637414..0a0e2301a 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,9 @@ +2020-05-29 Daiki Ueno + + read-file: disable buffering if RF_SENSITIVE is set + * lib/read-file.c (read_file): Call setvbuf if RF_SENSITIVE. + Suggested by Glenn Strauss. + 2020-05-29 Daiki Ueno fopen-gnu-tests: fix "\x" escape usage diff --git a/lib/read-file.c b/lib/read-file.c index 36780cc15..3520cbb7b 100644 --- a/lib/read-file.c +++ b/lib/read-file.c @@ -195,6 +195,9 @@ read_file (const char *filename, int flags, size_t *length) if (!stream) return NULL; + if (flags & RF_SENSITIVE) +setvbuf (stream, NULL, _IONBF, 0); + out = fread_file (stream, flags, length); save_errno = errno; -- 2.26.2 Regards, -- Daiki Ueno
Re: [PATCH] gettimeofday: do not use LoadLibrary when built for Windows Store apps
Hello, Bruno Haible writes: > diff --git a/lib/stat-w32.c b/lib/stat-w32.c > index b9163f5..02ad9ab 100644 > --- a/lib/stat-w32.c > +++ b/lib/stat-w32.c > @@ -40,18 +40,20 @@ > #include "pathmax.h" > #include "verify.h" > > +#if !(_WIN32_WINNT >= _WIN32_WINNT_VISTA) > + I am totally unfamiliar with Windows code, but this change seems to break MinGW cross build, because GetFinalPathNameByHandleFunc is defined only if !(_WIN32_WINNT >= _WIN32_WINNT_VISTA), but referred to from _gl_convert_FILETIME_to_timespec without the guard: ../../gl/stat-w32.c: In function '_gl_fstat_by_handle': ../../gl/stat-w32.c:259:23: error: 'GetFinalPathNameByHandleFunc' undeclared (first use in this function); did you mean 'GetFinalPathNameByHandleW'? 259 | || (GetFinalPathNameByHandleFunc != NULL | ^~~~ | GetFinalPathNameByHandleW Regards, -- Daiki Ueno
Re: [PATCH] read-file: add variants that clear internal memory
Bruno Haible writes: >> On a different note, it was suggested to disable stdio buffering if >> RF_SENSITIVE is set. I am attaching a patch for this. > > Reading from a regular file in an unbuffered way can be terribly slow. > But here, fread_file reads in large chunks, therefore it's OK. > > Also, in the specification of fread_file, I would add a note. Maybe like > this? Thank you, amended the documentation and pushed. Regards, -- Daiki Ueno
Re: parse-datetime: Fix compilation error with bison 3.7
Bruno Haible writes:
>> 2020-07-28 Bruno Haible
>>
>> parse-datetime: Fix compilation error with bison 3.7.
>> * modules/parse-datetime (Makefile.am): Create a generated header file
>> parse-datetime-gen.h in the source directory. Correct #include and
>> #line statements during preprocessing.
Thank you for those fixes; they are going to fix long standing build
failures in GnuTLS. One thing I noticed though is that
parse-datetime-gen.h has no dependency declaration and fails to build
with parallel make:
make[2]: *** No rule to make target 'parse-datetime-gen.h', needed by 'all'.
Stop.
make[2]: *** Waiting for unfinished jobs
YACC parse-datetime.c
make[2]: Leaving directory '/tmp/gnutls/build/src/gl'
make[1]: *** [Makefile:1761: all-recursive] Error 1
The attached patch fixes it; OK to push?
Regards,
--
Daiki Ueno
>From 1ba354a39b0ae34852a4ab9b25137fe65e59cd0f Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Sun, 13 Sep 2020 08:17:13 +0200
Subject: [PATCH] parse-datetime: fix compilation error with parallel make
* modules/parse-datetime (Makefile.am): Mark explicit dependency
from parse-datetime-gen.h to parse-datetime.c generation.
---
ChangeLog | 6 ++
modules/parse-datetime | 1 +
2 files changed, 7 insertions(+)
diff --git a/ChangeLog b/ChangeLog
index 66aec61cb..2ef8d7585 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+2020-09-13 Daiki Ueno
+
+ parse-datetime: fix compilation error with parallel make
+ * modules/parse-datetime (Makefile.am): Mark explicit dependency
+ from parse-datetime-gen.h to parse-datetime.c generation.
+
2020-09-12 Paul Eggert
dfa: epsilon-closure tweaks (Bug#40634)
diff --git a/modules/parse-datetime b/modules/parse-datetime
index 14675194e..ced61abf4 100644
--- a/modules/parse-datetime
+++ b/modules/parse-datetime
@@ -43,6 +43,7 @@ Makefile.am:
# Additionally, here we assume GNU Bison and therefore don't need the ylwrap
# script.
# Therefore we override this rule.
+parse-datetime-gen.h: parse-datetime.c
parse-datetime.c: parse-datetime.y
$(AM_V_YACC)$(PARSE_DATETIME_BISON) -d $(YFLAGS) $(AM_YFLAGS) $(srcdir)/parse-datetime.y \
&& test ':' = '$(PARSE_DATETIME_BISON)' || { \
--
2.26.2
Re: parse-datetime: Fix compilation error with bison 3.7
Bruno Haible writes: > (III) Use portable make syntax and still allow parallel make. > > Below I apply the approach (III). > > [1] https://lists.gnu.org/archive/html/bug-make/2019-05/msg00020.html > [2] https://lists.gnu.org/archive/html/bug-make/2020-09/msg8.html > > > 2020-09-13 Bruno Haible > > parse-datetime: Make the build rule work with parallel 'make'. > Reported by Daiki Ueno in > <https://lists.gnu.org/archive/html/bug-gnulib/2020-09/msg00036.html>. > * modules/parse-datetime (Makefile.am): Use a phony target and the > general idiom for rules that produce multiple files. Thank you! I confirm that it works nicely in our CI. Regards, -- Daiki Ueno
"GNULIB_STRERROR" redefined in single-configure package
Hello, In GnuTLS, we import Gnulib sources in two phases, first LGPL sources to be linked with the library, and then GPL sources for the tools and tests, something like following: In bootstrap: gnulib-tool --extract-recursive-dependencies "$gnulib_modules" gnulib-tool --import --local-dir=src/gl/override --lib=libgnu_gpl --source-base=src/gl --m4-base=src/gl/m4 --doc-base=doc --aux-dir=build-aux --no-conditional-dependencies --libtool --macro-prefix=ggl --without-tests --no-vc-files "$src_modules" In configure.ac: gl_INIT ggl_INIT It had worked until we added 'xalloc' to $src_modules, but now, confdefs.h generated during the configure phase contains two definitions of GNULIB_STRERROR and it prevents compilation with -Werror: configure:53786: gcc -o conftest -O0 -Wall -Werror -g3 conftest.c -lev >&5 conftest.c:412: error: "GNULIB_STRERROR" redefined [-Werror] 412 | #define GNULIB_STRERROR 1 | conftest.c:305: note: this is the location of the previous definition 305 | #define GNULIB_STRERROR IN_GNUTLS_GNULIB_TESTS | cc1: all warnings being treated as errors Could anyone shed some light on this? Regards, -- Daiki Ueno
Re: "GNULIB_STRERROR" redefined in single-configure package
Hello Bruno, Bruno Haible writes: > Currently your situation is: > > same configure module set (A) + tests of (A) > module set (A∪B) > > I can see two reasonable ways to modify the situation: > > (a) 1/configure module set (A) > module set (A∪B) > > 2/configure module set (A) + tests of (A) > > In other words, use a second directory purely for testing (A). > Two two directories 1/configure and 2/configure will be completely > separate, in particular no -I options that point from 2/Makefile to 1/. > > (b) same configure module set (A) > module set (A∪B) + tests of (A∪B) > > In other words, test more modules, not only those of the subset (A). > This should work, because gnulib-tool has been tested numerous times > with module sets (U) ⊂ (V) ⊂ (W), whereas the situation that causes > the problem is when it's not well-ordered. Thank you so much for the explanation and the suggestions. I've tried (b) as it seems to be slighly more lightweight, and it worked: https://gitlab.com/gnutls/gnutls/-/merge_requests/1345/diffs?commit_id=1fc78a5a3e6646e4082be7ec1c4d60b9d40213d0 Regards, -- Daiki Ueno
[PATCH] read-file: remove dead assignment
>From 69b8df2c5b011b09c8d03c13566356a906bd89dc Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Sat, 21 Nov 2020 17:32:58 +0100
Subject: [PATCH] read-file: remove dead assignment
* lib/read-file.c (fread_file): Remove dead assignment when
RF_SENSITIVE is set, flagged by clang-analyzer.
---
ChangeLog | 6 ++
lib/read-file.c | 1 -
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/ChangeLog b/ChangeLog
index 64be77149..95d40a496 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+2020-11-21 Daiki Ueno
+
+ read-file: remove dead assignment
+ * lib/read-file.c (fread_file): Remove dead assignment when
+ RF_SENSITIVE is set, flagged by clang-analyzer.
+
2020-11-19 Akim Demaille
bitset: tests: exercise the stats too
diff --git a/lib/read-file.c b/lib/read-file.c
index 88bc0070a..be93a7e09 100644
--- a/lib/read-file.c
+++ b/lib/read-file.c
@@ -156,7 +156,6 @@ fread_file (FILE *stream, int flags, size_t *length)
memcpy (new_buf, buf, save_alloc);
explicit_bzero (buf, save_alloc);
free (buf);
- buf = new_buf;
}
else if (!(new_buf = realloc (buf, alloc)))
{
--
2.26.2
relicense 'reallocarray' to LGPLv2+?
Hello Darshit, I'd like to use the 'reallocarray' module[1] for the integer overflow hardening in GnuTLS[2]. Currently the module is licensed under "LGPL" (which I believe means LGPLv3+), while the library part of GnuTLS requires the imported code to be LGPLv2+. Therefore I would like to ask your permission to change it to "LGPLv2+". Would you agree with that? Footnotes: [1] https://lists.gnu.org/archive/html/bug-gnulib/2017-08/msg00057.html [2] https://gitlab.com/gnutls/gnutls/-/issues/1179 Regards, -- Daiki Ueno
gl_YEAR2038_EARLY is called too late
Hello Paul, I have seen a test failure when cross-compiling GnuTLS to mingw32 and running the testsuites with Wine[1]. It looks like the library mis-calculates timeout value for DTLS, because of a confusion in time_t size detection. The configure output shows some clue: checking for 64-bit time_t... no checking for 64-bit time_t with _TIME_BITS=64... no configure: WARNING: This package requires a 64-bit 'time_t' type if there is any way to access timestamps outside the year range 1901-2038 on your platform. Perhaps you should configure with 'CPPFLAGS="-m64" LDFLAGS="-m64"'? checking whether time_t is signed... yes checking size of time_t... 8 where I observe: - AC_CHECK_SIZEOF reports time_t is 64-bit, as gl_YEAR2038_EARLY sets __MINGW_USE_VC2005_COMPAT - gl_YEAR2038_BODY is called before gl_YEAR2038_EARLY as a dependency of AC_SYS_LARGEFILE - Therefore checks in gl_YEAR2038_BODY are not affected by __MINGW_USE_VC2005_COMPAT and report that time_t is NOT 64-bit I guess a simple solution would be to swap the order of gl_YEAR2038_EARLY and AC_SYS_LARGEFILE in modules/largefile, as with the attached patch. This doesn't fully resolve the issue with DTLS tests as mentioned, and I suspect there might be some other confusions. Any suggestions would be appreciated. Footnotes: [1] https://gitlab.com/dueno/gnutls/-/jobs/2613997489 Regards, -- Daiki Ueno >From eac9e4fe9fd9e432bce8765059ae29559258344f Mon Sep 17 00:00:00 2001 From: Daiki Ueno Date: Mon, 26 Sep 2022 13:48:42 +0900 Subject: [PATCH] largefile: fix detection of time_t size on mingw32 * modules/largefile (configure.ac-early): Ensure gl_YEAR2038_EARLY is invoked before gl_YEAR2038_BODY. --- ChangeLog | 6 ++ modules/largefile | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/ChangeLog b/ChangeLog index 6027e5ed94..3b97909de5 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,9 @@ +2022-09-26 Daiki Ueno + + largefile: fix detection of time_t size on mingw32 + * modules/largefile (configure.ac-early): Ensure gl_YEAR2038_EARLY + is invoked before gl_YEAR2038_BODY. + 2022-09-25 Paul Eggert fts: fix errno handling if dirfd fails diff --git a/modules/largefile b/modules/largefile index 8eb438a8e9..9e1d9f052a 100644 --- a/modules/largefile +++ b/modules/largefile @@ -13,8 +13,8 @@ m4/year2038.m4 Depends-on: configure.ac-early: -AC_REQUIRE([AC_SYS_LARGEFILE]) AC_REQUIRE([gl_YEAR2038_EARLY]) +AC_REQUIRE([AC_SYS_LARGEFILE]) configure.ac: AC_REQUIRE([gl_LARGEFILE]) -- 2.37.3
Re: gl_YEAR2038_EARLY is called too late
Paul Eggert writes:
> On 9/25/22 21:56, Daiki Ueno wrote:
>> I guess a simple solution would be to swap the order of
>> gl_YEAR2038_EARLY and AC_SYS_LARGEFILE in modules/largefile, as with the
>> attached patch.
>
> Thanks, I installed that.
>
>> This doesn't fully resolve the issue with DTLS tests as mentioned, and I
>> suspect there might be some other confusions. Any suggestions would be
>> appreciated.
>
> What are the symptoms of these other failures? The URL you sent
> doesn't make that obvious to me (I don't know MinGW).
The symptoms are the same for all other failures: the DTLS stack always
reports timeout at this condition:
https://gitlab.com/gnutls/gnutls/-/blob/cde0061c33d56258281a4d6033eba1d85e75e32d/lib/handshake.h#L124
even if timeout has not reached.
I suspect this might not be an issue in Gnulib, as I see the following
standalone program behaves differently depending on
__MINGW_USE_VC2005_COMPAT:
$ cat > test.c
#include
#include
int
main (void)
{
struct timespec start, end, now;
clock_gettime (CLOCK_REALTIME, &start);
end.tv_sec = start.tv_sec + 10;
end.tv_nsec = start.tv_nsec;
clock_gettime (CLOCK_REALTIME, &now);
printf ("%zd\n", start.tv_sec);
printf ("%zd\n", now.tv_sec);
printf ("%zd\n", end.tv_sec);
printf ("%d\n", start.tv_sec < now.tv_sec);
printf ("%d\n", now.tv_sec < end.tv_sec);
return 0;
}
^D
$ i686-w64-mingw32-gcc -o test test.c
$ wine ./test.exe
1664173364
1664173364
1664173374
0
1
$ i686-w64-mingw32-gcc -D__MINGW_USE_VC2005_COMPAT=1 -o test test.c
$ wine ./test.exe
1664173389
1664173389
1664173399
1
0
Regards,
--
Daiki Ueno
Re: license of lib/uniwidth/width.c
Bruno Haible writes: > The reason is the modules mbchar, mbiter, mbsnlen (all under LGPLv2+) depend > on > the modules uniwidth/base, uniwidth/width, which are "only" under > LGPLv3+|GPLv2. [...] > Daiki, would you agree to put lib/uniwidth/width.c under LGPLv2+ as well? Sure, I agree with that change. Regards, -- Daiki Ueno
Re: [bug-libunistring] [PATCH] unicase: Add missing prefix/suffix-context modules
Hello Bruno,
Bruno Haible writes:
> Genki Sky wrote:
>> These gnulib modules contain the following 12 functions:
>> - u{8,16,32}_casing_{prefix,suffix}_context
>> - u{8,16,32}_casing_{prefix,suffix}es_context
>>
>> As it stands, these functions were exposed in the header files, but were
>> not available in the shared object. Of course, this resulted in linker
>> errors.
>
> Thanks for the report. Part of the fix goes into gnulib:
>
>
> 2018-04-07 Bruno Haible
>
> unicase/u*-context: Fix link errors with libunistring <= 0.9.9.
> Reported by Genki Sky .
> * modules/unicase/u8-prefix-context (configure.ac): Require libunistring
> version 0.9.10 or newer.
> * modules/unicase/u8-suffix-context (configure.ac): Likewise.
> * modules/unicase/u16-prefix-context (configure.ac): Likewise.
> * modules/unicase/u16-suffix-context (configure.ac): Likewise.
> * modules/unicase/u32-prefix-context (configure.ac): Likewise.
While I am glad to see this has been fixed, I now wonder when we should
increase the version number and whether the above list is complete.
Usually, I bump the required version when there is any change in the
dependent modules since the last release, using the scripts:
https://git.savannah.gnu.org/gitweb/?p=libunistring.git;a=blob;f=Admin/README.update;h=76c9925c2cb63b86b6bbdc62037706c95ac399ed;hb=HEAD#l45
However, I don't see such changes since 0.9.8. Could you elaborate why
this change is appropriate, for future improvements of the release
process?
Regards,
--
Daiki Ueno
Re: [bug-libunistring] [PATCH] unicase: Add missing prefix/suffix-context modules
Bruno Haible writes: >> Usually, I bump the required version when there is any change in the >> dependent modules since the last release, using the scripts: >> >> https://git.savannah.gnu.org/gitweb/?p=libunistring.git;a=blob;f=Admin/README.update;h=76c9925c2cb63b86b6bbdc62037706c95ac399ed;hb=HEAD#l45 >> >> However, I don't see such changes since 0.9.8. Could you elaborate why >> this change is appropriate, for future improvements of the release >> process? > > Between 0.9.8 and 0.9.9 we did not change anything, because the only > relevant change was the multithread-safety fix of 'malloca'. Hmm, I don't > know whether a malloca() from before the change and a freea() from after > the change (or vice versa) (one coming from the libunistring.so, one > from the executable) will work well together... > > Between 0.9.9 and 0.9.10 we added a couple of modules to libunistring, > that were already included in gnulib. > Why the change is needed? When a program requests a module such as > 'unicase/u8-prefix-context' and the module description has > > gl_LIBUNISTRING_MODULE([0.9.8], [unicase/u8-prefix-context]) > > the autoconfiguration (when it finds a preinstalled libunistring > version 0.9.8 or 0.9.9) will not compile the module locally for the > executable, but reference the installed libunistring - thus leading > to a link error. Whereas with > > gl_LIBUNISTRING_MODULE([0.9.10], [unicase/u8-prefix-context]) > > the autoconfiguration will compile the module locally - thus filling > the "hole" in the installed libunistring. So, is my understanding correct that this only affects unreleased version of libunistring (0.9.10 is not released)? I was confused because you urged a new release for this: https://lists.gnu.org/archive/html/bug-libunistring/2018-04/msg3.html And if it is the case, my answer again would be "not this time", because the fix in the Gnulib git should be sufficient. Regards, -- Daiki Ueno
Re: bootstrap/autogen.sh and git submodules
Hello Bruno, Bruno Haible writes: > On 2019-03-03 I listed the problems that I have with the existing practices > of using gnulib as a submodule. > > Here's my attempt to fix this for the packages I maintain (gettext, > libunistring, etc.). It's a new script 'gitsub.sh'. I should have commented earlier, but I wanted to express my motivation behind the switch to using "git submodule" in gettext. That was not for making upgrades easier, but for making change tracking easier. Prior to the switch, I could only see the "Update after gnulib changed." messages in the "git log" output, without any mention of the target revisions of gnulib. That was making "git bisect" hard. With "git submodule", those gnulib revisions are automatically recorded and I can easily move back to a specific revision of the gettext, which is somehow guaranteed to compile. Therefore, if the new script provides a similar feature, without adding a manual process to our current practice, I wouldn't oppose. Otherwise it seems like a bit of a step back to me. Regards, -- Daiki Ueno
javacomp: conftest.java is deleted before compiling
Hi,
Regarding the bug report for gettext:
https://lists.gnu.org/archive/html/bug-gettext/2013-09/msg00010.html
javacomp.c has the following code to check if javac can compile some
Java programs. First, it writes out some code snippet to a temporary
file and then call javac command on the written file. Like this:
write_temp_file (tmpdir, conftest_file_name,
get_goodcode_snippet (source_version));
...
java_sources[0] = conftest_file_name;
compile_using_javac (java_sources, 1, ...);
write_temp_file opens the file with fopen_temp and close it with
fwriteerror_temp at the end:
static bool
write_temp_file (struct temp_dir *tmpdir, const char *file_name,
const char *contents)
{
FILE *fp;
register_temp_file (tmpdir, file_name);
fp = fopen_temp (file_name, "w");
...
if (fwriteerror_temp (fp))
}
As fopen_temp uses "D" mode on Windows, which implies the file will be
deleted on close (fwriteerror_temp here). Thus, after calling
write_temp_file, the file will be deleted immediately. Maybe we should
use plain fopen/fwriteerror here?
Regards,
--
Daiki Ueno
Re: javacomp: conftest.java is deleted before compiling
Eric Blake writes:
> That's not the first time I've encountered a problem with mingw's
> temporary files being deleted on close. See this m4 commit:
>
> git.sv.gnu.org/cgit/m4.git/commit/?id=11199a26
>
> So yes, it sounds like we need to avoid temporary files if we want them
> to persist past the close. Are you up to writing a patch along those lines?
I'm attaching a patch, though I wonder if the delete-on-close flag is
really useful - if temporary files don't persist, does it make sense to
check write errors in fwriteerror_temp?
Also, I found cleanup_temp_dir is missing in a few places in
javacomp.c. See the second patch for that.
Regards,
--
Daiki Ueno
>From 54c6e156989df415c53d44fb516d3b545b2ed80a Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Sun, 22 Sep 2013 18:50:43 +0900
Subject: [PATCH 1/2] javacomp: make conftest.java persist after write
* lib/javacomp.c (write_temp_file): Use plain fopen/fwriteerror instead of
fopen_temp/fwriteerror_temp to make the file persist after the call of this
function.
---
lib/javacomp.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/javacomp.c b/lib/javacomp.c
index fca7343..d37c924 100644
--- a/lib/javacomp.c
+++ b/lib/javacomp.c
@@ -497,7 +497,7 @@ write_temp_file (struct temp_dir *tmpdir, const char *file_name,
FILE *fp;
register_temp_file (tmpdir, file_name);
- fp = fopen_temp (file_name, "w");
+ fp = fopen (file_name, "w");
if (fp == NULL)
{
error (0, errno, _("failed to create \"%s\""), file_name);
@@ -505,7 +505,7 @@ write_temp_file (struct temp_dir *tmpdir, const char *file_name,
return true;
}
fputs (contents, fp);
- if (fwriteerror_temp (fp))
+ if (fwriteerror (fp))
{
error (0, errno, _("error while writing \"%s\" file"), file_name);
return true;
--
1.8.3.2
>From 95b6e3e2a7b9e63963bd55e1ae4d8c505e89784c Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Sun, 22 Sep 2013 19:05:57 +0900
Subject: [PATCH 2/2] javacomp: add missing cleanup of temporary directories
* lib/javacomp.c (is_envjavac_gcj43_usable)
(is_envjavac_nongcj_usable, is_gcj43_usable, is_javac_usable):
Cleanup temporary directory used to compile test programs.
---
lib/javacomp.c | 8
1 file changed, 8 insertions(+)
diff --git a/lib/javacomp.c b/lib/javacomp.c
index d37c924..4829654 100644
--- a/lib/javacomp.c
+++ b/lib/javacomp.c
@@ -858,6 +858,8 @@ is_envjavac_gcj43_usable (const char *javac,
free (compiled_file_name);
free (conftest_file_name);
+ cleanup_temp_dir (tmpdir);
+
resultp->tested = true;
}
@@ -1301,6 +1303,8 @@ is_envjavac_nongcj_usable (const char *javac,
free (compiled_file_name);
free (conftest_file_name);
+ cleanup_temp_dir (tmpdir);
+
resultp->tested = true;
}
@@ -1638,6 +1642,8 @@ is_gcj43_usable (const char *source_version,
free (compiled_file_name);
free (conftest_file_name);
+ cleanup_temp_dir (tmpdir);
+
resultp->tested = true;
}
@@ -2033,6 +2039,8 @@ is_javac_usable (const char *source_version, const char *target_version,
free (compiled_file_name);
free (conftest_file_name);
+ cleanup_temp_dir (tmpdir);
+
resultp->tested = true;
}
--
1.8.3.2
[PATCH] gettext: update macros to version 0.19
gettext-0.19 (and 0.19.1) has been released a while ago, this is the update of the M4 macros. There's no difference between 0.19 and 0.19.1. >From e3fd44d010a9eb0afb52d091268c80e8ac61e306 Mon Sep 17 00:00:00 2001 From: Daiki Ueno Date: Wed, 11 Jun 2014 18:06:11 +0900 Subject: [PATCH] gettext: update macros to version 0.19 * m4/intl.m4, m4/po.m4: Update from gettext-0.19. In particular, depend on gl_EXTERN_INLINE and drop support for older Bison versions. --- ChangeLog | 7 +++ m4/intl.m4 | 8 m4/po.m4 | 4 ++-- 3 files changed, 13 insertions(+), 6 deletions(-) diff --git a/ChangeLog b/ChangeLog index d2a42fe..0e9cc40 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,10 @@ +2014-06-11 Daiki Ueno + + gettext: update macros to version 0.19 + * m4/intl.m4, m4/po.m4: Update from gettext-0.19. In particular, + depend on gl_EXTERN_INLINE and drop support for older Bison + versions. + 2014-06-10 Eli Zaretskii select: fix waiting on anonymous pipes on MS-Windows diff --git a/m4/intl.m4 b/m4/intl.m4 index 1d782d1..1fd396c 100644 --- a/m4/intl.m4 +++ b/m4/intl.m4 @@ -1,4 +1,4 @@ -# intl.m4 serial 24 (gettext-0.18.3) +# intl.m4 serial 26 (gettext-0.19) dnl Copyright (C) 1995-2014 Free Software Foundation, Inc. dnl This file is free software; the Free Software Foundation dnl gives unlimited permission to copy and/or distribute it, @@ -42,6 +42,7 @@ AC_DEFUN([AM_INTL_SUBDIR], AC_REQUIRE([gl_XSIZE])dnl AC_REQUIRE([gl_FCNTL_O_FLAGS])dnl AC_REQUIRE([gt_INTL_MACOSX])dnl + AC_REQUIRE([gl_EXTERN_INLINE])dnl dnl Support for automake's --enable-silent-rules. case "$enable_silent_rules" in @@ -240,8 +241,7 @@ AC_DEFUN([gt_INTL_SUBDIR_CORE], dnl intl/plural.c is generated from intl/plural.y. It requires bison, dnl because plural.y uses bison specific features. It requires at least - dnl bison-1.26 because earlier versions generate a plural.c that doesn't - dnl compile. + dnl bison-2.7 for %define api.pure. dnl bison is only needed for the maintainer (who touches plural.y). But in dnl order to avoid separate Makefiles or --enable-maintainer-mode, we put dnl the rule in general Makefile. Now, some people carelessly touch the @@ -258,7 +258,7 @@ changequote(<<,>>)dnl ac_prog_version=`$INTLBISON --version 2>&1 | sed -n 's/^.*GNU Bison.* \([0-9]*\.[0-9.]*\).*$/\1/p'` case $ac_prog_version in '') ac_prog_version="v. ?.??, bad"; ac_verc_fail=yes;; - 1.2[6-9]* | 1.[3-9][0-9]* | [2-9].*) + 2.[7-9]* | [3-9].*) changequote([,])dnl ac_prog_version="$ac_prog_version, ok"; ac_verc_fail=no;; *) ac_prog_version="$ac_prog_version, bad"; ac_verc_fail=yes;; diff --git a/m4/po.m4 b/m4/po.m4 index bd210c5..84659ea 100644 --- a/m4/po.m4 +++ b/m4/po.m4 @@ -1,4 +1,4 @@ -# po.m4 serial 21 (gettext-0.18.3) +# po.m4 serial 22 (gettext-0.19) dnl Copyright (C) 1995-2014 Free Software Foundation, Inc. dnl This file is free software; the Free Software Foundation dnl gives unlimited permission to copy and/or distribute it, @@ -30,7 +30,7 @@ AC_DEFUN([AM_PO_SUBDIRS], dnl Release version of the gettext macros. This is used to ensure that dnl the gettext macros and po/Makefile.in.in are in sync. - AC_SUBST([GETTEXT_MACRO_VERSION], [0.18]) + AC_SUBST([GETTEXT_MACRO_VERSION], [0.19]) dnl Perform the following tests also if --disable-nls has been given, dnl because they are needed for "make dist" to work. -- 1.9.3 Regards, -- Daiki Ueno
Re: Fix libunistring in MS-Windows locales
k...@freefriends.org (Karl Berry) writes: > Daiki, would you like to become the new maintainer? If nobody else is interested, I can take it. > Alternatively, if you think libunistring as a separate entity should > go away and it just be part of gnulib, that's fine. Ditto libiconv > (another of Bruno's packages), which I suspect is in a similar > situation, on both counts. I think a separate project (at least a git repository) is still needed for standalone releases. Since those libraries include large data, providing them as a shared library can be more efficient than as part of gnulib. Regards, -- Daiki Ueno
Re: [PATCH] gettext: update macros to version 0.19
Pádraig Brady writes: > make[2]: Entering directory `/home/padraig/git/coreutils/po' > *** error: gettext infrastructure mismatch: using a Makefile.in.in from > gettext version 0.18 but the autoconf macros are from gettext version 0.19 > make[2]: *** [check-macro-version] Error 1 > > I can easily update gettext and can confirm 0.19 works as expected. > Other have often expressed disdain for non readily available > distro requiremnts like this. Should we consider reverting > this until gettext-0.19 packages are easily available? I feel it rather strange that bootstrap replaces m4/po.m4, already pulled by autopoint. As coreutils explicitly specifies the gettext infrastructure version as 'AM_GNU_GETTEXT_VERSION([0.18.1])' in configure.ac, IMO it should still be able to use the 0.18.1 macros even after running bootstrap. The file replacement happens because of some Gnulib tests depending on the gettext module. How about removing the dependencies? >From c6e5277472b17de2a98e5e5c903b36efab7a039b Mon Sep 17 00:00:00 2001 From: Daiki Ueno Date: Thu, 10 Jul 2014 07:24:04 +0900 Subject: [PATCH] tests: remove dependency on gettext macros MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Some tests pulls m4/po.m4, which can be newer than the one pulled by autopoint. This causes problem when building packages which request older gettext. Problem reported by Pádraig Brady in: http://lists.gnu.org/archive/html/bug-gnulib/2014-07/msg00023.html * modules/utimens-tests (Depends-on): Remove gettext. * modules/futimens-tests (Depends-on): Likewise. * modules/quotearg-tests (Depends-on): Depend on gettext-h, not gettext. --- ChangeLog | 8 modules/futimens-tests | 1 - modules/quotearg-tests | 2 +- modules/utimens-tests | 1 - 4 files changed, 9 insertions(+), 3 deletions(-) diff --git a/ChangeLog b/ChangeLog index 66fd9c8..e63ef73 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,11 @@ +2014-07-10 Daiki Ueno + + tests: remove dependency on gettext macros + * modules/utimens-tests (Depends-on): Remove gettext. + * modules/futimens-tests (Depends-on): Likewise. + * modules/quotearg-tests (Depends-on): Depend on gettext-h, not + gettext. + 2014-07-09 Pavel Hrdina (tiny change) nl_langinfo: fix build under mingw diff --git a/modules/futimens-tests b/modules/futimens-tests index a1948eb..28ff826 100644 --- a/modules/futimens-tests +++ b/modules/futimens-tests @@ -7,7 +7,6 @@ tests/signature.h tests/macros.h Depends-on: -gettext fcntl-h ignore-value nanosleep diff --git a/modules/quotearg-tests b/modules/quotearg-tests index 1af4eff..5e062a7 100644 --- a/modules/quotearg-tests +++ b/modules/quotearg-tests @@ -12,7 +12,7 @@ m4/locale-fr.m4 m4/codeset.m4 Depends-on: -gettext +gettext-h progname stdint setenv diff --git a/modules/utimens-tests b/modules/utimens-tests index 76dad1b..260c26c 100644 --- a/modules/utimens-tests +++ b/modules/utimens-tests @@ -9,7 +9,6 @@ tests/macros.h Depends-on: dup -gettext ignore-value nanosleep symlink -- 2.0.0 Regards, -- Daiki Ueno
Re: [PATCH] gettext: update macros to version 0.19
Pádraig Brady writes:
> I'm not sure that removing the gettext dependencies is OK because
> they're used to provide @LIB_INTL@ which was added to support solaris 2.[67]
> http://git.sv.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=41dca647
Thanks for the pointer, I missed it.
> they're used to provide @LIB_INTL@ which was added to support solaris 2.[67]
If this is the only use-case, perhaps we could separate out the
@LIBINTL@ check from the gettext module (possibly to the gettext-h
module) and let the tests depend on it?
I'm attaching a patch as a proof of concept.
>From 875d647f9bb0ffc8f8b507083cff737c56f3a9e8 Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Thu, 10 Jul 2014 13:35:57 +0900
Subject: [PATCH] gettext: move libintl check to gettext-h
The gettext module copies the entire gettext infrastructure,
including po/ directory handling, etc., while some modules only
need the @LIBINTL@ definition. This patch moves the
libintl-related stuff to the gettext-h module and make it
self-contained.
* m4/gettext.m4 (AM_GNU_GETTEXT): Separate out libintl check to..
(AM_GNU_GETTEXT_LIBINTL): ..here.
* modules/gettext (Files): Move m4/gettext.m4 to..
* modules/gettext-h (Files): ..here.
* modules/quotearg-tests (Depends-on): Depend on gettext-h, not
gettext.
* modules/utimens-tests (Depends-on): Likewise.
* modules/futimens-tests (Depends-on): Likewise.
* gnulib-tool (func_create_testdir): Look for po.m4, instead of
gettext.m4, to check if autopoint invocation is needed.
---
ChangeLog | 19 +++
gnulib-tool| 6 ++---
m4/gettext.m4 | 63 +-
modules/futimens-tests | 2 +-
modules/gettext| 1 -
modules/gettext-h | 4 ++--
modules/libintl| 23 ++
modules/quotearg-tests | 2 +-
modules/utimens-tests | 2 +-
9 files changed, 87 insertions(+), 35 deletions(-)
create mode 100644 modules/libintl
diff --git a/ChangeLog b/ChangeLog
index 66fd9c8..7e773c2 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,22 @@
+2014-07-10 Daiki Ueno
+
+ gettext: move libintl check to gettext-h
+ The gettext module copies the entire gettext infrastructure,
+ including po/ directory handling, etc., while some modules only
+ need the @LIBINTL@ definition. This patch moves the
+ libintl-related stuff to the gettext-h module and make it
+ self-contained.
+ * m4/gettext.m4 (AM_GNU_GETTEXT): Separate out libintl check to..
+ (AM_GNU_GETTEXT_LIBINTL): ..here.
+ * modules/gettext (Files): Move m4/gettext.m4 to..
+ * modules/gettext-h (Files): ..here.
+ * modules/quotearg-tests (Depends-on): Depend on gettext-h, not
+ gettext.
+ * modules/utimens-tests (Depends-on): Likewise.
+ * modules/futimens-tests (Depends-on): Likewise.
+ * gnulib-tool (func_create_testdir): Look for po.m4, instead of
+ gettext.m4, to check if autopoint invocation is needed.
+
2014-07-09 Pavel Hrdina (tiny change)
nl_langinfo: fix build under mingw
diff --git a/gnulib-tool b/gnulib-tool
index 7353920..da39cdc 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -5770,7 +5770,7 @@ func_create_testdir ()
subdirs="$sourcebase $m4base"
subdirs_with_configure_ac=""
- if false && test -f "$testdir"/$m4base/gettext.m4; then
+ if false && test -f "$testdir"/$m4base/po.m4; then
# Avoid stupid error message from automake:
# "AM_GNU_GETTEXT used but `po' not in SUBDIRS"
mkdir -p "$testdir/po"
@@ -6033,7 +6033,7 @@ func_create_testdir ()
(cd "$testdir"
# Do not use "${AUTORECONF} --force --install", because it may invoke
# autopoint, which brings in older versions of some of our .m4 files.
- if test -f $m4base/gettext.m4; then
+ if test -f $m4base/po.m4; then
func_execute_command ${AUTOPOINT} --force || func_exit 1
for f in $m4base/*.m4~; do
if test -f $f; then
@@ -6057,7 +6057,7 @@ func_create_testdir ()
(cd "$testdir/$testsbase" || func_exit 1
# Do not use "${AUTORECONF} --force --install", because it may invoke
# autopoint, which brings in older versions of some of our .m4 files.
- if test -f ../$m4base/gettext.m4; then
+ if test -f ../$m4base/po.m4; then
func_execute_command ${AUTOPOINT} --force || func_exit 1
for f in ../$m4base/*.m4~; do
if test -f $f; then
diff --git a/m4/gettext.m4 b/m4/gettext.m4
index be247bf..f06edca 100644
--- a/m4/gettext.m4
+++ b/m4/gettext.m4
@@ -1,4 +1,4 @@
-# gettext.m4 serial 66 (gettext-0.18.2)
+# gettext.m4 serial 67 (gettext-0.18.2)
dnl Copyright (C) 1995-2014 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
@@ -56,6 +56,42 @@ dnl maintainers.
dnl
AC_DEFUN([AM_GNU_GETTEXT],
[
+ dnl Check for libintl.
+ AM_GNU_GETTEXT_LIBINTL([$1],
Re: [PATCH] gettext: update macros to version 0.19
Philipp Thomas writes: > The number of packages that failed to build with 0.19.1 was too high > to be acceptable and the person who did the update refused to resolve > the problems with the the affected packages the update was rejected. I > just requested the update again so let's see whether it gets accepted. Could you provide an example of the build failures with 0.19.1? Yes, 0.19 had such problems[1], but with 0.19.1 we haven't received any single bug report that affects building other packages, except this: http://lists.gnu.org/archive/html/bug-gettext/2014-06/msg00022.html and this could be worked around somehow, though I'm planning 0.19.1.1 at some point soon. If there is any critical problem, please let us know. (For the record, all the affected Fedora 21 packages have been rebuilt with 0.19.1 already.) Thanks, Footnotes: [1] http://lists.gnu.org/archive/html/bug-gettext/2014-06/msg00021.html -- Daiki Ueno
Re: [PATCH] gettext: update macros to version 0.19
Bernhard Voelker writes: > On 07/11/2014 03:13 PM, Daiki Ueno wrote: >> Could you provide an example of the build failures with 0.19.1? > > https://build.opensuse.org/package/live_build_log/Base:System/coreutils/openSUSE_Factory/i586 > > [ 172s] *** Error in `/usr/bin/xgettext': realloc(): invalid pointer: > 0x098f4ea8 *** Thanks, it should be fixed with: http://git.savannah.gnu.org/cgit/gettext.git/commit/?h=maint&id=7b2d8d61 It's a shame that it wasn't found before the 0.19.1 release. Will make a new bug-fix release this weekend. Regards, -- Daiki Ueno
Re: [PATCH] gettext: update macros to version 0.19
Pádraig Brady writes: > Having realized that this update would break the continuous > integration server also I've decided to revert for now, > so that there is a less problematic sync point for the coreutil-8.23 release. > We can reinstate soon when the gettext 0.19 issues are sorted. Sorry for the inconvenience, fine with me. >> I tracked this down to autoconf 2.68 being used on this older system. >> The sequence is: >> >> ./bootstrap >> if gettext --version == 0.18.3 >> apply autopoint m4 include workaround so compat with 2.68 >> else >> rely on autopoint avoiding m4 include errors >> This autopoint code though relies on autoconf >= 2.69 >> fi I've pushed a fix for this anyway: http://git.savannah.gnu.org/cgit/gettext.git/commit/?h=maint&id=e1bd11a1 Regards, -- Daiki Ueno
Re: [PATCH] announce-gen: avoid failure when Digest::SHA is installed
Hi, Daiki Ueno writes: > The 'announce-gen' script fails when Digest::SHA (not Digest::SHA1) is > installed: > > Can't locate object method "new" via package "Digest::SHA1" (perhaps you > forgot to load "Digest::SHA1"?) at ../gnulib/build-aux/announce-gen line 165. > zsh: exit 25./gnulib/build-aux/announce-gen --bootstrap-tools=automake I've just noticed that this is still the case on my Fedora installation. Could anyone take a look at the patch? https://lists.gnu.org/archive/html/bug-gnulib/2013-07/msg00014.html Or perhaps --no-print-checksums could be the default, in favor or GPG sig? Regards, -- Daiki Ueno
Re: [PATCH] gettext: update macros to version 0.19
Philipp Thomas writes:
>> I've pushed a fix for this anyway:
>> http://git.savannah.gnu.org/cgit/gettext.git/commit/?h=maint&id=e1bd11a1
>
> Seems the fix doesn't adapt the testsuite as I get failures in the autopoint
> tests with the patch applied:
>
> FAIL: autopoint-2
> FAIL: autopoint-3
Yes, the patch was incomplete and here is the amendment:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=db326c5a
I forgot that AM_GNU_GETTEXT{,_VERSION} cannot be captured by autoconf
trace. 0.19.2 released today shouldn't have the issue.
Regards,
--
Daiki Ueno
fallback _GL_ATTRIBUTE_* definitions in intl.m4
Hi, There are growing number of uses of _GL_ATTRIBUTE_* in Gnulib source files, which are also shared with libintl. Since libintl can be copied into non-Gnulib projects, we have to add fallback definitions to the libintl source. However, this might cause divergence between Gnulib and gettext in the future. So, in the gettext git, I added a new M4 macro gt_GL_ATTRIBUTE, which copies necessary C macros from gnulib-common.m4: AC_DEFUN([AM_INTL_SUBDIR], [... AC_REQUIRE([gt_GL_ATTRIBUTE])dnl ]) AC_DEFUN([gt_GL_ATTRIBUTE], [ m4_ifndef([gl_[]COMMON], AH_VERBATIM([gt_gl_attribute], [#ifndef _GL_ATTRIBUTE_PURE #define _GL_ATTRIBUTE_PURE ... Actual commit in gettext: http://git.savannah.gnu.org/cgit/gettext.git/commit/?h=maint&id=0f8e2cca Does this look OK? I'd like someone to double-check if it does not break anything. Or, are there any better ideas? Regards, -- Daiki Ueno
Re: Fix libunistring in MS-Windows locales
Eli Zaretskii writes: >> Date: Tue, 15 Jul 2014 12:20:44 -0700 >> From: Paul Eggert >> >> Thanks, I installed that, with minor changes to indentation and suchlike. > > Thank you. Sorry for not looking at it timely. Do you need a new libunistring release with this change? I don't even know if Guile is still using the packaged libunistring. By the way, isn't there a minor race in gl_locale_name_thread, since it depends on a global variable found_lcid? Regards, -- Daiki Ueno
Re: Fix libunistring in MS-Windows locales
Eli Zaretskii writes: >> By the way, isn't there a minor race in gl_locale_name_thread, since it >> depends on a global variable found_lcid? > > Yes. (It is not a problem for Guile, since Guile built with threads > on Windows is severely broken, so the only way of having a useful > Guile on Windows is to build it without threads.) Suggestions for > fixing that are welcome. Maybe just use the '__thread' qualifier for > that variable? I'm not familiar with Windows TLS support, but '__thread' seems to be compiler specific. Gnulib has glthread/tls but I'm not sure if it is worth adding another dependency to localename.c (which is also shared with libintl...) for this specific issue. Maybe just lock around the use? Regards, -- Daiki Ueno
Re: Fix libunistring in MS-Windows locales
Eli Zaretskii writes: >> I'm not familiar with Windows TLS support, but '__thread' seems to be >> compiler specific. > > Do we care for any Windows compilers except GCC? I was rather worried about particular versions of GCC, which might not support it at all (< 3.3?), or might require extra compiler options or runtime dependency: http://sourceforge.net/p/mingw/bugs/1333/ http://sourceforge.net/p/mingw-w64/mailman/message/31213279/ I could be wrong and perhaps it might not be a problem with the recent MinGW releases. Regards, -- Daiki Ueno
Re: Fix libunistring in MS-Windows locales
Daiki Ueno writes:
> I could be wrong and perhaps it might not be a problem with the recent
> MinGW releases.
FWIW I just tried myself (on wine, though) and it seems to work with:
- GCC 4.8.3 (Fedora mingw64-gcc package, cross compiling)
- GCC 4.7.2 (mingw-w32-bin_i686-mingw_20111219, self compiling)
So, I guess it is OK to use __thread, though other 3 static variables
(lname, last_lcid, and last_locale) would also need the same care.
For the archive, I'm attaching a test program:
#include
#include
#include
#include
#include
#include
static void
func (LPVOID p)
{
char *locale = (char *) p;
_configthreadlocale (_ENABLE_PER_THREAD_LOCALE);
setlocale (LC_CTYPE, locale);
while (1)
{
const char *actual = gl_locale_name_thread (LC_CTYPE, "LC_CTYPE");
if (actual == NULL || strcmp (actual, locale) != 0)
printf ("%s != %s\n", actual, locale);
}
}
int
main (void)
{
HANDLE threads[2];
threads[0] = (HANDLE) _beginthread (func, 0, "de_DE");
threads[1] = (HANDLE) _beginthread (func, 0, "ja_JP");
WaitForMultipleObjects (2, threads, TRUE, INFINITE);
CloseHandle (threads[0]);
CloseHandle (threads[1]);
return 0;
}
Regards,
--
Daiki Ueno
Re: Fix libunistring in MS-Windows locales
Eli Zaretskii writes: > . __thread works as expected in MinGW GCC since version 4.5.0 > > . DLLs that use __thread (a.k.a. "implicit TLS") cannot be safely > loaded at run time, using LoadLibrary, on Windows versions before > Vista, they can only be loaded at program startup time (IOW, the > corresponding -lLIBRARY switch should be passed on the link > command line) > > . DLLs that need to be able to support LoadLibrary on Windows XP and > older need to use explicit TLS API functions, which requires a > call to TlsAlloc in the DllMain function, another complication Thanks for checking in detail. Then it would make more sense to use locking rather than TLS. > Actually, I see that libunistring already includes functions for > locking (which on Windows wrap the 2 APIs I mentioned above). So I > guess it's best to use them. Yes, could you create a patch in that direction? By the way, though a minor nitpicking, the indentation in gl_locale_name_thread still doesn't look correct (there is a variable declaration in the middle of block). Regards, -- Daiki Ueno
Re: Missing symbols when compiling gettext on OSX 10.8
Paul Eggert writes: > Thanks for reporting this. Since your patch is against the gettext > version of install-reloc, I am CC'ing this to bug-gnu-gettext. I > don't know why the two versions of install-reloc have diverged -- > perhaps it's time to sync gettext back into gnulib? But the problem > should be fixed in the upstream version anyway. As far as I checked, gettext uses build-aux/install-reloc from Gnulib as it is, and the patch still applies to the Gnulib master. Regards, -- Daiki Ueno
Re: Fix libunistring in MS-Windows locales
Eli Zaretskii writes:
>> > Actually, I see that libunistring already includes functions for
>> > locking (which on Windows wrap the 2 APIs I mentioned above). So I
>> > guess it's best to use them.
>>
>> Yes, could you create a patch in that direction?
>
> Will do.
Here is a simple patch for this. Is it OK for you?
>From a9576350604e97fc3c3fa16778e6e3d10094f97a Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Thu, 7 Aug 2014 08:46:52 +0900
Subject: [PATCH] localename: make gl_locale_name_thread really thread-safe on
Windows
* lib/localename.c [WINDOWS_NATIVE && !IN_LIBINTL]: Include
"glthread/lock.h".
(get_lcid_lock) [WINDOWS_NATIVE]: New variable.
(get_lcid) [WINDOWS_NATIVE]: Lock while looking for an LCID.
---
ChangeLog| 8
lib/localename.c | 15 ++-
2 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/ChangeLog b/ChangeLog
index 8e74a54..99b0dec 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2014-08-07 Daiki Ueno
+
+ localename: make gl_locale_name_thread really thread-safe on Windows
+ * lib/localename.c [WINDOWS_NATIVE && !IN_LIBINTL]: Include
+ "glthread/lock.h".
+ (get_lcid_lock) [WINDOWS_NATIVE]: New variable.
+ (get_lcid) [WINDOWS_NATIVE]: Lock while looking for an LCID.
+
2014-08-05 Paul Eggert
sys_select: fix FD_ZERO problem on Solaris 10
diff --git a/lib/localename.c b/lib/localename.c
index 4cce060..78dc344 100644
--- a/lib/localename.c
+++ b/lib/localename.c
@@ -55,6 +55,9 @@
#if defined _WIN32 || defined __WIN32__
# define WINDOWS_NATIVE
+# if !defined IN_LIBINTL
+# include "glthread/lock.h"
+# endif
#endif
#if defined WINDOWS_NATIVE || defined __CYGWIN__ /* Native Windows or Cygwin */
@@ -2542,6 +2545,9 @@ enum_locales_fn (LPTSTR locale_num_str)
return TRUE;
}
+/* This lock protects the get_lcid against multiple simultaneous calls. */
+gl_lock_define_initialized(static, get_lcid_lock)
+
/* Return the Locale ID (LCID) number given the locale's name, a
string, in LOCALE_NAME. This works by enumerating all the locales
supported by the system, until we find one whose name matches
@@ -2553,8 +2559,14 @@ get_lcid (const char *locale_name)
static LCID last_lcid;
static char last_locale[1000];
+ /* Lock while looking for an LCID, to protect access to static
+ variables: last_lcid, last_locale, found_lcid, and lname. */
+ gl_lock_lock (get_lcid_lock);
if (last_lcid > 0 && strcmp (locale_name, last_locale) == 0)
-return last_lcid;
+{
+ gl_lock_unlock (get_lcid_lock);
+ return last_lcid;
+}
strncpy (lname, locale_name, sizeof (lname) - 1);
lname[sizeof (lname) - 1] = '\0';
found_lcid = 0;
@@ -2564,6 +2576,7 @@ get_lcid (const char *locale_name)
last_lcid = found_lcid;
strcpy (last_locale, locale_name);
}
+ gl_lock_unlock (get_lcid_lock);
return found_lcid;
}
--
1.9.3
Regards,
--
Daiki Ueno
Re: Missing symbols when compiling gettext on OSX 10.8
Paul Eggert writes: > Daiki Ueno wrote: >> As far as I checked, gettext uses build-aux/install-reloc from Gnulib as >> it is, and the patch still applies to the Gnulib master. For the record, porting it to Gnulib was completed in 2007: http://git.sv.gnu.org/cgit/gettext.git/commit/?id=acc93e34 > I'm a bit confused, since the two files seem to differ, but oh well. I guess you are referring to an older release of gettext. There was a commit to build-aux/install-reloc (in Gnulib) in last October, which 0.18.3.2 missed to include. With 0.19 or later, those files are identical. > I installed the patch to the Gnulib master. Thanks, Alessandro, for > reporting it. Thanks. Regards, -- Daiki Ueno
Re: [bug-libunistring] [PATCH v4] unistr: New modules for backward iteration in string.
Ben Pfaff writes:
> New module 'unistr/u8-mb-prev-uc'.
> New module 'unistr/u16-mb-prev-uc'.
> New module 'unistr/u32-mb-prev-uc'.
Thanks, looks good to me. Some nit-picking below.
* _GL_UNUSED_PARAMETER of u32_mb_prev_uc seems to be a leftover, as the
argument is actually used in the implementation.
* The largest value of possible leading octet tested in
test-u8-mb-prev-uc.c:exhaustive_test is 0xf5, while there are checks
against 0xf8 in u8-mb-prev-uc.c. Also code units above the surrogate
code-point are not checked in u16 and u32 tests.
* The license of u{8,16,32}-mbtouc changed to LGPLv2+ some
time ago, maybe good to follow the change?
> v3->v4: Changed the code to always be "safe". It looks to me like the
> "unsafe" version that I had written originally reflected a misunderstanding
> of how the gnulib option for that was supposed to work.
Are you going to add "unsafe" version later, or is it not useful at all?
Thanks,
--
Daiki Ueno
Re: [bug-libunistring] [PATCH v4] unistr: New modules for backward iteration in string.
Ben Pfaff writes:
>>> Are you going to add "unsafe" version later, or is it not useful at all?
>>
>> I do not have a use myself for unsafe versions, but I will add them if
>> you think they are a good idea. Let me know.
>
> What is your opinion on this issue?
Oh, sorry, it wasn't my intention to block the patch getting in. Just
wondered if there is any expected flaw in the "unsafe" version (e.g. it
would be a duplicate of u{8,16,32}_prev?). If the lack of use-case is
the only reason, it should be fine to skip it for now.
mbrtowc test fails on Solaris 11
Hello, On Solaris 11, a couple of mbrtowc tests fails: FAIL: test-mbrtowc3.sh == test-mbrtowc.c:116: assertion 'wc == (wchar_t) 0xBADFACE' failed FAIL: test-mbrtowc4.sh == test-mbrtowc.c:303: assertion 'ret == 1' failed The former case checks if the WC value is unchanged around the call: memset (&state, 0, sizeof (mbstate_t)); mbrtowc (&wc, NULL, 7, &state); As the Solaris manual page[1] states that PWC and N are ignored if S is NULL, it seems like a bug in the system mbrtowc. The latter case checks if the return value of mbrtowc, when scanning a string starting with "\271\201", under the zh_CN.GB18030 locale. char input[] = "B\250\271\201\060\211\070er"; memset (&state, '\0', sizeof (mbstate_t)); wc = (wchar_t) 0xBADFACE; ret = mbrtowc (&wc, input + 2, 7, &state); ASSERT (ret == 1); However, "\271\201" is a valid GB18030 sequence which encodes U+7B92. Maybe it should expect ret == 2? Footnotes: [1] http://docs.oracle.com/cd/E36784_01/html/E36874/mbrtowc-3c.html Regards, -- Daiki Ueno
Re: GNU gettext 0.18.3.1 fails with undefined _Bool on Solaris
I'm very sorry for the long delay. Paul Eggert writes: > The _Bool thing is due to a problem in gettext's configure.ac files. > gettext-tools/configure.ac invokes gtpo_EARLY, which invokes > AM_PROG_CC_C_O, which means gettext-tools is configured with > '-D_STDC_C99=', so stdbool.h works; whereas gettext-runtime/configure.ac > does not do that, so gettext-runtime is not configured with '-D_STDC_C99=', > so stdbool.h does not work for it and it supplies a substitute, which > is incompatible with the system stdbool.h used by gettext-tools. Thanks for tracking it down. I've applied the patch and confirmed it effectively fixes the problem. Regards, -- Daiki ueno
Re: mbrtowc test fails on Solaris 11
Daiki Ueno writes: > The latter case checks if the return value of mbrtowc, when scanning a > string starting with "\271\201", under the zh_CN.GB18030 locale. > > char input[] = "B\250\271\201\060\211\070er"; > memset (&state, '\0', sizeof (mbstate_t)); > wc = (wchar_t) 0xBADFACE; > ret = mbrtowc (&wc, input + 2, 7, &state); > ASSERT (ret == 1); > > However, "\271\201" is a valid GB18030 sequence which encodes U+7B92. > Maybe it should expect ret == 2? No, sorry, the above snippet was wrong. In the original test code, there is no reset of STATE before the mbrtowc call and it is already shifted. So, Solaris 11's mbrtowc seems broken here as well.
Re: [bug #43404] gl_locale_name_default() thread issues on OS X
Hello, Forwarding to the mailing lists to get more feedback; the original report can be found at <http://savannah.gnu.org/bugs/?43404>. (I've changed the tracker configuration so that all reports will be sent to the mailing list hereafter.) Peter Eisentraut writes: > Something in libintl calls setlocale(), which calls gl_locale_name_default(), > which calls CFLocaleCopyCurrent() on OS X. Our analysis shows that > CFLocaleCopyCurrent() spins up an additional thread, which causes all kinds of > chaos in signal handlers and forked processes. Here is the full analysis: > http://www.postgresql.org/message-id/20140915045114.ga1332...@tornado.leadboat.com > > Clearly, this is a somewhat special application, compared to all the > single-threaded, single-process programs out there, but if adding gettext > support to a server causes it to crash in random ways, that doesn't seem > nice. > > The original change in gettext/libintl is > d4fd77b3f57a386b63083a7246ae5808510ac072. > > Later in the above thread, a patch is proposed to move the Core Foundation > calls into a subprocess, which you might be interested in. That sounds like an interesting idea. Perhaps gl_locale_name_default could use the pipe/fork trick around the Core Foundation calls, for single-threaded consumer (using pthread_is_threaded_np). Is this kind of change acceptable in Gnulib? By the way, the issue reminds me of: http://lists.gnu.org/archive/html/bug-gettext/2013-11/msg00020.html though it is a different (multi-threaded consumer) case. Thanks, -- Daiki Ueno
Re: [bug #43404] gl_locale_name_default() thread issues on OS X
Pádraig Brady writes:
> There are many consequences to having multiple threads,
> so it's worth avoiding/isolating that where possible.
>
> You mentioned it would be conditionalized on pthread_is_threaded_np
> for performance I suppose, as threaded progs would already
> have to deal with the consequences of a separate thread.
> So races in pthread_is_threaded_np would only be a small
> performance issue.
>
> Seems like a good idea to me.
Thanks for the comment. Here is a tentative patch based on the
PostgreSQL patches. Perhaps it might require a copyright assignment
from the original author (Cc'ed).
>From d8e9c96ef0c1e1c1c410cfbaec547c2e3d442dbe Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Tue, 14 Oct 2014 12:47:06 +0900
Subject: [PATCH] localename: Avoid implicit thread creation in CoreFoundation
Problem reported by Peter Eisentraut at:
http://savannah.gnu.org/bugs/?43404.
Since some CoreFoundation functions (CFLocaleCopyCurrent for instance)
create a new thread internally, it would be better to move the calls
into a subprocess, when the caller is single-threaded.
Based on the patch created by Noah Misch:
http://www.postgresql.org/message-id/20141011002433.ga15...@tornado.leadboat.com
* tests/test-localename.c (test_locale_name_default) [__APPLE__]:
Check the return value of pthread_is_threaded_np before and after the
gl_locale_name_default.
* m4/localename.m4 (gl_LOCALENAME): Check pthread_is_threaded_np.
* lib/localename.c [__APPLE__]: Include , , and
if pthread_is_threaded_np is available.
(gl_locale_name_default_from_CoreFoundation) [__APPLE__]: New
function, split off from gl_locale_name_default.
(gl_locale_name_default_from_CoreFoundation_forked) [__APPLE__]: New
function.
(gl_locale_name_default) [__APPLE__]: Call _from_CoreFoundation or
_from_CoreFoundation_forked depending on whether or not the caller is
multi-threaded.
---
lib/localename.c| 161
m4/localename.m4| 2 +-
tests/test-localename.c | 16 -
3 files changed, 151 insertions(+), 28 deletions(-)
diff --git a/lib/localename.c b/lib/localename.c
index 78dc344..745bc92 100644
--- a/lib/localename.c
+++ b/lib/localename.c
@@ -51,6 +51,12 @@
# elif HAVE_CFPREFERENCESCOPYAPPVALUE
# include
# endif
+# if HAVE_PTHREAD_IS_THREADED_NP
+# define _DARWIN_C_SOURCE 1
+# include
+# include
+# include
+# endif
#endif
#if defined _WIN32 || defined __WIN32__
@@ -2836,6 +2842,122 @@ gl_locale_name_environ (int category, const char *categoryname)
return NULL;
}
+#if HAVE_CFLOCALECOPYCURRENT || HAVE_CFPREFERENCESCOPYAPPVALUE
+static char *
+gl_locale_name_default_from_CoreFoundation (void)
+{
+ char namebuf[256];
+ char *localename = "C";
+# if HAVE_CFLOCALECOPYCURRENT /* Mac OS X 10.3 or newer */
+ CFLocaleRef locale = CFLocaleCopyCurrent ();
+ CFStringRef name = CFLocaleGetIdentifier (locale);
+
+ if (CFStringGetCString (name, namebuf, sizeof (namebuf),
+ kCFStringEncodingASCII))
+{
+ gl_locale_name_canonicalize (namebuf);
+ localename = namebuf;
+}
+ CFRelease (locale);
+# elif HAVE_CFPREFERENCESCOPYAPPVALUE /* Mac OS X 10.2 or newer */
+ CFTypeRef value =
+CFPreferencesCopyAppValue (CFSTR ("AppleLocale"),
+ kCFPreferencesCurrentApplication);
+ if (value != NULL
+ && CFGetTypeID (value) == CFStringGetTypeID ()
+ && CFStringGetCString ((CFStringRef)value,
+ namebuf, sizeof (namebuf),
+ kCFStringEncodingASCII))
+{
+ gl_locale_name_canonicalize (namebuf);
+ localename = namebuf;
+}
+# endif
+ return strdup (localename);
+}
+
+# if HAVE_PTHREAD_IS_THREADED_NP
+static char *
+gl_locale_name_default_from_CoreFoundation_forked (void)
+{
+ char namebuf[256];
+ char *localename = "C";
+ int fd[2];
+ sigset_t sigs, old_sigs;
+ pid_t pid;
+ int child_status;
+ ssize_t nread;
+
+ /* Block SIGCHLD so we can get an exit status. */
+ sigemptyset (&sigs);
+ sigaddset (&sigs, SIGCHLD);
+ sigprocmask (SIG_BLOCK, &sigs, &old_sigs);
+
+ if (pipe (fd) < 0)
+goto done;
+
+ pid = fork ();
+ if (pid < 0)
+{
+ close (fd[0]);
+ close (fd[1]);
+ goto done;
+}
+
+ if (pid == 0)
+{
+ char *locname = gl_locale_name_default_from_CoreFoundation ();
+ size_t locname_len = strlen (locname);
+ int status = EXIT_SUCCESS;
+
+ if (close (fd[0]) < 0)
+status = EXIT_FAILURE;
+
+ if (write (fd[1], locname, locname_len) < locname_len)
+status = EXIT_FAILURE;
+
+ if (close (fd[1]) < 0)
+status = EXIT_FAILURE;
+
+ free (locname);
+ _exit (status);
+}
+
+ if (close (fd[1]) < 0)
+{
+ close (fd[0]);
+ goto done;
+}
+
+ if (waitpid (pid, &child_status, 0) != pid
+ || !(WIFEXITED (child_status)
+ && WEXITSTATUS (child_status) == EXIT_S
Re: [bug #43404] gl_locale_name_default() thread issues on OS X
Pádraig Brady writes: >> Thanks for the comment. Here is a tentative patch based on the >> PostgreSQL patches. Perhaps it might require a copyright assignment >> from the original author (Cc'ed). > > Is AC_CHECK_FUNCS sufficient to find pthread_is_threaded_np()? > I.E. is LIB_PTHREAD required to find this function on darwin? Good point. It is detected on 10.8 without -lpthread, but maybe the link flag is needed on older versions of Mac OS X. I'll fix it in the next version. > It's tempting to simplify actually and unconditionally call > the forked version, as this isn't a function that would > be in a hot path. I agree, since this function is mainly called from libintl_setlocale, which is called at very early stage in typical programs, and probably we can assume that there is only one thread at that point. Regards, -- Daiki Ueno
Re: libiconv detection failure
Hello,
Werner LEMBERG writes:
> Eli reported a failure in the detection code of iconv on MinGW while
> building groff:
>
> conftest.cpp:62:51: error: invalid conversion from 'char**' to 'const
> char**' [-fpermissive]
> In file included from conftest.cpp:43:0:
> d:\usr\bin\../lib/gcc/mingw32/4.7.2/../../../../include/iconv.h:83:15:
> error: initializing argument 2 of 'size_t libiconv(libiconv_t, const char**,
> size_t*, char**, size_t*)' [-fpermissive]
[...]
> It seems to me that the problem is a bug in gnulib's `iconv.m4'
> serial 18 file: Looking into this file I see that the body of
> `AM_ICONV' calls the macro `AM_ICONV_LINK' before the test that checks
> whether `const' is needed. Obviously, it should be vice versa.
Given that the intention of the above test is to check if the iconv
function works properly, maybe a workaround would be to use the fixed
prototype of iconv (as attached)? I'm still not able to reproduce the
original error on my MinGW environment, though.
Regards,
--
Daiki Ueno
>From 39e7957f1f724ae39524a82c230441df2557686c Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Tue, 21 Oct 2014 13:01:10 +0900
Subject: [PATCH] iconv: avoid false link error with g++ and libiconv
Reported by Eli Zaretskii and Werner LEMBERG in:
<https://lists.gnu.org/archive/html/bug-gnulib/2014-10/msg00023.html>.
* m4/iconv.m4 (AM_ICONV_LINK): Use the fixed prototype of the
iconv function while checking if iconv works properly.
---
ChangeLog | 8
m4/iconv.m4 | 14 +-
2 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/ChangeLog b/ChangeLog
index 23234b7..852005c 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2014-10-21 Daiki Ueno
+
+ iconv: avoid false link error with g++ and libiconv
+ Reported by Eli Zaretskii and Werner LEMBERG in:
+ <https://lists.gnu.org/archive/html/bug-gnulib/2014-10/msg00023.html>.
+ * m4/iconv.m4 (AM_ICONV_LINK): Use the fixed prototype of the
+ iconv function while checking if iconv works properly.
+
2014-10-18 Paul Eggert
symlinkat: port to AIX 7.1
diff --git a/m4/iconv.m4 b/m4/iconv.m4
index 4b29c5f..1aa41f3 100644
--- a/m4/iconv.m4
+++ b/m4/iconv.m4
@@ -1,4 +1,4 @@
-# iconv.m4 serial 18 (gettext-0.18.2)
+# iconv.m4 serial 19 (gettext-0.18.2)
dnl Copyright (C) 2000-2002, 2007-2014 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
@@ -76,6 +76,18 @@ AC_DEFUN([AM_ICONV_LINK],
[AC_LANG_SOURCE([[
#include
#include
+
+/* We are not really interested in whether INPTR and OUTPTR need
+ 'const' qualifier here. Use the fixed prototype to avoid false
+ link error when compiled with -fpermissive or g++. */
+#undef iconv
+extern
+#ifdef __cplusplus
+"C"
+#endif
+size_t iconv (iconv_t cd, char * *inbuf, size_t *inbytesleft,
+ char * *outbuf, size_t *outbytesleft);
+
int main ()
{
int result = 0;
--
1.9.3
Re: libiconv detection failure
Daiki Ueno writes: > Given that the intention of the above test is to check if the iconv > function works properly, maybe a workaround would be to use the fixed > prototype of iconv (as attached)? Sorry, that was obviously wrong. The other idea is to suppress the error with a pragma, but I guess it would be better to actually check the prototype in AM_ICONV_LINK. Regards, -- Daiki Ueno
Re: libiconv detection failure
Werner LEMBERG writes:
>>> Given that the intention of the above test is to check if the iconv
>>> function works properly, maybe a workaround would be to use the
>>> fixed prototype of iconv (as attached)?
>>
>> Sorry, that was obviously wrong. The other idea is to suppress the
>> error with a pragma, but I guess it would be better to actually
>> check the prototype in AM_ICONV_LINK.
>
> What about using m4_foreach, running the link tests with and without
> `const'?
Thanks for the tip. The attached patch seems to work with both cases.
Regards,
--
Daiki Ueno
>From fb53151f8c9456f186b2f192c02bd45e85c35476 Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Tue, 21 Oct 2014 18:07:00 +0900
Subject: [PATCH] iconv: avoid false detection of non-working iconv
The INBUF/OUTBUF arguments of iconv can be either 'const char **'
or 'char **'. If CC is g++, the difference causes a compile error
and thus leads to a false detection of non-working iconv.
Reported by Eli Zaretskii and Werner LEMBERG in:
<https://lists.gnu.org/archive/html/bug-gnulib/2014-10/msg00023.html>.
* m4/iconv.m4 (AM_ICONV_LINK): Try all possible argument types of
iconv. Bump serial number.
---
ChangeLog | 11 +++
m4/iconv.m4 | 52 ++--
2 files changed, 37 insertions(+), 26 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 23234b7..49639fd 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2014-10-21 Daiki Ueno
+
+ iconv: avoid false detection of non-working iconv
+ The INBUF/OUTBUF arguments of iconv can be either 'const char **'
+ or 'char **'. If CC is g++, the difference causes a compile error
+ and thus leads to a false detection of non-working iconv.
+ Reported by Eli Zaretskii and Werner LEMBERG in:
+ <https://lists.gnu.org/archive/html/bug-gnulib/2014-10/msg00023.html>.
+ * m4/iconv.m4 (AM_ICONV_LINK): Try all possible argument types of
+ iconv. Bump serial number.
+
2014-10-18 Paul Eggert
symlinkat: port to AIX 7.1
diff --git a/m4/iconv.m4 b/m4/iconv.m4
index 4b29c5f..9726a33 100644
--- a/m4/iconv.m4
+++ b/m4/iconv.m4
@@ -1,4 +1,4 @@
-# iconv.m4 serial 18 (gettext-0.18.2)
+# iconv.m4 serial 19 (gettext-0.18.2)
dnl Copyright (C) 2000-2002, 2007-2014 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
@@ -72,13 +72,16 @@ AC_DEFUN([AM_ICONV_LINK],
if test $am_cv_lib_iconv = yes; then
LIBS="$LIBS $LIBICONV"
fi
- AC_RUN_IFELSE(
-[AC_LANG_SOURCE([[
+ am_cv_func_iconv_works=no
+ m4_foreach([gl_iconv_buf_t], [[char **], [const char **]], [
+AC_RUN_IFELSE(
+ [AC_LANG_PROGRAM(
+ [[
#include
#include
-int main ()
-{
- int result = 0;
+
+typedef] gl_iconv_buf_t [gl_iconv_buf_t;]],
+ [[int result = 0;
/* Test against AIX 5.1 bug: Failures are not distinguishable from successful
returns. */
{
@@ -92,8 +95,8 @@ int main ()
char *outptr = buf;
size_t outbytesleft = sizeof (buf);
size_t res = iconv (cd_utf8_to_88591,
-(char **) &inptr, &inbytesleft,
-&outptr, &outbytesleft);
+(gl_iconv_buf_t) &inptr, &inbytesleft,
+(gl_iconv_buf_t) &outptr, &outbytesleft);
if (res == 0)
result |= 1;
iconv_close (cd_utf8_to_88591);
@@ -112,8 +115,8 @@ int main ()
char *outptr = buf;
size_t outbytesleft = sizeof (buf);
size_t res = iconv (cd_ascii_to_88591,
-(char **) &inptr, &inbytesleft,
-&outptr, &outbytesleft);
+(gl_iconv_buf_t) &inptr, &inbytesleft,
+(gl_iconv_buf_t) &outptr, &outbytesleft);
if (res == 0)
result |= 2;
iconv_close (cd_ascii_to_88591);
@@ -131,8 +134,8 @@ int main ()
char *outptr = buf;
size_t outbytesleft = 1;
size_t res = iconv (cd_88591_to_utf8,
-(char **) &inptr, &inbytesleft,
-&outptr, &outbytesleft);
+(gl_iconv_buf_t) &inptr, &inbytesleft,
+(gl_iconv_buf_t) &outptr, &outbytesleft);
if (res != (size_t)(-1) || outptr - buf > 1 || buf[1] != (char)0xAD)
result |= 4;
iconv_close (cd_88591_to_utf8);
@@ -151,8 +154,8 @@ int main ()
char *outptr = buf;
size_t outbytesleft = sizeof (buf);
size_t res = iconv (cd_88591_to_utf8,
-(char **) &inptr, &inbytesleft,
-
Re: libiconv detection failure
Eric Blake writes:
> m4_foreach causes the loop to be expanded at m4 time (a larger configure
> file). In this case, you could probably easily get by with a shell loop
> instead, for a smaller configure file (untested):
>
>for type in 'char **' 'const char **'; do
> AC_RUN_IFELSE([AC_LANG_PROGRAM([[
>...
>typedef $type gl_iconv_buf_t;
Yes, thanks for the suggestion. I've ended up with the attached patch,
with some more cleanup.
Eli: I think it is sufficient to compare the configure output of
before/after the change, using test packages:
Before: http://du-a.org/~ueno/junk/test-iconv-0.tar.gz
After: http://du-a.org/~ueno/junk/test-iconv-1.tar.gz
Thanks,
--
Daiki Ueno
>From 46969a8560460084f6770f3c21025f0faac33f9f Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Tue, 21 Oct 2014 18:07:00 +0900
Subject: [PATCH] iconv: avoid false detection of non-working iconv
The INBUF/OUTBUF arguments of iconv can be either 'const char **'
or 'char **'. If CC is g++, the difference causes a compile error
and thus leads to a false detection of non-working iconv.
Reported by Eli Zaretskii and Werner LEMBERG in:
<https://lists.gnu.org/archive/html/bug-gnulib/2014-10/msg00023.html>.
* m4/iconv.m4 (AM_ICONV_LINK): Try all possible argument types of
iconv. Bump serial number.
---
ChangeLog | 11 +++
m4/iconv.m4 | 51 ++-
2 files changed, 37 insertions(+), 25 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 23234b7..49639fd 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2014-10-21 Daiki Ueno
+
+ iconv: avoid false detection of non-working iconv
+ The INBUF/OUTBUF arguments of iconv can be either 'const char **'
+ or 'char **'. If CC is g++, the difference causes a compile error
+ and thus leads to a false detection of non-working iconv.
+ Reported by Eli Zaretskii and Werner LEMBERG in:
+ <https://lists.gnu.org/archive/html/bug-gnulib/2014-10/msg00023.html>.
+ * m4/iconv.m4 (AM_ICONV_LINK): Try all possible argument types of
+ iconv. Bump serial number.
+
2014-10-18 Paul Eggert
symlinkat: port to AIX 7.1
diff --git a/m4/iconv.m4 b/m4/iconv.m4
index 4b29c5f..2c3085f 100644
--- a/m4/iconv.m4
+++ b/m4/iconv.m4
@@ -1,4 +1,4 @@
-# iconv.m4 serial 18 (gettext-0.18.2)
+# iconv.m4 serial 19 (gettext-0.18.2)
dnl Copyright (C) 2000-2002, 2007-2014 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
@@ -72,13 +72,17 @@ AC_DEFUN([AM_ICONV_LINK],
if test $am_cv_lib_iconv = yes; then
LIBS="$LIBS $LIBICONV"
fi
- AC_RUN_IFELSE(
-[AC_LANG_SOURCE([[
+ am_cv_func_iconv_works=no
+ for ac_type in 'char **' 'const char **'; do
+AC_RUN_IFELSE(
+ [AC_LANG_PROGRAM(
+ [[
#include
#include
-int main ()
-{
- int result = 0;
+
+typedef $ac_type gl_iconv_buf_t;
+ ]],
+ [[int result = 0;
/* Test against AIX 5.1 bug: Failures are not distinguishable from successful
returns. */
{
@@ -92,8 +96,8 @@ int main ()
char *outptr = buf;
size_t outbytesleft = sizeof (buf);
size_t res = iconv (cd_utf8_to_88591,
-(char **) &inptr, &inbytesleft,
-&outptr, &outbytesleft);
+(gl_iconv_buf_t) &inptr, &inbytesleft,
+(gl_iconv_buf_t) &outptr, &outbytesleft);
if (res == 0)
result |= 1;
iconv_close (cd_utf8_to_88591);
@@ -112,8 +116,8 @@ int main ()
char *outptr = buf;
size_t outbytesleft = sizeof (buf);
size_t res = iconv (cd_ascii_to_88591,
-(char **) &inptr, &inbytesleft,
-&outptr, &outbytesleft);
+(gl_iconv_buf_t) &inptr, &inbytesleft,
+(gl_iconv_buf_t) &outptr, &outbytesleft);
if (res == 0)
result |= 2;
iconv_close (cd_ascii_to_88591);
@@ -131,8 +135,8 @@ int main ()
char *outptr = buf;
size_t outbytesleft = 1;
size_t res = iconv (cd_88591_to_utf8,
-(char **) &inptr, &inbytesleft,
-&outptr, &outbytesleft);
+(gl_iconv_buf_t) &inptr, &inbytesleft,
+(gl_iconv_buf_t) &outptr, &outbytesleft);
if (res != (size_t)(-1) || outptr - buf > 1 || buf[1] != (char)0xAD)
result |= 4;
iconv_close (cd_88591_to_utf8);
@@ -151,8 +155,8 @@ int main ()
char *outptr = buf;
size_t outbytesle
Re: libiconv detection failure
Eli Zaretskii writes:
> The prototype of 'iconv' in GNU libiconv's iconv.h is this:
>
> size_t iconv (iconv_t cd, const char** inbuf, size_t *inbytesleft,
> char** outbuf, size_t *outbytesleft);
>
> The 'const' qualifier of the 2nd argument is decided at libiconv
> configuration time, and may be absent. But the 4th argument is
> _never_ given a 'const' qualifier.
Sorry about that. I have no idea why I type-casted OUTBUF (maybe I was
confused by the Linux manual page saying: "Although inbuf and outbuf are
typed as char **, ...").
> Next, it looks to me that the test program in iconv.m4 is
> inappropriate for a C++ compiler, because (AFAIK) C++ does not allow
> type-casting of the kind that the original test program (before your
> changes) did to the 2nd argument. (With a C compiler, you get a
> warning, which the configure script ignores.)
I'm not familiar with C++, but does that really cause an error?
Yes, I see warnings with:
./configure CC=g++ CFLAGS=-Wcast-qual
The attached patch tries to suppress it. Perhaps
gnulib/tests/test-iconv.c also needs a fix.
I've tested it on MSYS after manually adding 'const' to the iconv
declaration in /usr/local/include/iconv.h, and it seems to be detected.
Here is a new test package:
http://du-a.org/~ueno/junk/test-iconv-2.tar.gz
Thanks,
--
Daiki Ueno
>From 621cb2d9483053aa33cfeb1c6c6c20316aeac432 Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Tue, 21 Oct 2014 18:07:00 +0900
Subject: [PATCH] iconv: avoid false detection of non-working iconv
The INBUF arguments of iconv can be either 'const char **'
or 'char **'. If CC is g++, the difference causes a compile error
and thus leads to a false detection of non-working iconv.
Reported by Eli Zaretskii and Werner LEMBERG in:
<https://lists.gnu.org/archive/html/bug-gnulib/2014-10/msg00023.html>.
* m4/iconv.m4 (AM_ICONV_LINK): Try all possible argument types of
iconv. Bump serial number.
---
ChangeLog | 11 +++
m4/iconv.m4 | 61 -
2 files changed, 43 insertions(+), 29 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 23234b7..62e4824 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2014-10-21 Daiki Ueno
+
+ iconv: avoid false detection of non-working iconv
+ The INBUF arguments of iconv can be either 'const char **'
+ or 'char **'. If CC is g++, the difference causes a compile error
+ and thus leads to a false detection of non-working iconv.
+ Reported by Eli Zaretskii and Werner LEMBERG in:
+ <https://lists.gnu.org/archive/html/bug-gnulib/2014-10/msg00023.html>.
+ * m4/iconv.m4 (AM_ICONV_LINK): Try all possible argument types of
+ iconv. Bump serial number.
+
2014-10-18 Paul Eggert
symlinkat: port to AIX 7.1
diff --git a/m4/iconv.m4 b/m4/iconv.m4
index 4b29c5f..4e37363 100644
--- a/m4/iconv.m4
+++ b/m4/iconv.m4
@@ -1,4 +1,4 @@
-# iconv.m4 serial 18 (gettext-0.18.2)
+# iconv.m4 serial 19 (gettext-0.18.2)
dnl Copyright (C) 2000-2002, 2007-2014 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
@@ -72,27 +72,33 @@ AC_DEFUN([AM_ICONV_LINK],
if test $am_cv_lib_iconv = yes; then
LIBS="$LIBS $LIBICONV"
fi
- AC_RUN_IFELSE(
-[AC_LANG_SOURCE([[
+ am_cv_func_iconv_works=no
+ for ac_iconv_const in '' 'const'; do
+AC_RUN_IFELSE(
+ [AC_LANG_PROGRAM(
+ [[
#include
#include
-int main ()
-{
- int result = 0;
+
+#ifndef ICONV_CONST
+# define ICONV_CONST $ac_iconv_const
+#endif
+ ]],
+ [[int result = 0;
/* Test against AIX 5.1 bug: Failures are not distinguishable from successful
returns. */
{
iconv_t cd_utf8_to_88591 = iconv_open ("ISO8859-1", "UTF-8");
if (cd_utf8_to_88591 != (iconv_t)(-1))
{
-static const char input[] = "\342\202\254"; /* EURO SIGN */
+static ICONV_CONST char input[] = "\342\202\254"; /* EURO SIGN */
char buf[10];
-const char *inptr = input;
+ICONV_CONST char *inptr = input;
size_t inbytesleft = strlen (input);
char *outptr = buf;
size_t outbytesleft = sizeof (buf);
size_t res = iconv (cd_utf8_to_88591,
-(char **) &inptr, &inbytesleft,
+&inptr, &inbytesleft,
&outptr, &outbytesleft);
if (res == 0)
result |= 1;
@@ -105,14 +111,14 @@ int main ()
iconv_t cd_ascii_to_88591 = iconv_open ("ISO8859-1", "646");
if (cd_ascii_to_88591 != (iconv_t)(-1))
{
-static const char input[] = &quo
[PATCH v2 00/10] Update libunistring-related modules to Unicode 7.0.0
Hello, This is an update of the patches I posted on bug-libunistring two weeks ago: https://lists.gnu.org/archive/html/bug-libunistring/2014-10/msg1.html Changes from v1: - The line-breaking implementation has been updated for LBP_HL and LBP_RI - The changes to gen-uninames.lisp is included - Some minor cleanup of gen-uni-tables.c Like the previous post, I didn't include generated files, which you can generate them with: http://du-a.org/~ueno/gen-uni-tables.sh ChangeLog updates are also omitted in order to avoid merge conflicts while testing. The snapshot libunistring distribution is also available: ftp://alpha.gnu.org/gnu/libunistring/libunistring-0.9.5-alpha3.tar.xz I'd really appreciate if someone could go through the patches and find any mistakes before committing. Thanks in advance. Daiki Ueno (10): gen-uni-tables: Minor style fixes gen-uni-tables: Check out-of-range values added to 3-level tables unictype/joininggroup-of: Switch to 3-level table uniwbrk: Ignore Extended/Format at the beginning of the line uniwbrk/u32-wordbreaks-tests: Test using WordBreakTest.txt from UCD uniname: Make codepoint transformation more flexible Update to Unicode 6.1.0 Update to Unicode 6.2.0 Update to Unicode 6.3.0 Update to Unicode 7.0.0 lib/gen-uni-tables.c | 436 -- lib/unictype.in.h | 37 ++- lib/unictype/bidi_byname.gperf| 12 + lib/unictype/joininggroup_byname.gperf| 59 lib/unictype/joininggroup_name.h | 29 ++ lib/unictype/joininggroup_of.c| 29 +- lib/unigbrk.in.h | 3 +- lib/unigbrk/uc-is-grapheme-break.c| 9 +- lib/unilbrk/lbrktables.c | 56 ++-- lib/unilbrk/lbrktables.h | 23 +- lib/uniname/gen-uninames.lisp | 86 +++--- lib/uniname/uniname.c | 203 -- lib/uniwbrk.in.h | 6 +- lib/uniwbrk/u-wordbreaks.h| 83 -- lib/uniwbrk/wbrktable.c | 52 ++-- lib/uniwbrk/wbrktable.h | 2 +- modules/uniwbrk/u32-wordbreaks-tests | 9 +- tests/unigbrk/test-uc-gbrk-prop.c | 1 + tests/unigbrk/test-uc-is-grapheme-break.c | 1 + tests/uniwbrk/test-uc-wordbreaks.c| 181 + tests/uniwbrk/test-uc-wordbreaks.sh | 3 + 21 files changed, 1009 insertions(+), 311 deletions(-) create mode 100644 tests/uniwbrk/test-uc-wordbreaks.c create mode 100755 tests/uniwbrk/test-uc-wordbreaks.sh -- 1.9.3
[PATCH v2 01/10] gen-uni-tables: Minor style fixes
* lib/gen-uni-tables.c: Escape newlines in the Emacs file local
variables specification.
(is_outdigit): Comment out the unused function.
---
lib/gen-uni-tables.c | 43 ++-
1 file changed, 22 insertions(+), 21 deletions(-)
diff --git a/lib/gen-uni-tables.c b/lib/gen-uni-tables.c
index d00013e..6a84503 100644
--- a/lib/gen-uni-tables.c
+++ b/lib/gen-uni-tables.c
@@ -5361,11 +5361,13 @@ is_digit (unsigned int ch)
#endif
}
+#if 0
static bool
is_outdigit (unsigned int ch)
{
return (ch >= 0x0030 && ch <= 0x0039);
}
+#endif
static bool
is_alnum (unsigned int ch)
@@ -9755,26 +9757,25 @@ main (int argc, char * argv[])
/*
* For Emacs M-x compile
* Local Variables:
- * compile-command: "
- gcc -O -Wall gen-uni-tables.c -Iunictype -o gen-uni-tables && \
- ./gen-uni-tables \
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/UnicodeData.txt
\
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/PropList.txt
\
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/DerivedCoreProperties.txt
\
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/ArabicShaping.txt
\
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/Scripts.txt
\
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/Blocks.txt
\
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/3.0.1/PropList-3.0.1.txt
\
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/EastAsianWidth.txt
\
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/LineBreak.txt
\
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/auxiliary/WordBreakProperty.txt
\
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/auxiliary/GraphemeBreakProperty.txt
\
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/CompositionExclusions.txt
\
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/SpecialCasing.txt
\
-
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/CaseFolding.txt
\
-6.0.0 \
- && diff unilbrk/lbrkprop_org.txt unilbrk/lbrkprop.txt \
- && diff uniwbrk/wbrkprop_org.txt uniwbrk/wbrkprop.txt
- "
+ * compile-command: "\
+ * gcc -O -Wall gen-uni-tables.c -Iunictype -o gen-uni-tables && \\
+ * ./gen-uni-tables \\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/UnicodeData.txt
\\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/PropList.txt
\\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/DerivedCoreProperties.txt
\\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/ArabicShaping.txt
\\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/Scripts.txt
\\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/Blocks.txt
\\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/3.0.1/PropList-3.0.1.txt
\\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/EastAsianWidth.txt
\\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/LineBreak.txt
\\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/auxiliary/WordBreakProperty.txt
\\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/auxiliary/GraphemeBreakProperty.txt
\\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/CompositionExclusions.txt
\\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/SpecialCasing.txt
\\
+ *
/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/CaseFolding.txt
\\
+ *
[PATCH v2 04/10] uniwbrk: Ignore Extended/Format at the beginning of the line
* lib/uniwbrk/u-wordbreaks.h (FUNC): Ignore Extend and Format
characters if the previous character property is one of
WBP_NEWLINE, WBP_CR, and WBP_LF.
---
lib/uniwbrk/u-wordbreaks.h | 9 +++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/lib/uniwbrk/u-wordbreaks.h b/lib/uniwbrk/u-wordbreaks.h
index 2d88015..33ca7eb 100644
--- a/lib/uniwbrk/u-wordbreaks.h
+++ b/lib/uniwbrk/u-wordbreaks.h
@@ -112,8 +112,13 @@ FUNC (const UNIT *s, size_t n, char *p)
}
last_char_prop = prop;
- /* Ignore Format and Extend characters, except at the start of the
string. */
- if (last_compchar_prop < 0 || !(prop == WBP_EXTEND || prop ==
WBP_FORMAT))
+ /* Ignore Format and Extend characters, except at the start
+ of the line. */
+ if (last_compchar_prop < 0
+ || last_compchar_prop == WBP_CR
+ || last_compchar_prop == WBP_LF
+ || last_compchar_prop == WBP_NEWLINE
+ || !(prop == WBP_EXTEND || prop == WBP_FORMAT))
{
secondlast_compchar_prop = last_compchar_prop;
last_compchar_prop = prop;
--
1.9.3
[PATCH v2 09/10] Update to Unicode 6.3.0
* tests/uniwbrk/test-uc-wordbreaks.c
(wordbreakproperty_to_string): Support WBP_DQ, WBP_SQ, and WBP_HL.
* lib/uniwbrk.in.h (WBP_DQ, WBP_SQ, WBP_HL): New enumeration values.
* lib/uniwbrk/u-wordbreaks.h (FUNC): Support WB7a, WB7b, and WB7c.
Update WB5, WB6, WB7, WB9, WB11, WB12, WB13a, and WB13b.
* lib/uniwbrk/wbrktable.h (uniwbrk_table): Adjust table size.
* lib/uniwbrk/wbrktable.c (uniwbrk_table): Support rule WB7a.
Update WB5, WB9, WB10, WB13a, and WB13b.
* lib/gen-uni-tables.c (UC_BIDI_LRI, UC_BIDI_RLI, UC_BIDI_FSI)
(UC_BIDI_PDI): New enumeration values.
(bidi_category_byname): Support those enum values.
(is_WBP_MIDNUMLET): Exclude 0x0027 (SINGLE QUOTE), which is now a
dedicated property assigned.
(is_property_case_ignorable): Check 0x0027.
(WBP_DQ, WBP_SQ, WBP_HL): New enumeration values.
(get_wbp, debug_output_wbp, fill_org_wbp, debug_output_org_wbp)
(output_wbp): Support those enum values.
* lib/unictype.in.h (UC_BIDI_LRI, UC_BIDI_RLI, UC_BIDI_FSI)
(UC_BIDI_PDI): New enumeration values.
* lib/unictype/bidi_byname.gperf: Add those property names.
---
lib/gen-uni-tables.c | 76 ++
lib/unictype.in.h | 6 ++-
lib/unictype/bidi_byname.gperf | 12 ++
lib/uniwbrk.in.h | 5 ++-
lib/uniwbrk/u-wordbreaks.h | 38 ---
lib/uniwbrk/wbrktable.c| 52 ++
lib/uniwbrk/wbrktable.h| 2 +-
tests/uniwbrk/test-uc-wordbreaks.c | 3 ++
8 files changed, 145 insertions(+), 49 deletions(-)
diff --git a/lib/gen-uni-tables.c b/lib/gen-uni-tables.c
index f833777..af541d1 100644
--- a/lib/gen-uni-tables.c
+++ b/lib/gen-uni-tables.c
@@ -32,7 +32,7 @@
/usr/local/share/Unidata/CompositionExclusions.txt \
/usr/local/share/Unidata/SpecialCasing.txt \
/usr/local/share/Unidata/CaseFolding.txt \
- 6.2.0
+ 6.3.0
*/
#include
@@ -1307,7 +1307,11 @@ enum
UC_BIDI_B, /* Paragraph Separator */
UC_BIDI_S, /* Segment Separator */
UC_BIDI_WS, /* Whitespace */
- UC_BIDI_ON /* Other Neutral */
+ UC_BIDI_ON, /* Other Neutral */
+ UC_BIDI_LRI, /* Left-to-Right Isolate */
+ UC_BIDI_RLI, /* Right-to-Left Isolate */
+ UC_BIDI_FSI, /* First Strong Isolate */
+ UC_BIDI_PDI /* Pop Directional Isolate */
};
static int
@@ -1365,7 +1369,20 @@ bidi_category_byname (const char *category_name)
break;
}
break;
-case 'L':
+case 'F':
+ switch (category_name[1])
+{
+case 'S':
+ switch (category_name[2])
+{
+case 'I':
+ if (category_name[3] == '\0')
+return UC_BIDI_FSI;
+ break;
+}
+}
+ break;
+ case 'L':
switch (category_name[1])
{
case '\0':
@@ -1381,7 +1398,11 @@ bidi_category_byname (const char *category_name)
if (category_name[3] == '\0')
return UC_BIDI_LRO;
break;
-}
+case 'I':
+ if (category_name[3] == '\0')
+return UC_BIDI_LRI;
+ break;
+ }
break;
}
break;
@@ -1418,6 +1439,10 @@ bidi_category_byname (const char *category_name)
if (category_name[3] == '\0')
return UC_BIDI_PDF;
break;
+case 'I':
+ if (category_name[3] == '\0')
+return UC_BIDI_PDI;
+ break;
}
break;
}
@@ -1438,7 +1463,11 @@ bidi_category_byname (const char *category_name)
if (category_name[3] == '\0')
return UC_BIDI_RLO;
break;
-}
+case 'I':
+ if (category_name[3] == '\0')
+return UC_BIDI_RLI;
+ break;
+ }
break;
}
break;
@@ -2518,7 +2547,7 @@ output_mirror (const char *filename, const char *version)
static bool
is_WBP_MIDNUMLET (unsigned int ch)
{
- return (ch == 0x0027 || ch == 0x002E || ch == 0x2018 || ch == 0x2019
+ return (ch == 0x002E || ch == 0x2018 || ch == 0x2019
|| ch == 0x2024 || ch == 0xFE52 || ch == 0xFF07 || ch == 0xFF0E);
}
@@ -2999,6 +3028,7 @@ static bool
is_property_case_ignorable (unsigned int ch)
{
bool result1 = (is_WBP_MIDLETTER (ch) || is_WBP_MIDNUMLET (ch)
+ || ch == 0x0027
|| is_category_Mn (ch)
|| is_category_Me (ch)
|| is_category_Cf (ch)
@@ -7467,7 +7497,10 @@ enum
WBP_MIDNUM = 5,
WBP_NUMERIC = 6,
WBP_EXTENDNUMLET = 7,
- WBP_RI = 13
+ WBP_RI = 13,
+ WBP_DQ = 14,
+ WBP_SQ = 15,
+ WBP_HL = 16
};
/* Returns the word breaking property for ch, as a bit mask. */
@@ -7506,6 +7539,11 @@
[PATCH v2 06/10] uniname: Make codepoint transformation more flexible
The uniname module tries to reduce memory usage by grouping Unicode codepoints so that each codepoint fit into a 16-bit integer (4-bit tag plus 12-bit index). However, this transformation cannot represent all the characters in Unicode 7.0, where > 16 groups are needed. The attached patch tries to remove the limitation. It basically switches the hard-coded transformation rule into a binary search on a table of contiguous codepoint ranges. * lib/uniname/gen-uninames.lisp (unicode-char): Rename CODE member to INDEX, as it no longer represents a code-point. (range): New struct. (main): Switch to intervals list from a bit-pattern based classification. --- lib/uniname/gen-uninames.lisp | 86 +- lib/uniname/uniname.c | 203 +- 2 files changed, 164 insertions(+), 125 deletions(-) diff --git a/lib/uniname/gen-uninames.lisp b/lib/uniname/gen-uninames.lisp index d08e93f..e7de0a1 100755 --- a/lib/uniname/gen-uninames.lisp +++ b/lib/uniname/gen-uninames.lisp @@ -6,12 +6,18 @@ (defparameter add-comments nil) (defstruct unicode-char - (code nil :type integer) + (index nil :type integer) (name nil :type string) word-indices word-indices-index ) +(defstruct range + (index nil :type integer) + (start-code nil :type integer) + (end-code nil :type integer) +) + (defstruct word-list (hashed nil :type hash-table) (sorted nil :type list) @@ -22,7 +28,10 @@ (defun main (inputfile outputfile) (declare (type string inputfile outputfile)) #+UNICODE (setq *default-file-encoding* charset:utf-8) - (let ((all-chars '())) + (let ((all-chars '()) +(all-ranges '()) +(name-index 0) +range) ;; Read all characters and names from the input file. (with-open-file (istream inputfile :direction :input) (loop @@ -41,40 +50,26 @@ ; specially as well. (unless (or (<= #xF900 code #xFA2D) (<= #xFA30 code #xFA6A) (<= #xFA70 code #xFAD9) (<= #x2F800 code #x2FA1D)) - ; Transform the code so that it fits in 16 bits. In - ; Unicode 5.1 the following ranges are used. - ; 0x0..0x04DFF >>12= 0x00..0x04 -> 0x0..0x4 - ; 0x0A000..0x0AAFF >>12= 0x0A-> 0x5 - ; 0x0F900..0x0 >>12= 0x0F-> 0x6 - ; 0x1..0x10A58 >>12= 0x10-> 0x7 - ; 0x12000..0x12473 >>12= 0x12-> 0x8 - ; 0x1D000..0x1D7FF >>12= 0x1D-> 0x9 - ; 0x1F000..0x1F093 >>12= 0x1F-> 0xA - ; 0x2F800..0x2FAFF >>12= 0x2F-> 0xB - ; 0xE..0xE00FF >>12= 0xE0-> 0xC - (flet ((transform (x) - (dpb - (case (ash x -12) - ((#x00 #x01 #x02 #x03 #x04) (ash x -12)) - (#x0A 5) - (#x0F 6) - (#x10 7) - (#x12 8) - (#x1D 9) - (#x1F #xA) - (#x2F #xB) - (#xE0 #xC) - (t (error "Update the transform function for 0x~5,'0X" x)) - ) - (byte 8 12) - x -)) ) -(push (make-unicode-char :code (transform code) - :name name-string) - all-chars -) ) ) ) ) + (push (make-unicode-char :index name-index + :name name-string) +all-chars) + ;; Update the contiguous range, or start a new range. + (if (and range (= (1+ (range-end-code range)) code)) + (setf (range-end-code range) code) +(progn + (when range +(push range all-ranges)) + (setq range (make-range :index name-index + :start-code code + :end-code code + (incf name-index) + (setq last-code code) + ) ) ) ) ) ) ) (setq all-chars (nreverse all-chars)) +(if range +(push range all-ranges)) +(setq all-ranges (nreverse all-ranges)) ;; Split into words. (let ((words-by-length (make-array 0 :adjustable t))) (dolist (name (list* "HANGUL SYLLABLE" "CJK COMPATIBILITY" (mapcar #'unicode-char-name all-chars))) @@ -257,14 +252,14 @@ (incf i (length (unicode-char-word-indices uc))) ) ) (format ostream "};~%
[PATCH v2 05/10] uniwbrk/u32-wordbreaks-tests: Test using WordBreakTest.txt from UCD
* modules/uniwbrk/u32-wordbreaks-tests (Files): Add
tests/uniwbrk/test-uc-wordbreaks.c,
tests/uniwbrk/test-uc-wordbreaks.sh, and
tests/uniwbrk/WordBreakTest.txt.
(Makefile.am): Add uniwbrk/test-uc-wordbreaks.sh to $(TESTS), add
test-uc-wordbreaks to $(check_PROGRAMS), and define
test_uc_wordbreaks_SOURCES and test_uc_wordbreaks_LDADD.
* tests/uniwbrk/test-uc-wordbreaks.sh: New file.
* tests/uniwbrk/test-uc-wordbreaks.c: New file.
---
modules/uniwbrk/u32-wordbreaks-tests | 9 +-
tests/uniwbrk/test-uc-wordbreaks.c | 177 +++
tests/uniwbrk/test-uc-wordbreaks.sh | 3 +
3 files changed, 187 insertions(+), 2 deletions(-)
create mode 100644 tests/uniwbrk/test-uc-wordbreaks.c
create mode 100755 tests/uniwbrk/test-uc-wordbreaks.sh
diff --git a/modules/uniwbrk/u32-wordbreaks-tests
b/modules/uniwbrk/u32-wordbreaks-tests
index 1da80e4..897d4a4 100644
--- a/modules/uniwbrk/u32-wordbreaks-tests
+++ b/modules/uniwbrk/u32-wordbreaks-tests
@@ -1,5 +1,8 @@
Files:
tests/uniwbrk/test-u32-wordbreaks.c
+tests/uniwbrk/test-uc-wordbreaks.c
+tests/uniwbrk/test-uc-wordbreaks.sh
+tests/uniwbrk/WordBreakTest.txt
tests/macros.h
Depends-on:
@@ -7,7 +10,9 @@ Depends-on:
configure.ac:
Makefile.am:
-TESTS += test-u32-wordbreaks
-check_PROGRAMS += test-u32-wordbreaks
+TESTS += test-u32-wordbreaks uniwbrk/test-uc-wordbreaks.sh
+check_PROGRAMS += test-u32-wordbreaks test-uc-wordbreaks
test_u32_wordbreaks_SOURCES = uniwbrk/test-u32-wordbreaks.c
test_u32_wordbreaks_LDADD = $(LDADD) $(LIBUNISTRING)
+test_uc_wordbreaks_SOURCES = uniwbrk/test-uc-wordbreaks.c
+test_uc_wordbreaks_LDADD = $(LDADD) $(LIBUNISTRING)
diff --git a/tests/uniwbrk/test-uc-wordbreaks.c
b/tests/uniwbrk/test-uc-wordbreaks.c
new file mode 100644
index 000..736cdba
--- /dev/null
+++ b/tests/uniwbrk/test-uc-wordbreaks.c
@@ -0,0 +1,177 @@
+/* Word break function test, using test data from UCD.
+ Copyright (C) 2010-2014 Free Software Foundation, Inc.
+
+ This program is free software: you can redistribute it and/or modify it
+ under the terms of the GNU Lesser General Public License as published
+ by the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>. */
+
+/* Written by Daiki Ueno , 2014.
+
+ Largely based on unigbrk/test-uc-is-grapheme-break.c,
+ written by Ben Pfaff , 2010. */
+
+#include
+
+/* Specification. */
+#include
+
+#include
+#include
+#include
+
+const char *
+wordbreakproperty_to_string (int wbp)
+{
+ switch (wbp)
+{
+#define CASE(VALUE) case WBP_##VALUE: return #VALUE;
+ CASE(OTHER)
+ CASE(CR)
+ CASE(LF)
+ CASE(NEWLINE)
+ CASE(EXTEND)
+ CASE(FORMAT)
+ CASE(KATAKANA)
+ CASE(ALETTER)
+ CASE(MIDNUMLET)
+ CASE(MIDLETTER)
+ CASE(MIDNUM)
+ CASE(NUMERIC)
+ CASE(EXTENDNUMLET)
+}
+ abort ();
+}
+
+int
+main (int argc, char *argv[])
+{
+ const char *filename;
+ char line[4096];
+ int exit_code;
+ FILE *stream;
+ int lineno;
+
+ if (argc != 2)
+{
+ fprintf (stderr, "usage: %s FILENAME\n"
+ "where FILENAME is the location of the WordBreakTest.txt\n"
+ "test file.\n", argv[0]);
+ exit (1);
+}
+
+ filename = argv[1];
+ stream = fopen (filename, "r");
+ if (stream == NULL)
+{
+ fprintf (stderr, "error during fopen of '%s'\n", filename);
+ exit (1);
+}
+
+ exit_code = 0;
+ lineno = 0;
+ while (fgets (line, sizeof line, stream))
+{
+ char *comment;
+ const char *p;
+ uint32_t input[100];
+ char breaks[101];
+ char breaks_expected[101];
+ int i;
+
+ lineno++;
+
+ memset (breaks, 0, sizeof (breaks));
+ memset (breaks_expected, 0, sizeof (breaks_expected));
+
+ comment = strchr (line, '#');
+ if (comment != NULL)
+*comment = '\0';
+ if (line[strspn (line, " \t\r\n")] == '\0')
+continue;
+
+ i = 0;
+ p = line;
+ do
+{
+ p += strspn (p, " \t\r\n");
+ if (!strncmp (p, "\303\267" /* ÷ */, 2))
+{
+ breaks_expected[i] = 1;
+ p += 2;
+}
+ else if (!strncmp (p, "\303\227" /* × */, 2))
+{
+ breaks_expected[i] = 0;
+ p += 2;
+}
+ else
+{
+ fprintf (stderr, "%s:%d.%d: syntax error expecting '÷' or
[PATCH v2 08/10] Update to Unicode 6.2.0
* lib/unilbrk/lbrktables.h (LBP_RI): New enumeration value.
(unilbrk_table): Adjust table size.
* lib/unilbrk/lbrktables.c (unilbrk_table): Add a row and column
for LBP_RI.
* lib/uniwbrk.in.h (WBP_RI): New enumeration value.
* lib/uniwbrk/u-wordbreaks.h (FUNC): Support rule WB13c.
Normalize table index skipping ignored properties.
* lib/uniwbrk/wbrktable.c (uniwbrk_table): Support WBP_RI. Remove
WBP_EXTEND and WBP_FORMAT, which are now computed without using
the table.
* lib/uniwbrk/wbrktable.h: Adjust table size.
* tests/uniwbrk/test-uc-wordbreaks.c
(wordbreakproperty_to_string): Support WBP_RI.
* lib/unigbrk.in.h (GBP_RI): New enumeration value.
* lib/unigbrk/uc-is-grapheme-break.c (UC_IS_GRAPHEME_BREAK):
Support rule GB8a.
(UC_GRAPHEME_BREAKS_FOR, gb_table): Support GBP_RI.
* tests/unigbrk/test-uc-is-grapheme-break.c
(graphemebreakproperty_to_string): Support GBP_RI.
* lib/gen-uni-tables.c (LBP_RI): New enumeration value.
(get_lbp, debug_output_lbp, fill_org_lbp, debug_output_org_lbp)
(output_lbp): Support LBP_RI.
(WBP_RI): New enumeration value.
(debug_output_wbp, fill_org_wbp, debug_output_org_wbp)
(output_wbp): Support WBP_RI.
(GBP_RI): New enumeration value.
(output_gbp_test, fill_org_gbp): Support GBP_RI.
---
lib/gen-uni-tables.c | 49 ++
lib/unigbrk.in.h | 3 +-
lib/unigbrk/uc-is-grapheme-break.c| 9 +++--
lib/unilbrk/lbrktables.c | 57 ---
lib/unilbrk/lbrktables.h | 21 ++--
lib/uniwbrk.in.h | 3 +-
lib/uniwbrk/u-wordbreaks.h| 36 +--
lib/uniwbrk/wbrktable.c | 24 ++---
lib/uniwbrk/wbrktable.h | 2 +-
tests/unigbrk/test-uc-gbrk-prop.c | 1 +
tests/unigbrk/test-uc-is-grapheme-break.c | 1 +
tests/uniwbrk/test-uc-wordbreaks.c| 1 +
12 files changed, 127 insertions(+), 80 deletions(-)
diff --git a/lib/gen-uni-tables.c b/lib/gen-uni-tables.c
index ec1aba5..f833777 100644
--- a/lib/gen-uni-tables.c
+++ b/lib/gen-uni-tables.c
@@ -32,7 +32,7 @@
/usr/local/share/Unidata/CompositionExclusions.txt \
/usr/local/share/Unidata/SpecialCasing.txt \
/usr/local/share/Unidata/CaseFolding.txt \
- 6.1.0
+ 6.2.0
*/
#include
@@ -6213,22 +6213,22 @@ output_width_property_test (const char *filename)
enum
{
- /* Values >= 26 are resolved at run time. */
- LBP_BK = 26, /* mandatory break */
+ /* Values >= 27 are resolved at run time. */
+ LBP_BK = 27, /* mandatory break */
/*LBP_CR, carriage return - not used here because it's a DOSism */
/*LBP_LF, line feed - not used here because it's a DOSism */
- LBP_CM = 27, /* attached characters and combining marks */
+ LBP_CM = 28, /* attached characters and combining marks */
/*LBP_NL, next line - not used here because it's equivalent to LBP_BK
*/
/*LBP_SG, surrogates - not used here because they are not characters */
LBP_WJ = 0, /* word joiner */
- LBP_ZW = 28, /* zero width space */
+ LBP_ZW = 29, /* zero width space */
LBP_GL = 1, /* non-breaking (glue) */
- LBP_SP = 29, /* space */
+ LBP_SP = 30, /* space */
LBP_B2 = 2, /* break opportunity before and after */
LBP_BA = 3, /* break opportunity after */
LBP_BB = 4, /* break opportunity before */
LBP_HY = 5, /* hyphen */
- LBP_CB = 30, /* contingent break opportunity */
+ LBP_CB = 31, /* contingent break opportunity */
LBP_CL = 6, /* closing punctuation */
LBP_CP = 7, /* closing parenthesis */
LBP_EX = 8, /* exclamation/interrogation */
@@ -6241,7 +6241,7 @@ enum
LBP_PO = 15, /* postfix (numeric) */
LBP_PR = 16, /* prefix (numeric) */
LBP_SY = 17, /* symbols allowing breaks */
- LBP_AI = 31, /* ambiguous (alphabetic or ideograph) */
+ LBP_AI = 32, /* ambiguous (alphabetic or ideograph) */
LBP_AL = 18, /* ordinary alphabetic and symbol characters */
/*LBP_CJ, conditional Japanese starter, resolved to NS */
LBP_H2 = 19, /* Hangul LV syllable */
@@ -6251,8 +6251,9 @@ enum
LBP_JL = 22, /* Hangul L Jamo */
LBP_JV = 23, /* Hangul V Jamo */
LBP_JT = 24, /* Hangul T Jamo */
- LBP_SA = 32, /* complex context (South East Asian) */
- LBP_XX = 33 /* unknown */
+ LBP_RI = 26, /* regional indicator */
+ LBP_SA = 33, /* complex context (South East Asian) */
+ LBP_XX = 34 /* unknown */
};
/* Returns the line breaking classification for ch, as a bit mask. */
@@ -6710,6 +6711,10 @@ get_lbp (unsigned int ch)
if ((ch >= 0x11A8 && ch <= 0x11FF) || (ch >= 0xD7CB && ch <= 0xD7FB))
attr |= (int64_t) 1 << LBP_JT;
+ /* regional indicator */
+ if (ch >= 0x1F1E6 && ch <= 0x1F1FF)
+attr |= (int64_t) 1 << LBP_RI;
+
/* complex context (South East Asian) */
if (((unicode_attributes[ch].cate
[PATCH v2 07/10] Update to Unicode 6.1.0
* lib/unictype/joininggroup_byname.gperf: Add Rohingya Yeh
joining group name.
* lib/unictype/joininggroup_name.h: Likewise.
* lib/unilbrk/lbrktables.h (LBP_HL): New enumeration value.
(unilbrk_table): Adjust table size.
* lib/unilbrk/lbrktables.c (unilbrk_table): Add a row and column
for LBP_HL.
* lib/gen-uni-tables.c (UC_JOINING_GROUP_ROHINGYA_YEH): New
enumeration value.
(fill_arabicshaping, joining_group_as_c_identifier): Support
UC_JOINING_GROUP_ROHINGYA_YEH.
(is_property_default_ignorable_code_point): Reject U+0604.
(LBP_HL): New enumeration value.
(get_lbp, debug_output_lbp, fill_org_lbp, debug_output_org_lbp)
(output_lbp): Support LBP_HL.
(fill_org_lbp): Resolve CJ as NS, for backward compatibility.
---
lib/gen-uni-tables.c | 42 +-
lib/unictype.in.h | 3 +-
lib/unictype/joininggroup_byname.gperf | 2 ++
lib/unictype/joininggroup_name.h | 1 +
lib/unilbrk/lbrktables.c | 55 +-
lib/unilbrk/lbrktables.h | 22 +++---
6 files changed, 73 insertions(+), 52 deletions(-)
diff --git a/lib/gen-uni-tables.c b/lib/gen-uni-tables.c
index 1af832e..ec1aba5 100644
--- a/lib/gen-uni-tables.c
+++ b/lib/gen-uni-tables.c
@@ -32,7 +32,7 @@
/usr/local/share/Unidata/CompositionExclusions.txt \
/usr/local/share/Unidata/SpecialCasing.txt \
/usr/local/share/Unidata/CaseFolding.txt \
- 6.0.0
+ 6.1.0
*/
#include
@@ -2868,7 +2868,7 @@ is_property_default_ignorable_code_point (unsigned int ch)
bool result1 =
(is_category_Cf (ch)
&& !(ch >= 0xFFF9 && ch <= 0xFFFB) /* Annotations */
- && !((ch >= 0x0600 && ch <= 0x0603) || ch == 0x06DD || ch == 0x070F)
+ && !((ch >= 0x0600 && ch <= 0x0604) || ch == 0x06DD || ch == 0x070F)
/* For some reason, the following are not listed as having property
Default_Ignorable_Code_Point. */
&& !(ch == 0x110BD))
@@ -3746,7 +3746,8 @@ enum
UC_JOINING_GROUP_YUDH, /* Yudh */
UC_JOINING_GROUP_YUDH_HE, /* Yudh_He */
UC_JOINING_GROUP_ZAIN, /* Zain */
- UC_JOINING_GROUP_ZHAIN /* Zhain */
+ UC_JOINING_GROUP_ZHAIN, /* Zhain */
+ UC_JOINING_GROUP_ROHINGYA_YEH /* Rohingya_Yeh */
};
static uint8_t unicode_joining_group[0x11];
@@ -3886,6 +3887,7 @@ fill_arabicshaping (const char *arabicshaping_filename)
TRY(UC_JOINING_GROUP_YUDH_HE, "YUDH HE")
TRY(UC_JOINING_GROUP_ZAIN, "ZAIN")
TRY(UC_JOINING_GROUP_ZHAIN, "ZHAIN")
+ TRY(UC_JOINING_GROUP_ROHINGYA_YEH, "ROHINGYA YEH")
#undef TRY
else
{
@@ -3987,7 +3989,7 @@ output_joining_type (const char *filename, const char
*version)
}
fprintf (stream, "/* DO NOT EDIT! GENERATED AUTOMATICALLY! */\n");
- fprintf (stream, "/* Arabic joining group of Unicode characters. */\n");
+ fprintf (stream, "/* Arabic joining type of Unicode characters. */\n");
fprintf (stream, "/* Generated automatically by gen-uni-tables.c for Unicode
%s. */\n",
version);
@@ -4167,6 +4169,7 @@ joining_group_as_c_identifier (int joining_group)
TRY(UC_JOINING_GROUP_YUDH_HE)
TRY(UC_JOINING_GROUP_ZAIN)
TRY(UC_JOINING_GROUP_ZHAIN)
+ TRY(UC_JOINING_GROUP_ROHINGYA_YEH)
#undef TRY
abort ();
}
@@ -6210,22 +6213,22 @@ output_width_property_test (const char *filename)
enum
{
- /* Values >= 25 are resolved at run time. */
- LBP_BK = 25, /* mandatory break */
+ /* Values >= 26 are resolved at run time. */
+ LBP_BK = 26, /* mandatory break */
/*LBP_CR, carriage return - not used here because it's a DOSism */
/*LBP_LF, line feed - not used here because it's a DOSism */
- LBP_CM = 26, /* attached characters and combining marks */
+ LBP_CM = 27, /* attached characters and combining marks */
/*LBP_NL, next line - not used here because it's equivalent to LBP_BK
*/
/*LBP_SG, surrogates - not used here because they are not characters */
LBP_WJ = 0, /* word joiner */
- LBP_ZW = 27, /* zero width space */
+ LBP_ZW = 28, /* zero width space */
LBP_GL = 1, /* non-breaking (glue) */
- LBP_SP = 28, /* space */
+ LBP_SP = 29, /* space */
LBP_B2 = 2, /* break opportunity before and after */
LBP_BA = 3, /* break opportunity after */
LBP_BB = 4, /* break opportunity before */
LBP_HY = 5, /* hyphen */
- LBP_CB = 29, /* contingent break opportunity */
+ LBP_CB = 30, /* contingent break opportunity */
LBP_CL = 6, /* closing punctuation */
LBP_CP = 7, /* closing parenthesis */
LBP_EX = 8, /* exclamation/interrogation */
@@ -6238,16 +6241,18 @@ enum
LBP_PO = 15, /* postfix (numeric) */
LBP_PR = 16, /* prefix (numeric) */
LBP_SY = 17, /* symbols allowing breaks */
- LBP_AI = 30, /* ambiguous
[PATCH v2 03/10] unictype/joininggroup-of: Switch to 3-level table
* lib/gen-uni-tables.c (output_joining_group): Switch to
3-level table to accommodate a joining group defined with higher
code-point value. Since there are only 88 groups defined in
Unicode 7.0.0, use 7-bit packed format for level3 entries.
* lib/unictype/joininggroup_of.c (uc_joining_group): Adjust to use
3-level table.
* lib/unictype/joininggroup_of.h: Regenerate.
---
lib/gen-uni-tables.c | 155 ++---
lib/unictype/joininggroup_of.c | 29 ++--
2 files changed, 139 insertions(+), 45 deletions(-)
diff --git a/lib/gen-uni-tables.c b/lib/gen-uni-tables.c
index 7a273fc..1af832e 100644
--- a/lib/gen-uni-tables.c
+++ b/lib/gen-uni-tables.c
@@ -3987,7 +3987,7 @@ output_joining_type (const char *filename, const char
*version)
}
fprintf (stream, "/* DO NOT EDIT! GENERATED AUTOMATICALLY! */\n");
- fprintf (stream, "/* Arabic joining type of Unicode characters. */\n");
+ fprintf (stream, "/* Arabic joining group of Unicode characters. */\n");
fprintf (stream, "/* Generated automatically by gen-uni-tables.c for Unicode
%s. */\n",
version);
@@ -4213,11 +4213,22 @@ output_joining_group_test (const char *filename, const
char *version)
}
}
+/* Construction of sparse 3-level tables. */
+#define TABLE joining_group_table
+#define ELEMENT uint8_t
+#define DEFAULT UC_JOINING_GROUP_NONE
+#define xmalloc malloc
+#define xrealloc realloc
+#include "3level.h"
+
static void
output_joining_group (const char *filename, const char *version)
{
FILE *stream;
- unsigned int ch_min, ch_max, ch, i;
+ unsigned int ch, i;
+ struct joining_group_table t;
+ unsigned int level1_offset, level2_offset, level3_offset;
+ uint16_t *level3_packed;
stream = fopen (filename, "w");
if (stream == NULL)
@@ -4231,51 +4242,115 @@ output_joining_group (const char *filename, const char
*version)
fprintf (stream, "/* Generated automatically by gen-uni-tables.c for Unicode
%s. */\n",
version);
- ch_min = 0x10;
+ t.p = 7;
+ t.q = 9;
+ joining_group_table_init (&t);
+
for (ch = 0; ch < 0x11; ch++)
-if (unicode_joining_group[ch] != UC_JOINING_GROUP_NONE)
- {
-ch_min = ch;
-break;
- }
+{
+ uint8_t value = unicode_joining_group[ch];
- ch_max = 0;
- for (ch = 0x10; ch > 0; ch--)
-if (unicode_joining_group[ch] != UC_JOINING_GROUP_NONE)
- {
-ch_max = ch;
-break;
- }
+ if (value > 0x7f)
+abort ();
- if (!(ch_min <= ch_max))
-abort ();
+ joining_group_table_add (&t, ch, value);
+}
- /* If the interval [ch_min, ch_max] is too large, we should better use a
- 3-level table. */
- if (!(ch_max - ch_min < 0x200))
-abort ();
+ joining_group_table_finalize (&t);
- fprintf (stream, "#define joining_group_header_0 0x%x\n", ch_min);
- fprintf (stream, "static const unsigned char u_joining_group[0x%x - 0x%x]
=\n",
- ch_max + 1, ch_min);
- fprintf (stream, "{");
- for (i = 0; i <= ch_max - ch_min; i++)
-{
- const char *s;
+ /* Offsets in t.result, in memory of this process. */
+ level1_offset =
+5 * sizeof (uint32_t);
+ level2_offset =
+5 * sizeof (uint32_t)
++ t.level1_size * sizeof (uint32_t);
+ level3_offset =
+5 * sizeof (uint32_t)
++ t.level1_size * sizeof (uint32_t)
++ (t.level2_size << t.q) * sizeof (uint32_t);
- ch = ch_min + i;
- if ((i % 2) == 0)
-fprintf (stream, "\n ");
- s = joining_group_as_c_identifier (unicode_joining_group[ch]);
- fprintf (stream, " %s", s);
- if (i+1 <= ch_max - ch_min)
-{
- fprintf (stream, ",");
- if (((i+1) % 2) != 0)
-fprintf (stream, "%*s", 38 - (int) strlen (s), "");
-}
+ for (i = 0; i < 5; i++)
+fprintf (stream, "#define joining_group_header_%d %d\n", i,
+ ((uint32_t *) t.result)[i]);
+ fprintf (stream, "static const\n");
+ fprintf (stream, "struct\n");
+ fprintf (stream, " {\n");
+ fprintf (stream, "int level1[%zu];\n", t.level1_size);
+ fprintf (stream, "short level2[%zu << %d];\n", t.level2_size, t.q);
+ fprintf (stream, "unsigned short level3[%zu * %d + 1];\n", t.level3_size,
+ (1 << t.p) * 7 / 16);
+ fprintf (stream, " }\n");
+ fprintf (stream, "u_joining_group =\n");
+ fprintf (stream, "{\n");
+ fprintf (stream, " {");
+ if (t.level1_size > 8)
+fprintf (stream, "\n ");
+ for (i = 0; i < t.level1_size; i++)
+{
+ uint32_t offset;
+ if (i > 0 && (i % 8) == 0)
+fprintf (stream, "\n ");
+ offset = ((uint32_t *) (t.result + level1_offset))[i];
+ if (offset == 0)
+fprintf (stream, " %5d", -1);
+ else
+fprintf (stream, " %5zu",
+ (offset - level2_offset) / sizeof (uint32_t));
+ if (i+1 < t.level1_size)
+fprintf (stream, ",");
}
- fprintf (stream, "\n");
+ if (t.level1_size > 8)
+fprintf (str
[PATCH v2 02/10] gen-uni-tables: Check out-of-range values added to 3-level tables
* lib/gen-uni-tables.c (output_category, output_bidi_category)
(output_joining_type, output_ident_category): Check out-of-range
values added to 3-level tables.
---
lib/gen-uni-tables.c | 13 +
1 file changed, 13 insertions(+)
diff --git a/lib/gen-uni-tables.c b/lib/gen-uni-tables.c
index 6a84503..7a273fc 100644
--- a/lib/gen-uni-tables.c
+++ b/lib/gen-uni-tables.c
@@ -1035,6 +1035,9 @@ output_category (const char *filename, const char
*version)
for (log2_value = 0; value > 1; value >>= 1, log2_value++);
+ if (log2_value > 0x1f)
+abort ();
+
category_table_add (&t, ch, log2_value);
}
@@ -1525,6 +1528,9 @@ output_bidi_category (const char *filename, const char
*version)
{
int value = get_bidi_category (ch);
+ if (value > 0x1f)
+abort ();
+
bidi_category_table_add (&t, ch, value);
}
@@ -3993,6 +3999,9 @@ output_joining_type (const char *filename, const char
*version)
{
uint8_t value = unicode_joining_type[ch];
+ if (value != (uint8_t)~(uint8_t)0 && value > 0x0f)
+abort ();
+
joining_type_table_add (&t, ch, value);
}
@@ -5137,6 +5146,10 @@ output_ident_category (const char *filename, int
(*predicate) (unsigned int), co
for (ch = 0; ch < 0x11; ch++)
{
int syntaxcode = predicate (ch);
+
+ if (syntaxcode > 0x03)
+abort ();
+
if (syntaxcode != UC_IDENTIFIER_INVALID)
identsyntax_table_add (&t, ch, syntaxcode);
}
--
1.9.3
Re: libiconv detection failure
Eli Zaretskii writes: >> The attached patch tries to suppress it. Perhaps >> gnulib/tests/test-iconv.c also needs a fix. >> >> I've tested it on MSYS after manually adding 'const' to the iconv >> declaration in /usr/local/include/iconv.h, and it seems to be detected. >> >> Here is a new test package: >> http://du-a.org/~ueno/junk/test-iconv-2.tar.gz > > Thanks, this works. Thanks for testing, pushed. Regards, -- Daiki Ueno
Re: [PATCH v2 00/10] Update libunistring-related modules to Unicode 7.0.0
Paul Eggert writes: > On 10/23/2014 04:43 AM, Pádraig Brady wrote: >> Except for the assert() nit, I can't find any issues. > > My only nit was adding the "#if 0"; might as well just delete the code > instead. I only read the code; I didn't test it. Thanks. Will fix those in the next update. Also, I wasn't aware that a manual update is needed for uniwidth/width.h; that will be fixed as well. Regards, -- Daiki Ueno
Re: m4-1.4.17 test-mbrtowc3.sh fails on Solaris without European localization
Eric Blake writes: > [adding gnulib] > > On 11/06/2014 11:19 AM, Kiyoshi KANAZAWA wrote: >> Hello, >> >> On Solaris 10 x86/x64, >> "test-mbrtowc3.sh" fails if European localization is not installed. >> It passes with European localization. I can reproduce it, and it seems to be the same issue as: https://lists.gnu.org/archive/html/bug-gnulib/2014-10/msg0.html which I originally encountered on Solaris 11. (It's curious that you could work it around by installing European localization, while the test checks the mbrtowc behavior under Japanese locale.) Perhaps it would be good to replace mbrtowc on Solaris, though I have no idea how to detect the wrong behavior without depending on particular locale data. Alternatively, we could simply ignore this test failure on Solaris, since the tested calling convention is not very common. Thoughts? Regards, -- Daiki Ueno
Re: [PATCH v2 00/10] Update libunistring-related modules to Unicode 7.0.0
Daiki Ueno writes:
> Paul Eggert writes:
>
>> On 10/23/2014 04:43 AM, Pádraig Brady wrote:
>>> Except for the assert() nit, I can't find any issues.
>>
>> My only nit was adding the "#if 0"; might as well just delete the code
>> instead. I only read the code; I didn't test it.
>
> Thanks. Will fix those in the next update. Also, I wasn't aware that a
> manual update is needed for uniwidth/width.h; that will be fixed as
> well.
In advance of the mega updates, I've pushed the first two patches and
additional one, which tidy up gen-uni-tables.c.
The actual patches are attached below:
>From b6ffce655f8639bac7f86c40ec9d74b709f8a2d9 Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Thu, 23 Oct 2014 16:05:37 +0900
Subject: [PATCH 1/3] gen-uni-tables: cosmetic improvements
* lib/gen-uni-tables.c: Escape newlines in the Emacs file local
variables specification.
(is_outdigit): Remove unused function.
---
ChangeLog| 7 +++
lib/gen-uni-tables.c | 47 ---
2 files changed, 27 insertions(+), 27 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index adabc17..fa31e58 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2014-11-13 Daiki Ueno
+
+ gen-uni-tables: cosmetic improvements
+ * lib/gen-uni-tables.c: Escape newlines in the Emacs file local
+ variables specification.
+ (is_outdigit): Remove unused function.
+
2014-11-12 Jeroen Roovers(tiny change)
fcntl-h-tests: port to PA-RISC GNU/Linux
diff --git a/lib/gen-uni-tables.c b/lib/gen-uni-tables.c
index d00013e..aed3d9a 100644
--- a/lib/gen-uni-tables.c
+++ b/lib/gen-uni-tables.c
@@ -5362,12 +5362,6 @@ is_digit (unsigned int ch)
}
static bool
-is_outdigit (unsigned int ch)
-{
- return (ch >= 0x0030 && ch <= 0x0039);
-}
-
-static bool
is_alnum (unsigned int ch)
{
return is_alpha (ch) || is_digit (ch);
@@ -9755,26 +9749,25 @@ main (int argc, char * argv[])
/*
* For Emacs M-x compile
* Local Variables:
- * compile-command: "
- gcc -O -Wall gen-uni-tables.c -Iunictype -o gen-uni-tables && \
- ./gen-uni-tables \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/UnicodeData.txt \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/PropList.txt \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/DerivedCoreProperties.txt \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/ArabicShaping.txt \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/Scripts.txt \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/Blocks.txt \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/3.0.1/PropList-3.0.1.txt \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/EastAsianWidth.txt \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/LineBreak.txt \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/auxiliary/WordBreakProperty.txt \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/auxiliary/GraphemeBreakProperty.txt \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/CompositionExclusions.txt \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/SpecialCasing.txt \
-/gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/CaseFolding.txt \
-6.0.0 \
- && diff unilbrk/lbrkprop_org.txt unilbrk/lbrkprop.txt \
- && diff uniwbrk/wbrkprop_org.txt uniwbrk/wbrkprop.txt
- "
+ * compile-command: "\
+ * gcc -O -Wall gen-uni-tables.c -Iunictype -o gen-uni-tables && \\
+ * ./gen-uni-tables \\
+ */gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/UnicodeData.txt \\
+ */gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/PropList.txt \\
+ */gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/DerivedCoreProperties.txt \\
+ */gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/ArchiveVersions/6.0.0/ucd/ArabicShaping.txt \\
+ */gfs/petix/Volumes/ExtData/www-archive/software/i18n/unicode/ftp.unicode.org/Archi
Re: "test-lock" stuck on FreeBSD
Assaf Gordon writes: > On 09/04/2014 06:02 PM, Paul Eggert wrote: >> I'm afraid you're going to have to debug this one, as I don't have >> easy access to FreeBSD > > SadlyI will not be able to devote time for that in the near future. > So it will have to wait. As I'm getting more reports (perhaps) related to this, I've filed a bug at the FreeBSD bugzilla, with a reproducer written in plain pthreads: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=195098 Regards, -- Daiki Ueno
[PATCH] pipe-filter-gi, pipe-filter-ii: port to AIX
Hello,
I got reports saying that pipe-filter-* modules don't compile with xlc
on AIX. The error is:
"pipe-filter-aux.h", line 99.14: 1506-780 (S) Reference to "select" with
internal linkage is not allowed within inline definition of "nonintr_select".
It seems that 'select' is indeed defined as static on AIX and referring
it from an inline function causes the error. Since the use of the
redefined 'select' in pipe-filter-* is limited, how about moving the
definition to the caller? I'm attaching a patch in that direction.
I've also uploaded a test package at:
http://du-a.org/~ueno/junk/test-pipe-filter-ii-aix.tar.gz
Regards,
--
Daiki Ueno
>From 17a0216558375b833ca11adeb39eedd42fbd42e3 Mon Sep 17 00:00:00 2001
From: Daiki Ueno
Date: Tue, 18 Nov 2014 21:11:09 +0900
Subject: [PATCH] pipe-filter-gi, pipe-filter-ii: port to AIX
On AIX 7.1, 'select' is defined as static and cannot be referred
from inline function.
* lib/pipe-filter-aux.h (nointr_select): Remove, manually expand
the definition...
* lib/pipe-filter-gi.c (filter_loop): ...here, and...
* lib/pipe-filter-ii.c (pipe_filter_ii_execute): ...here.
---
ChangeLog | 10 ++
lib/pipe-filter-aux.h | 19 ---
lib/pipe-filter-gi.c | 16 +++-
lib/pipe-filter-ii.c | 13 ++---
4 files changed, 31 insertions(+), 27 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 16dc620..5d42c53 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,13 @@
+2014-11-18 Daiki Ueno
+
+ pipe-filter-gi, pipe-filter-ii: port to AIX
+ On AIX 7.1, 'select' is defined as static and cannot be referred
+ from inline function.
+ * lib/pipe-filter-aux.h (nointr_select): Remove, manually expand
+ the definition...
+ * lib/pipe-filter-gi.c (filter_loop): ...here, and...
+ * lib/pipe-filter-ii.c (pipe_filter_ii_execute): ...here.
+
2014-11-14 Paul Eggert
extern-inline: update commentary about GCC bugs
diff --git a/lib/pipe-filter-aux.h b/lib/pipe-filter-aux.h
index c3f1be8..8f2a707 100644
--- a/lib/pipe-filter-aux.h
+++ b/lib/pipe-filter-aux.h
@@ -87,25 +87,6 @@ nonintr_write (int fd, const void *buf, size_t count)
#undef write /* avoid warning on VMS */
#define write nonintr_write
-# if HAVE_SELECT
-
-PIPE_FILTER_AUX_INLINE int
-nonintr_select (int n, fd_set *readfds, fd_set *writefds, fd_set *exceptfds,
-struct timeval *timeout)
-{
- int retval;
-
- do
-retval = select (n, readfds, writefds, exceptfds, timeout);
- while (retval < 0 && errno == EINTR);
-
- return retval;
-}
-# undef select /* avoid warning on VMS */
-# define select nonintr_select
-
-# endif
-
#endif
/* Non-blocking I/O. */
diff --git a/lib/pipe-filter-gi.c b/lib/pipe-filter-gi.c
index de7b257..cd3ead7 100644
--- a/lib/pipe-filter-gi.c
+++ b/lib/pipe-filter-gi.c
@@ -302,7 +302,7 @@ filter_loop (struct pipe_filter_gi *filter, const char *wbuf, size_t count)
/* Here, if done_writing, filter->reader_terminated is false. When
filter->reader_terminated becomes true, this loop is terminated. */
# if HAVE_SELECT
- int n;
+ int n, retval;
/* See whether reading or writing is possible. */
n = 1;
@@ -317,10 +317,16 @@ filter_loop (struct pipe_filter_gi *filter, const char *wbuf, size_t count)
if (n <= filter->fd[1])
n = filter->fd[1] + 1;
}
- n = select (n,
- (!filter->reader_terminated ? &filter->readfds : NULL),
- (!done_writing ? &filter->writefds : NULL),
- NULL, NULL);
+ /* Do EINTR handling here instead of in pipe-filter-aux.h,
+ because select() cannot be referred from an inline function
+ on AIX 7.1. */
+ do
+retval = select (n,
+ (!filter->reader_terminated ? &filter->readfds : NULL),
+ (!done_writing ? &filter->writefds : NULL),
+ NULL, NULL);
+ while (retval < 0 && errno == EINTR);
+ n = retval;
if (n < 0)
{
diff --git a/lib/pipe-filter-ii.c b/lib/pipe-filter-ii.c
index 2072ea3..85c7af2 100644
--- a/lib/pipe-filter-ii.c
+++ b/lib/pipe-filter-ii.c
@@ -309,7 +309,7 @@ pipe_filter_ii_execute (const char *progname,
for (;;)
{
# if HAVE_SELECT
-int n;
+int n, retval;
FD_SET (fd[0], &readfds);
n = fd[0] + 1;
@@ -320,8 +320,15 @@ pipe_filter_ii_execute (const char *progname,
n = fd[1] + 1;
}
-n = select (n, &readfds, (!done_writing ? &writefds : NULL), NULL,
-NULL);
+/* Do EINTR handling here instead of in pipe-filter-aux.h,
+ because select() cannot be referred from an inline function
+ on AIX 7.1. */
+do
+ retval = select (n, &
Re: [PATCH] pipe-filter-gi, pipe-filter-ii: port to AIX
Daiki Ueno writes: > On AIX 7.1, 'select' is defined as static and cannot be referred > from inline function. > * lib/pipe-filter-aux.h (nointr_select): Remove, manually expand > the definition... > * lib/pipe-filter-gi.c (filter_loop): ...here, and... > * lib/pipe-filter-ii.c (pipe_filter_ii_execute): ...here. > --- > ChangeLog | 10 ++ > lib/pipe-filter-aux.h | 19 --- > lib/pipe-filter-gi.c | 16 +++- > lib/pipe-filter-ii.c | 13 ++--- > 4 files changed, 31 insertions(+), 27 deletions(-) Pushed after Paolo's review. Regards, -- Daiki Ueno
[PATCH] uniname/uniname-tests: skip if system's libunistring is used
Hello, When trying to compile an older gettext tarball with libunistring 0.9.5-alpha3 (with an update to Unicode 7.0.0), I encountered a test failure caused by a data mismatch: FAIL: uniname/test-uninames.sh == \u037F name lookup returned wrong name: GREEK CAPITAL LETTER YOT where \u037F is not defined in the test data bundled in the gettext tarball (based on Unicode 5.0.0), but in the system's libunistring. For the meantime I plan to go with the attached a patch, which skips the test when system's libunistring is used. However, I'm not really sure of the relationship between Gnulib modules and system's libunistring. For example, modules/unistr/u32-mbtouc-tests has a line: test_u32_mbtouc_LDADD = $(LDADD) $(LIBUNISTRING) This prefers system's libunistring to the corresponding Gnulib module. I think Gnulib tests should rather prefer Gnulib modules. Does anyone know what's the rationale behind this? Regards, -- Daiki Ueno >From 1125daef9fc55de5af4fd9ebd71cc63a26c5051b Mon Sep 17 00:00:00 2001 From: Daiki Ueno Date: Thu, 27 Nov 2014 16:46:38 +0900 Subject: [PATCH] uniname/uniname-tests: skip if system's libunistring is used The test fails if the system's libunistring defines new characters not in the bundled UnicodeDataNames.txt. * tests/uniname/test-uninames.c (main): Skip the test if the system's libunistring is linked. --- ChangeLog | 8 tests/uniname/test-uninames.c | 5 + 2 files changed, 13 insertions(+) diff --git a/ChangeLog b/ChangeLog index e051495..2db70d0 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,11 @@ +2014-11-27 Daiki Ueno + + uniname/uniname-tests: skip if system's libunistring is used + The test fails if the system's libunistring defines new characters + not in the bundled UnicodeDataNames.txt. + * tests/uniname/test-uninames.c (main): Skip the test if the + system's libunistring is linked. + 2014-11-22 Daiki Ueno pipe-filter-gi, pipe-filter-ii: port to AIX diff --git a/tests/uniname/test-uninames.c b/tests/uniname/test-uninames.c index a11e82b..b154fb6 100644 --- a/tests/uniname/test-uninames.c +++ b/tests/uniname/test-uninames.c @@ -253,10 +253,15 @@ main (int argc, char *argv[]) set_program_name (argv[0]); +#if HAVE_LIBUNISTRING + fprintf (stderr, "Skipping test: system's libunistring is used\n"); + error = 77; +#else fill_names (argv[1]); error |= test_name_lookup (); error |= test_inverse_lookup (); +#endif return error; } -- 2.1.0
Re: [bug-libunistring] [PATCH] uniname/uniname-tests: skip if system's libunistring is used
Pádraig Brady writes: >> This prefers system's libunistring to the corresponding Gnulib module. >> I think Gnulib tests should rather prefer Gnulib modules. Does anyone >> know what's the rationale behind this? > > The tests generally test the combination of system and gnulib bits. > Ideally the test would be split based on features per libunistring version, > but if that's awkward then skipping is fine. Ah, I see. Thanks for the explanation. I'm attaching a patch in that direction (i.e. skipping the test at the Makefile.am level, based on whether the Gnulib module is compiled). Regards, -- Daiki Ueno >From 72d21c164fe1ab24b508f770f436d3ceacbefeec Mon Sep 17 00:00:00 2001 From: Daiki Ueno Date: Thu, 27 Nov 2014 18:46:43 +0900 Subject: [PATCH] uniname/uniname-tests: skip if system's libunistring is used * modules/uniname/uniname-tests (Makefile.am): Skip test if uniname/uniname module is not compiled. --- ChangeLog | 6 ++ modules/uniname/uniname-tests | 2 ++ 2 files changed, 8 insertions(+) diff --git a/ChangeLog b/ChangeLog index e051495..92bcd77 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,9 @@ +2014-11-27 Daiki Ueno + + uniname/uniname-tests: skip if system's libunistring is used + * modules/uniname/uniname-tests (Makefile.am): Skip test if + uniname/uniname module is not compiled. + 2014-11-22 Daiki Ueno pipe-filter-gi, pipe-filter-ii: port to AIX diff --git a/modules/uniname/uniname-tests b/modules/uniname/uniname-tests index a18f942..305e6a1 100644 --- a/modules/uniname/uniname-tests +++ b/modules/uniname/uniname-tests @@ -10,8 +10,10 @@ progname configure.ac: Makefile.am: +if LIBUNISTRING_COMPILE_UNINAME_UNINAME TESTS += uniname/test-uninames.sh check_PROGRAMS += test-uninames test_uninames_SOURCES = uniname/test-uninames.c test_uninames_LDADD = $(LDADD) $(LIBUNISTRING) @LIBINTL@ +endif -- 2.1.0
