*printf string functions without INT_MAX limitations

[Bruno Haible]

As discussed in the thread starting at
,
I'm adding *zs*printf modules, that are like the existing variants
without 'z', except that
  - they support returning results of length > INT_MAX,
  - thus the length type is changed to 'ptrdiff_t' from 'int',
  - an attempt to return a result > PTRDIFF_MAX produces an error
ENOMEM (consistently with malloc, calloc) instead of EOVERFLOW.

For file stream and file descriptor based function, a similar thing
will be done, with 'off64_t' (instead of ptrdiff_t) replacing 'int'.

Here comes the first part of the string functions.

 snprintf -> vzsnprintf
 zsnprintf -> vasnprintf
 sprintf -> vzsprintf
 zsprintf -> vasnprintf
 vasprintf -> vazsprintf
 vazsprintf -> vasnprintf
 vsnprintf -> vzsnprintf
 vzsnprintf -> vasnprintf
 vsprintf -> vzsprintf
 vzsprintf -> vasnprintf
 xvasprintf -> vazsprintf

Note that the last patch, in xvasprintf, also fixes a long-standing bug:
The code was examining errno after vasprintf failed, but vasprintf is
not documented to set errno upon failure.


2024-06-22  Bruno Haible  

xvasprintf: Guarantee a non-NULL result.
* lib/xvasprintf.h: Clarify the programmer's responsibilities.
(xasprintf, xvasprintf): Declare as returning non-NULL.
* lib/xvasprintf.c: Include .
(xstrcat): Allow results longer than INT_MAX characters. Upon size
overflow, invoke xalloc_die.
(xvasprintf): Call vazsprintf instead of vasprintf. When some other
error occurs, emit an error message and abort.
* m4/strerrorname_np.m4 (gl_CHECK_STRERRORNAME_NP): New macro, extracted
from gl_FUNC_STRERRORNAME_NP.
(gl_FUNC_STRERRORNAME_NP): Invoke it.
(gl_OPTIONAL_STRERRORNAME_NP): New macro.
* m4/xvasprintf.m4 (gl_XVASPRINTF): Invoke gl_OPTIONAL_STRERRORNAME_NP.
* modules/xvasprintf (Files): Add m4/strerrorname_np.m4.
(Depends-on): Add extensions, vazsprintf. Remove vasprintf.
* NEWS: Mention the change.

2024-06-22  Bruno Haible  

vasprintf: Make return convention consistent with other modules.
* lib/vasprintf.c: Include .
(vasprintf): If the length is > PTRDIFF_MAX, fail with ENOMEM, not
EOVERFLOW.
* modules/vasprintf (Depends-on): Add stdint.

vazsprintf: New module.
* lib/stdio.in.h (azsprintf, vazsprintf): New declarations.
* lib/vazsprintf.c: New file, based on lib/vasprintf.c.
* lib/azsprintf.c: New file, based on lib/asprintf.c.
* m4/stdio_h.m4 (gl_STDIO_H_REQUIRE_DEFAULTS): Initialize
GNULIB_VAZSPRINTF.
* modules/stdio (Makefile.am): Substitute GNULIB_VAZSPRINTF.
* modules/vazsprintf: New file.

2024-06-22  Bruno Haible  

sprintf-posix: Use vzsprintf.
* lib/stdio.in.h (sprintf): Move specification to here.
* lib/sprintf.c: Don't include , vasnprintf.h. Include
.
(sprintf): Implement based on vzsprintf.
* modules/sprintf-posix (Depends-on): Add vzsprintf. Remove vasnprintf.

zsprintf: New module.
* lib/stdio.in.h (zsprintf): New declaration, based on
lib/sprintf.c.
* lib/zsprintf.c: New file, based on lib/sprintf.c.
* m4/stdio_h.m4 (gl_STDIO_H_REQUIRE_DEFAULTS): Initialize
GNULIB_ZSPRINTF.
* modules/stdio (Makefile.am): Substitute GNULIB_ZSPRINTF.
* modules/zsprintf: New file.

2024-06-22  Bruno Haible  

vsprintf-posix: Use vzsprintf.
* lib/stdio.in.h (vsprintf): Move specification to here.
* lib/vsprintf.c: Don't include , vasnprintf.h.
(SIZE_MAX): Remove macro.
(vsprintf): Implement based on vzsprintf.
* modules/vsprintf-posix (Depends-on): Add vzsprintf. Remove vasnprintf.

vzsprintf: New module.
* lib/stdio.in.h (vzsprintf): New declaration, based on
lib/vsprintf.c.
* lib/vzsprintf.c: New file, based on lib/vsprintf.c.
* m4/stdio_h.m4 (gl_STDIO_H_REQUIRE_DEFAULTS): Initialize
GNULIB_VZSPRINTF.
* modules/stdio (Makefile.am): Substitute GNULIB_VZSPRINTF.
* modules/vzsprintf: New file.

2024-06-22  Bruno Haible  

snprintf: Use vzsnprintf.
* lib/stdio.in.h (snprintf): Move specification to here.
* lib/snprintf.c: Don't include , , vasnprintf.h.
Include .
(snprintf): Implement based on vzsnprintf.
* modules/snprintf (Depends-on): Add stdint, vzsnprintf. Remove
vasnprintf.

zsnprintf: New module.
* lib/stdio.in.h (zsnprintf): New declaration, based on
lib/snprintf.c.
* lib/zsnprintf.c: New file, based on lib/snprintf.c.
* m4/stdio_h.m4 (gl_STDIO_H_REQUIRE_DEFAULTS): Initialize
GNULIB_ZSNPRINTF.
* modules/stdio (Makefile.am): Substitute GNULIB_ZSNPRINTF.
* modules/zsnprintf: New file.

2024-06

*printf string functions without INT_MAX limitations in C locale

[Bruno Haible]

Here come the c-*zs*printf modules, which are like the existing c-*s*printf
modules, except that
  - they support returning results of length > INT_MAX,
  - thus the length type is changed to 'ptrdiff_t' from 'int',
  - an attempt to return a result > PTRDIFF_MAX produces an error
ENOMEM (consistently with malloc, calloc) instead of EOVERFLOW.

Module dependencies:
 c-snprintf -> c-vzsnprintf
 c-zsnprintf -> c-vasnprintf
 c-vasprintf -> c-vasnprintf
 c-vazsprintf -> c-vasnprintf
 c-vsnprintf -> c-vzsnprintf
 c-vzsnprintf -> c-vasnprintf
 c-xvasprintf -> c-vazsprintf

Since I'm replacing the implementation of all these modules, I'm taking
the opportunity to change their licenses from GPL to LGPL or LGPLv2+.
This makes sense because
  - these functions can reasonably be used in libraries,
  - some of them could also be implemented based on *printf_l functions
that exist in the libc of macOS, FreeBSD, NetBSD.
I can do this because
  - I'm basing the new implementation on the modules without 'c-', which
are already under LGPL / LGPLv2+, and
  - the remaining code pieces contributed by Ben Pfaff are below the
10-lines threshold, cf.
.


2024-06-22  Bruno Haible  

c-xvasprintf: Guarantee a non-NULL result.
* lib/c-xvasprintf.h: Clarify the programmer's responsibilities.
(c_xasprintf, c_xvasprintf): Declare as returning non-NULL.
* lib/c-xvasprintf.c: Include , .
(c_xvasprintf): Call c_vazsprintf instead of c_vasprintf. When some
other error occurs, emit an error message and abort.
* modules/c-xvasprintf (Files): Add m4/strerrorname_np.m4.
(Depends-on): Add extensions, c-vazsprintf. Remove c-vasprintf.
(configure.ac): Invoke gl_OPTIONAL_STRERRORNAME_NP.
* NEWS: Mention the change.

2024-06-22  Bruno Haible  

c-vasprintf: Make return convention consistent with other modules.
* lib/c-vasprintf.h (c_asprintf, c_vasprintf): Add specification.
* lib/c-asprintf.c: Replaced with code based on lib/asprintf.c.
* lib/c-vasprintf.c: Replaced with code based on lib/vasprintf.c.
* modules/c-vasprintf (Depends-on): Add stdint.
(License): Change to LGPLv2+.

c-vazsprintf: New module.
* lib/c-vasprintf.h: Change license to LGPLv2+.
Include .
(c_azsprintf, c_vazsprintf): New declarations.
* lib/c-azsprintf.c: New file, based on lib/azsprintf.c.
* lib/c-vazsprintf.c: New file, based on lib/vazsprintf.c.
* modules/c-vazsprintf: New file.

2024-06-22  Bruno Haible  

c-snprintf: Use c-vzsnprintf.
* lib/c-snprintf.h (c_snprintf): Move specification to here.
* lib/c-snprintf.c: Change license to LGPLv2+.
Don't include , , c-vasnprintf.h. Include
, c-vsnprintf.h.
(c_snprintf): Implement based on c_vzsnprintf.
* modules/c-snprintf (Depends-on): Add stdint, c-vzsnprintf. Remove
c-vasnprintf.

c-zsnprintf: New module.
* lib/c-snprintf.h: Change license to LGPLv2+.
(c_zsnprintf): New declaration.
* lib/c-zsnprintf.c: New file, based on lib/zsnprintf.c.
* modules/c-zsnprintf: New file.

2024-06-22  Bruno Haible  

c-vsnprintf: Use c-vzsnprintf.
* lib/c-vsnprintf.h (c_vsnprintf): Move specification to here.
* lib/c-vsnprintf.c: Change license to LGPLv2+.
Don't include , , c-vasnprintf.h. Include
.
(c_vsnprintf): Implement based on c_vzsnprintf.
* modules/c-vsnprintf (Depends-on): Add stdint, c-vzsnprintf. Remove
c-vasnprintf.
(License): Change to LGPLv2+.

c-vzsnprintf: New module.
* lib/c-vsnprintf.h: Change license to LGPLv2+.
(c_vzsnprintf): New declaration.
* lib/c-vzsnprintf.c: New file, based on lib/vzsnprintf.c.
* modules/c-vzsnprintf: New file.

>From 9c456389c96b0532596ea62bcc320f549540d10c Mon Sep 17 00:00:00 2001
From: Bruno Haible 
Date: Sat, 22 Jun 2024 12:21:11 +0200
Subject: [PATCH 1/7] c-vzsnprintf: New module.

* lib/c-vsnprintf.h: Change license to LGPLv2+.
(c_vzsnprintf): New declaration.
* lib/c-vzsnprintf.c: New file, based on lib/vzsnprintf.c.
* modules/c-vzsnprintf: New file.
---
 ChangeLog|  8 ++
 lib/c-vsnprintf.h| 34 +--
 lib/c-vzsnprintf.c   | 66 
 modules/c-vzsnprintf | 27 ++
 4 files changed, 126 insertions(+), 9 deletions(-)
 create mode 100644 lib/c-vzsnprintf.c
 create mode 100644 modules/c-vzsnprintf

diff --git a/ChangeLog b/ChangeLog
index ef21dcc222..0f37171285 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2024-06-22  Bruno Haible  
+
+	c-vzsnprintf: New module.
+	* lib/c-vsnprintf.h: Change license to LGPLv2+.
+	(c_vzsnprintf): New declaration.
+	* lib/c-vzsnprintf.c: New file, based on

*printf obstack functions without INT_MAX limitations

[Bruno Haible]

This series of patches introduces an 'obstack-zprintf' module, that fixes the
INT_MAX limitation (and also the unpredictable errno value in case of failure)
of the 'obstack-printf' module.

I originally planned this set of module dependencies:
 obstack-printf -> obstack-zprintf
 obstack-zprintf -> vasnprintf
but what I did is:
 obstack-printf -> vasnprintf
 obstack-zprintf -> vasnprintf
because it's simpler. (In order to implement obstack-printf on top of
obstack-zprintf, one needs to be able to undo a previous obstack_grow
invocation. That sounds inefficient.) It means that obstack_printf.o
and obstack_zprintf.o will contain essentially the same binary code.
But since a package will usually only use of them, not both, it's not
a problem.


2024-06-22  Bruno Haible  

obstack-zprintf: Add more tests.
* tests/test-obstack-zprintf-big.c: New file, based on
tests/test-vasnprintf-big.c.
* modules/obstack-zprintf-extra-tests: New file.
* modules/obstack-zprintf-tests (Depends-on): Add it.

obstack-zprintf: Add tests.
* tests/test-obstack-printf.h: New file, based on
tests/test-obstack-printf.c.
* tests/test-obstack-printf.c: Include test-obstack-printf.h.
(obstack_chunk_alloc, obstack_chunk_free, test_function): Moved to
tests/test-obstack-printf.h.
(test_obstack_vprintf, test_obstack_printf): Remove functions.
(main): Inline them here.
* tests/test-obstack-zprintf.c: New file, based on
tests/test-obstack-printf.c.
* modules/obstack-printf-tests (Files): Add tests/test-obstack-printf.h.
* modules/obstack-zprintf-tests: New file.

obstack-zprintf: New module.
* lib/stdio.in.h (obstack_zprintf, obstack_vzprintf): New declarations.
(obstack_printf, obstack_vprintf): Tweak comment.
* lib/obstack_printf.c: Parameterize.
(RESULT_TYPE, OBSTACK_PRINTF, OBSTACK_VPRINTF): New macros.
* lib/obstack_zprintf.c: New file.
* m4/stdio_h.m4 (gl_STDIO_H_REQUIRE_DEFAULTS): Initialize
GNULIB_OBSTACK_ZPRINTF.
* modules/stdio (Makefile.am): Substitute GNULIB_OBSTACK_ZPRINTF.
* modules/obstack-zprintf: New file.

2024-06-22  Bruno Haible  

physmem: Fix typo.
* lib/physmem.h (physmem_claimable): Fix typo in comment.

>From aa910f1628d4adcfa967d4ef09bc2e82e6bbba89 Mon Sep 17 00:00:00 2001
From: Bruno Haible 
Date: Sat, 22 Jun 2024 13:05:11 +0200
Subject: [PATCH 1/4] physmem: Fix typo.

* lib/physmem.h (physmem_claimable): Fix typo in comment.
---
 ChangeLog | 5 +
 lib/physmem.h | 2 +-
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/ChangeLog b/ChangeLog
index f81ae03de7..8d28459959 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,8 @@
+2024-06-22  Bruno Haible  
+
+	physmem: Fix typo.
+	* lib/physmem.h (physmem_claimable): Fix typo in comment.
+
 2024-06-22  Bruno Haible  
 
 	c-xvasprintf: Guarantee a non-NULL result.
diff --git a/lib/physmem.h b/lib/physmem.h
index 542dbf214c..9c5438d016 100644
--- a/lib/physmem.h
+++ b/lib/physmem.h
@@ -39,7 +39,7 @@ double physmem_available (void);
aggressivity.
For AGGRESSIVITY == 0.0, the result is like physmem_available (): the amount
of memory the application can use without hindering any other process.
-   For AGGRESSIVITY == 1,0, the result is the amount of memory the application
+   For AGGRESSIVITY == 1.0, the result is the amount of memory the application
can use, while causing memory shortage to other processes, but without
bringing the machine into an out-of-memory state.
Values in between, for example AGGRESSIVITY == 0.5, are a reasonable middle
-- 
2.34.1

>From b216546a9d3e3fd6097d9a8ab3dd1e67a8a68434 Mon Sep 17 00:00:00 2001
From: Bruno Haible 
Date: Sat, 22 Jun 2024 21:31:46 +0200
Subject: [PATCH 2/4] obstack-zprintf: New module.

* lib/stdio.in.h (obstack_zprintf, obstack_vzprintf): New declarations.
(obstack_printf, obstack_vprintf): Tweak comment.
* lib/obstack_printf.c: Parameterize.
(RESULT_TYPE, OBSTACK_PRINTF, OBSTACK_VPRINTF): New macros.
* lib/obstack_zprintf.c: New file.
* m4/stdio_h.m4 (gl_STDIO_H_REQUIRE_DEFAULTS): Initialize
GNULIB_OBSTACK_ZPRINTF.
* modules/stdio (Makefile.am): Substitute GNULIB_OBSTACK_ZPRINTF.
* modules/obstack-zprintf: New file.
---
 ChangeLog   | 13 +
 lib/obstack_printf.c| 18 --
 lib/obstack_zprintf.c   | 20 
 lib/stdio.in.h  | 34 ++
 m4/stdio_h.m4   |  3 ++-
 modules/obstack-zprintf | 26 ++
 modules/stdio   |  1 +
 7 files changed, 104 insertions(+), 11 deletions(-)
 create mode 100644 lib/obstack_zprintf.c
 create mode 100644 modules/obstack-zprintf

diff --git a/ChangeLog b/ChangeLog
index 8d28459959..156dd40138 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,16 @@
+2024-06-22  Bruno Haible  
+
+	obstack-zprin

stdlib: Relax definition of EXIT_FAILURE.

[Collin Funk]

I was looking at the changes in the POSIX 2024 pdf. Historically
EXIT_FAILURE has been allowed to be any non-zero value. Gnulib made
sure that the value was 1 because on Tandem/NSK it is defined to -1.
This leads to issues since the exit status interpreted by 'xargs' is
255 which causes an early exit. Eric Blake reported this and now
EXIT_FAILURE may be any value 1-125 inclusive [1].

Can we relax this definition to allow for any POSIX 2024 compatible
value? I have attached a patch but not pushed it yet. I think it is
best to keep the system definitions as long as they don't cause
strange bugs like the Tandem/NSK case.

Collin

[1] https://www.austingroupbugs.net/view.php?id=1229&nbn=5From e0a0660690e5dd15acb6d98e5e1da1e3909e0221 Mon Sep 17 00:00:00 2001
From: Collin Funk 
Date: Sat, 22 Jun 2024 17:29:48 -0700
Subject: [PATCH] stdlib: Relax definition of EXIT_FAILURE.

* lib/stdlib.in.h (EXIT_FAILURE): Allow definitions in the range 1 to
125 inclusive according to POSIX 2024.
* tests/test-stdlib.c: Make use of #error.
(EXIT_FAILURE): Check for a POSIX 2024 compatible definition.
---
 ChangeLog   | 8 
 lib/stdlib.in.h | 2 +-
 tests/test-stdlib.c | 9 -
 3 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index bf82a188fc..70a71bfa95 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2024-06-22  Collin Funk  
+
+	stdlib: Relax definition of EXIT_FAILURE.
+	* lib/stdlib.in.h (EXIT_FAILURE): Allow definitions in the range 1 to
+	125 inclusive according to POSIX 2024.
+	* tests/test-stdlib.c: Make use of #error.
+	(EXIT_FAILURE): Check for a POSIX 2024 compatible definition.
+
 2024-06-22  Bruno Haible  
 
 	obstack-zprintf: Add more tests.
diff --git a/lib/stdlib.in.h b/lib/stdlib.in.h
index cfc69d0a50..6ab5274831 100644
--- a/lib/stdlib.in.h
+++ b/lib/stdlib.in.h
@@ -186,7 +186,7 @@ struct random_data
with proper operation of xargs.  */
 #ifndef EXIT_FAILURE
 # define EXIT_FAILURE 1
-#elif EXIT_FAILURE != 1
+#elif EXIT_FAILURE < 1 || EXIT_FAILURE > 125
 # undef EXIT_FAILURE
 # define EXIT_FAILURE 1
 #endif
diff --git a/tests/test-stdlib.c b/tests/test-stdlib.c
index cb6db8028f..d2e314b052 100644
--- a/tests/test-stdlib.c
+++ b/tests/test-stdlib.c
@@ -23,13 +23,12 @@
 /* Check that EXIT_SUCCESS is 0, per POSIX.  */
 static int exitcode = EXIT_SUCCESS;
 #if EXIT_SUCCESS
-"oops"
+# error EXIT_SUCCESS should equal 0.
 #endif
 
-/* Check for GNU value (not guaranteed by POSIX, but is guaranteed by
-   gnulib).  */
-#if EXIT_FAILURE != 1
-"oops"
+/* Check that EXIT_FAILURE is defined according to POSIX 2024.  */
+#if EXIT_FAILURE < 1 || EXIT_FAILURE > 125
+# error EXIT_FAILURE should be between 1 and 125 inclusive.
 #endif
 
 /* Check that NULL can be passed through varargs as a pointer type,
-- 
2.45.2

Re: stdlib: Relax definition of EXIT_FAILURE.

[Bruno Haible]

Hi Collin,

> Can we relax this definition to allow for any POSIX 2024 compatible
> value? (1-125)

We should *not* do that.

Our unit tests not only serve to check POSIX or GNU behaviour of
constants, header files, and functions. They also serve to check
assumptions that are so widely used, that things would horribly
break if they were violated.

In this case, if EXIT_FAILURE were 2 or 3 instead of 1 on some
platforms, the *documented* exit codes of the 'diff' or 'grep'
commands or other utilities would not hold any more.

If we ever encounter a platform where EXIT_FAILURE was like this,
we want to know about it quickly, so as to avoid silly bug reports
when some GNU packages are compiled on that platform and don't work
as expected.

If someone wants to create a pure POSIX test suite from the Gnulib
tests, they are welcome to do so. I would also applaud efforts if
some people were to create an interesting POSIX test suite that
is Free Software, with Gnulib bits included. But that is likely
some bigger effort, that the Linux Foundation maybe could host.
(The Open Group will not be in favour of it; AFAIU, they want to
sell their proprietary POSIX test suite for expensive money.)

> I think it is
> best to keep the system definitions as long as they don't cause
> strange bugs like the Tandem/NSK case.

Sure. The point here is that any value != 1 would cause strange bugs
as well.

Bruno

Re: stdlib: Relax definition of EXIT_FAILURE.

[Collin Funk]

Hi Bruno,

Bruno Haible wrote:
> In this case, if EXIT_FAILURE were 2 or 3 instead of 1 on some
> platforms, the *documented* exit codes of the 'diff' or 'grep'
> commands or other utilities would not hold any more.
> 
> If we ever encounter a platform where EXIT_FAILURE was like this,
> we want to know about it quickly, so as to avoid silly bug reports
> when some GNU packages are compiled on that platform and don't work
> as expected.

Sounds good. The situation I was thinking of was a system program
compiled with EXIT_FAILURE being some valid value not equal to 1. If a
GNU program using Gnulib calls that and then checks
"exit_status == EXIT_FAILURE" then it will behave incorrectly. I guess
that problem is annoying but at least it doesn't make the GNU program
disagree with it's associated documentation. The situation I am
thinking of seems rare enough that hopefully it should never occur.

> If someone wants to create a pure POSIX test suite from the Gnulib
> tests, they are welcome to do so. I would also applaud efforts if
> some people were to create an interesting POSIX test suite that
> is Free Software, with Gnulib bits included. But that is likely
> some bigger effort, that the Linux Foundation maybe could host.
> (The Open Group will not be in favour of it; AFAIU, they want to
> sell their proprietary POSIX test suite for expensive money.)

That would be nice, I agree. I think the Linux Standard Base has a
test suite including some POSIX conformance stuff contributed by The
Open Group. Probably out-of-date since, as far as I know, the LSB is
no longer maintained.

Collin