Re: "printf %d ''" should diagnose the empty string

[Chet Ramey]

On 2/7/25 10:21 PM, Martin D Kealey wrote:

Hi Chet

We seem to have very similar opinions about strong backwards compatibility in theory, and yet somehow we keep butting heads on how that pans out in practice. I'm concerned that the last ten years has seen a number of Linux distributions /stop/ including Bash by default, and it has ceased to be the language of choice for writing new scripts; for the most part that's now either Python, Node.js, or POSIX sh (dash). I now wonder whether Bash has much of a future. Each breaking change pushes Bash ever closer to becoming an irrelevant anachronism. Conversely, changes that make it less error prone or easier to use somewhat pull it back from the brink of oblivion.

Taking another run up this hill, eh? This is very similar to

https://lists.gnu.org/archive/html/bug-bash/2023-11/msg00150.html

even down to the professed concern about bash becoming irrelevant.

My key point is that an empty string /meaning/ zero can arise quite ordinarily from /not/ treating zero as a special case.

There it is.

  * printf '%u\n' "${x##*(0)}"   # make sure x is not interpreted as octal

The proposed change in 5.3-alpha won't transform any broken programs into working ones, but it will break (at least a few) working programs. For example:

As long as you have extglob enabled somewhere to make this work.

It is irrelevant whether it is possible to rewrite this to avoid the proposed warning; what matters is that correctly working scripts will have to be audited (and potentially modified) if this change goes ahead.

So if something worked only by chance -- not necessarily this, so we can
remove the emotion from the discussion -- and bash was changed to fix a
bug, or add a feature, it would be a problem if bash did that? It seems
like you only want the bugs fixed that don't break your code. Now, that's
fine -- self-interest is a powerful thing -- but it doesn't take into
account other users who want the bug fixed, does it?

I also think it is unreasonable to make printf inconsistent with other parts of Bash that /do/ (silently) allow empty to mean zero, including:

  * ${var:$empty:$length} or ${array[@]:$empty:$length};
  * $((empty));

This is not an arithmetic context like those. If you want the arguments to
printf %d to be treated as if they're in an arithmetic context, wrap them
in $(( ... )). (And note that the behavior of a missing expression in
$(( )) is a bash extension; the bash/mksh/ksh93 behavior is not shared by
the yash/ash-family shells).

Conversely, while we're looking at printf, why not produce warnings for:

  * negative arg for %u or %.*… precision;

No one warns on a negative arg for %u, and there's so much mixed behavior
with a negative precision argument that it's not worth warning about, since
POSIX explicitly says "A negative precision is taken as if the precision
were omitted."

  * spurious or repeated flags like %--9d (or %-*… with a negative width arg);

"A negative field width is taken as a '-' flag followed by a positive field
width."

and POSIX doesn't say anything about repeated flags being invalid, just
that zero or more flag characters may appear in any order.

  * the number of args to printf not being an whole multiple of the number
    of required args;

Why warn about this? POSIX specifies the behavior in this case of missing
or extra args, and it's quite useful.

  * numeric overflow in $((expression)).

Bash doesn't warn about overflow anywhere.

For that matter, why /only/ a warning? If the point is to detect bugs in scripts, surely Bash should crash out, like errexit or nounset?

Because POSIX says

"If an argument operand cannot be completely converted into an internal
value appropriate to the corresponding conversion specification, a
diagnostic message shall be written to standard error and the utility shall
not exit with a zero exit status, but shall continue processing any
remaining operands and shall write the value accumulated at the time the
error was detected to standard output."

So if you're running in errexit mode you'll exit like you expect. I
encourage you to try something like

printf '%d\n' 12345xyz
echo after: $?

when running with errexit enabled and not.

If that's not the intent, please publish the rationale that separates the allowed and prohibited cases.

You should be able to figure that out from this discussion.

Right up front the manual says:

  * /Bash is intended to be a conformant implementation of the Shell and
    Utilities portion of the IEEE POSIX specification (IEEE Standard 1003.1)./

Of course, that's not the only reason why I think that adding this warning would be the wrong approach: This proposed change would appear to contradict this intent, since it makes the default mode less conformant,

https://lists.gnu.org/archive/html/bug-bash/2024-11/msg00156.html

POSIX says <https://pubs.opengroup.org/ onlinepubs/9799919799/utilities/printf.html>:

  *

    11. If an /argument/ operand to be consumed by a conversion
    specification does not exist:

      o

        /[…]/

      o

        If it is an unnumbered argument conversion /[…]/ any other /[not b,
        c, or s]/ extra conversion specifiers *shall be evaluated as if a
        zero argument were supplied*.

This is not at all the same as an empty string, and you know it.

from which it follows that no diagnostic should be printed. While POSIX doesn't directly specify what happens when converting an empty-string to a number, it does seem very strange indeed to treat "missing" more harshly than "empty".

An empty string is not a valid "unsuffixed C integer constant," nor does
strtol accept it.


 Reinforcing this, the examples section says:

  * The /printf/ utility is required to notify the user when conversion
    errors are detected while producing numeric output; thus, the following
    results would be expected on an implementation with 32-bit two's-
    complement integers when %d is specified as the /format/ operand:

which is then followed by a table of examples that does /not/ include the empty string.

There is an infinite number of examples not included.


It then says:

  * The diagnostic message format is not specified, but these examples
    convey the type of information that should be reported. Note that the
    value shown on standard output is what would be expected as the return
    value from the /strtol/() function as defined in the System Interfaces
    volume of POSIX.1-2024.

It doesn't suggest a diagnostic along the lines of “empty arg treated as 0”, and if the correspondence with strtol() is to be taken at face value, this would likewise imply that empty string is a valid representation for zero, since strtol() reports that it has converted the entire string in that case.

Now, I know you know that's not the case.

"If the subject sequence is empty or does not have the expected form, no
conversion is performed; the value of nptr shall be stored in the object
pointed to by endptr, provided that endptr is not a null pointer."

It returns 0, to be sure, because "If no conversion could be performed, 0
shall be returned [CX] and errno may be set to [EINVAL]."

Bash seems to lack any document providing a clear rationale for which deviations from POSIX should be considered "bugs" vs "enhancements", which means that users cannot anticipate which "enhancements" might some time in the future be reclassified as "bugs" and taken away.

If something is listed in the "POSIX Mode" section of the manual, it's a
deliberate deviation. But bash may become more posix-conformant in the
future.

I'm also confused: why did you yield to my argument to allow «$((0x$empty))

Yeah, in retrospect that was a bad idea.

» but decided to go ahead with adding a warning about «printf %s "$empty"», when both cases for retaining the existing behaviour rely on essentially the same arguments?

The arguments to printf %d are more constrained than bash's arithmetic
expression evaluation.

    but I very much doubt that the practice of writing 0 as '' in an
    argument to printf is much used.

Please stop with the strawmen. Nobody literally writes printf %d '' - that would be silly, and a byte longer than necessary. But we can write «printf 'foo=%d\n' "$foo"» where foo is the result of a previous string manipulation that could yield an empty (not unset) result.

So you want a non-integer argument treated as an integer.

Failing all that, if you're dead-set on having this, please (a) send the warning to BASH_XTRACEFD, so that it can be suppressed without redirecting stderr;

No, that has nothing to do with BASH_XTRACEFD.

and (b) insert into the manual an explicit description of at least one expansion explaining that will reliably result in an empty string being interpreted as numeric zero, so that we can trust that one won't be broken in future.

I think it's reasonable to say that $(( expression )) evaluates to 0 if
`expression' is null or empty, or to add that to the ARITHMETIC EVALUATION
section.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    c...@case.edu    http://tiswww.cwru.edu/~chet/

OpenPGP_signature.asc
Description: OpenPGP digital signature