On Fri, Jul 19, 2024 at 02:04:43PM -0400, Boyuan Yang wrote:
> X-Debbugs-CC: kniel...@knielsen-hq.org dic...@his.com
>
> 在 2024-07-03星期三的 18:46 -0400，Thomas Dickey写道：
> > On Wed, Jul 03, 2024 at 05:07:10PM -0400, Thomas Dickey wrote:
> > > On Wed, Jul 03, 2024 at 11:03:09AM -0400, Boyuan Yang wrote:
> > > > X-Debbugs-CC: t...@invisible-island.net
> > > >
> > > > 在 2024-07-03星期三的 16:28 +0200，Kristian Nielsen写道：
> > > > > Package: mawk
> > > > > Version: 1.3.4.20240622-1
> > > > > Severity: normal
> > > > >
> > > > > Dear Maintainer,
> > > > >
> > > > > Running mawk produces wrong output from rand() on i386:
> > > > >
> > > > > $ echo | mawk 'BEGIN {srand(10);} {print rand()};'
> > > > > 0.158578
> > > > > $ echo | mawk 'BEGIN {srand(10);} {print rand()};'
> > > > > -0.276944
> > > > > $ echo | mawk 'BEGIN {srand(10);} {print rand()};'
> > > > > -0.387807
> > > > > $ echo | mawk 'BEGIN {srand(10);} {print rand()};'
> > > > > 0.470306
> > > > >
> > > > > The result of rand() should never be negative; and it should be
> > > > > deterministic after calling srand().
> > > > >
> > > > > This occurs on i386 (tested in sbuild --arch=i386 as part of
> > > > > debugging a
> > > > > reproducible build problem of package openscad).
> > > > >
> > > > > The expected output is the same value between 0 and 1, for example:
> > > > >
> > > > > $ echo | mawk 'BEGIN {srand(10);} {print rand()};'
> > > > > 0.559536
> > > > > $ echo | mawk 'BEGIN {srand(10);} {print rand()};'
> > > > > 0.559536
> > > > > $ echo | mawk 'BEGIN {srand(10);} {print rand()};'
> > > > > 0.559536
> > > > >
> > > > > The longer story, for completeness: I am the maintainer for openscad,
> > > > > and I
> > > > > see openscad sometimes failing to build reproducibly on (only) i386
> > > > > during
> > > > > the past year. The root cause seems to be a test script that calls
> > > > > awk and
> > > > > behaves unexpectedly due to getting a negative number out of rand().
> > > > >
> > > > > The problem does not seem to occur on amd64 or arm64. Openscad
> > > > > doesn't build
> > > > > on armhf, so not sure if the problem in mawk is specific to i386 or
> > > > > may be
> > > > > specific to 32-bit for example.
> > > >
> > > > Ack. Reproducible on Debian Sid (mawk/1.3.4.20240622), not reproducible
> > > > on Debian 12
> > > > Bookworm (mawk/1.3.4.20200120). Likely a regression since 2020.
> > >
> > > odd - I can reproduce the problem, but what I'm seeing is older.
> > > I'll investigate further.
> >
> > what I see is a problem with the configurable feature to use the system
> > srand/rand functions. Apparently the relatively new configure check
> > (and availability) for arc4random_stir/arc4random is not being scaled
> > properly.
> >
> > The other bug that I see is that if the system rand/srand is used,
> > then srand has no effect in the script. That is a drawback to the
> > new/improved(?) arc4random functions: their developers chose to not support
> > an
> > equivalent to srand.
> >
> > This bug can be resolved by adding --enable-builtin-srand to the configure
> > options. The issue with repeatability has come up before - see
> >
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=63843
> >
> > but the scaling is actually a bug. The configure script's choice of
> > random functions is a little awkward to work around other than by
> > setting the configure option (actually a small patch to the configure script
> > would work, by changing this line:
> >
> > for cf_func in arc4random_stir/arc4random srandom/random
> > srand48/lrand48 srand/rand
> > to
> > for cf_func in srandom/random srand48/lrand48 srand/rand
> >
> > but I seem to recall that doesn't mesh with dpkg-buildpackage (ymmv).
>
>
> I must be having difficulty understanding the analysis above.
> From what I read, there could be two issues:
>
>
> (1) the default, external, POSIX-compliant(?) implementation of srand()
> call (from libc?) has different behavior than the mawk-internally-implemented
> one, or whatever mawk expects. Switching to the internal one helps solve the
> randomness/reproducibility issue.https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html does say "The arithmetic functions, except for int, shall be based on the ISO C standard", however, implementations of awk do not necessarily match the C srand/rand functions. gawk and original-awk use srandom/random mawk (what we're discussing) would use those if arc4random wasn't found. Older platforms might use srand48/lrand48 or even srand/rand. > > -> However, I don't understand why this is only an issue on i386, while > other platforms (amd64, ...) seem not affected by this problem. > > > (2) The scaling bug of srand() return value should never be negative. > It is a different bug. Thomas proposed a patch to ditch the selection > from arc4random. In the next update, I'll have that as a change to the configure script. I mentioned workarounds/patches in case you get there first. (I have started working on more time-consuming bugs) > -> Still, I don't understand why this is only an issue on i386, while > other platforms (amd64, ...) seem not affected by this problem. prototype says uint32_t arc4random(void); That's because of sign-extension from unsigned numbers in the range 0 to 0xffffffff (and 64-bit machines wouldn't sign-extend that). > -> I noticed the existence of arc4random(3) and srandom(3) but obviously > don't have a good understanding to them yet. > > -> It looks like if we enabled --enable-builtin-srand, both problem (1) > and (2) will be gone. Is that the case? yes - but it's not as good as the srandom/random pair. > I would appreciate it if anyone could give a hint on some more insights, > or what could be done on the Debian side (isn't other distros also > affected by the bug?). I don't mind carrying a patch on aclocal.m4 > if absolutely needed. that's the simplest (and closest) to accomplish the goal of dropping arc2random from mawk. -- Thomas E. Dickey <dic...@invisible-island.net> https://invisible-island.net
signature.asc
Description: PGP signature
