Hi Ludo,
On Sat, 16 Nov 2019 at 17:12, Ludovic Courtès <l...@gnu.org> wrote:
> zimoun <zimon.touto...@gmail.com> skribis:
>
> > My intuition is: the non-determinism comes from the WAF configuration
> > files (./waf or wscript or wafadmin/).
> >
> > The process conditionally enters in functions such as "do_install" or
> > "symlink_as", so maybe something is hidden by Python module "os" or
> > similar. Moreover, note that wscript:l.990 "add_group()" is used after
> > "add_subdirs('libgxw/gxw')".
>
> My intuition :-) is that waf traverses files using directly
> opendir/readdir, which returns files in an order that’s file
> system-dependent. That, in turn, leads it to make .so a symlink or not
> in a non-deterministic fashion.
Yes, it should come from the function 'os.listdir' (readdir). The
documentation [1] says: "The list is in arbitrary order."
[1] https://docs.python.org/2/library/os.html#os.listdir
The function 'os.listdir' is called some times ;-)
--8<---------------cut here---------------end--------------->8---
$ egrep -r "os\.listdir\("
wafadmin3/py3kfixes.py: for x in os.listdir(os.path.join(dir,y)):
wafadmin3/Scripting.py: lst=os.listdir(cwd)
wafadmin3/Scripting.py: if WSCRIPT_FILE in os.listdir(calldir):
wafadmin3/Scripting.py: dirlst=os.listdir(cwd)
wafadmin3/Scripting.py: names=os.listdir(src)
wafadmin3/Scripting.py: lst=os.listdir('.')
wafadmin3/Tools/qt4.py: lst=os.listdir('/usr/local/Trolltech/')
wafadmin3/Tools/javaw.py: lst=os.listdir(path)
wafadmin3/Utils.py: return os.listdir(s)
tools/check_rpc: for f in os.listdir(basedir):
src/gx_head/builder/make:for f in os.listdir(__dir__):
wafadmin/py3kfixes.py: for x in os.listdir(os.path.join(dir,y)):
wafadmin/Scripting.py: lst=os.listdir(cwd)
wafadmin/Scripting.py: if WSCRIPT_FILE in os.listdir(calldir):
wafadmin/Scripting.py: dirlst=os.listdir(cwd)
wafadmin/Scripting.py: names=os.listdir(src)
wafadmin/Scripting.py: lst=os.listdir('.')
wafadmin/Tools/qt4.py: lst=os.listdir('/usr/local/Trolltech/')
wafadmin/Tools/javaw.py: lst=os.listdir(path)
wafadmin/Utils.py: return os.listdir(s)
--8<---------------cut here---------------end--------------->8---
And do not talk about the function 'walk' which internally use 'listdir'. :-)
I have no idea how to address this issue.
> It would also be worth checking what others involved in the Reproducible
> Builds effort have done (Debian, openSuSE, etc.).
Lot of sun for Debian [2] ;-)
I mean from what I understand, they do not find any reproducibility
issue and they apply only this patch [3].
[2]
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/guitarix.html
[3] https://sources.debian.org/patches/guitarix/0.36.1-1/02-libdl.patch/
Do not know about openSuSE.
Cheers,
simon
