Bug#905885: diffoscope: skipping tests on ci.debian.net is perhaps wrong?
Package: diffoscope
Version: 99
Severity: normal
Dear Maintainers,
diffoscope recommends apktool but apktool is not currently in buster. The
test suite of diffoscope currently passes without error because the tests
that use apktool are decorated with @tool_required('apktool').
Consequently:
* the code paths that use apktool are not being tested in buster
* these tests are not actually passing but ci.debian.net is saying that
the test suite has passed (skipping tests to hide errors isn't the
same as a test passing)
There is clear value in being able to skip tests for the casual user on
another platform or when not interested in diffoscope's abilities in
file formats about which one does not care. For the Debian package,
however, I think every functionality of the Debian package should be
tested in autopkgtest. The current situation does not pick up that apktool
is not installable in buster, for instance, and would not pick up
something like /usr/bin/apktool being renamed.
Perhaps @tool_required could be extended with an "all tools required"
mode such that no tests are skipped or missing tools are a test failure
(or error). An environment variable would work for setting that mode
and that mode used at package build time and in autopkgtest.
BTW other latent bugs here:
* diffoscope has an unsatisfiable recommends
* diffoscope cannot be built in buster (unsatisfiable build-deps,
this is RC)
cheers
Stuart
___
Reproducible-builds mailing list
Reproducible-builds@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Bug#905885: diffoscope: skipping tests on ci.debian.net is perhaps wrong?
> Braindumping here whilst I have a second... > > > diffoscope recommends apktool but apktool is not currently in buster. > > ^^ > > Ah, this was the bit I was missing in my head. ie. not a Build-Depends :) both, actually. diffoscope FTBFS in buster because: Build-Depends: apktool , … (which is an RC bug waiting to be filed) *also* diffoscope recommends apktool but tests on ci.d.n do not fail (but should, I think) > > Perhaps @tool_required could be extended with an "all tools required" > > mode such that no tests are skipped or missing tools are a test failure > > (or error). An environment variable would work for setting that mode > > and that mode used at package build time and in autopkgtest. > > Out of curiousity, is there an environment variable that ci.debian.net > reliably exports? You'd set that environment variable in d/rules and d/tests/* (and so could other distros who want to test that packages are working) > (One difficulty with "just" extending @tool_required is that sometimes > we really do want to skip the tests as they require a newer/older > version of a particular tool in order that we can correctly match up > the output.) If the newer/older version doesn't work, surely the test should fail? cheers Stuart -- Stuart Prescotthttp://www.nanonanonano.net/ stu...@nanonanonano.net Debian Developer http://www.debian.org/ stu...@debian.org GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7 ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Bug#913557: jenkins.debian.org: Vary setting TMPDIR during reproducibility testing?
Package: jenkins.debian.org Severity: wishlist Recent discussions on debian-policy about packages obeying (or not) the TMPDIR environment variable made me wonder if it was useful to vary too. Varying TMPDIR perhaps opens a wider discussion about how robust the build should be to the environment. I'd argue that TMPDIR is not an environment variable that is used to control build tools as it is an implementation detail of where files are stored. If a user varies CFLAGS (say), they are deliberately changing the configuration of build tools and get to keep both pieces when it breaks. However, a user can conceivably need to set TMPDIR to manage space used by a build or could have it routinely set for other reasons. I don't think we want build outputs or package contents to depend on the value of TMPDIR. However, one could conceive of situations where some tools honoured TMPDIR while others assume that /tmp is to be used and that varying TMPDIR could change the contents of files, which files are in the packages, or even cause the build to fail. (codesearch.d.n tells me that " /tmp" is used quite a lot in d/rules) cheers Stuart ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#875306: python-debian: include a type for buildinfo files
Dear reproducible-builds folks,
python-debian has classes to wrap many of Debian's deb822 format files, trying
to use an underlying parser that always exposes the data in more or less the
same key/value way via a dictionary, plus also providing extra syntactic sugar
to help make sense of the values that are included. For example, package
dependencies such as Depends or Build-Depends are split up into a list of
relationships, with the standard syntax that we use everywhere interpreted to
include version restrictions etc.
The .buildinfo files have two places where interpreting the values seems
worthwhile:
Environment: split the lines and extract the key="value" data into a
dictionary in the same format as Python's normal `os.environ`. (Some
dequoting is needed but not currently implemented.)
Installed-Build-Depends: use the standard package-relation code on this
to
return interpret the list of packages.
We could thus do something like:
In [1]: from debian import deb822
In [2]: info = deb822.BuildInfo(open("debian/tests/test_BuildInfo"))
In [3]: info.get_environment()
Out[3]:
{'DEB_BUILD_OPTIONS': 'parallel=4',
'LANG': 'en_AU.UTF-8',
'LC_ALL': 'C.UTF-8',
'LC_TIME': 'en_GB.UTF-8',
'LD_LIBRARY_PATH': '/usr/lib/libeatmydata',
'SOURCE_DATE_EPOCH': '1601784586'}
In [4]: info.relations['installed-build-depends']
Out[4]:
[[{'name': 'autoconf',
'archqual': None,
'version': ('=', '2.69-11.1'),
'arch': None,
'restrictions': None}],
[{'name': 'automake',
'archqual': None,
'version': ('=', '1:1.16.2-4'),
'arch': None,
'restrictions': None}],
[{'name': 'autopoint',
'archqual': None,
'version': ('=', '0.19.8.1-10'),
'arch': None,
'restrictions': None}],
[{'name': 'autotools-dev',
'archqual': None,
'version': ('=', '20180224.1'),
'arch': None,
'restrictions': None}],
[{'name': 'base-files',
'archqual': None,
'version': ('=', '11'),
'arch': None,
'restrictions': None}],
...(trimmed)...
In [5]: for dep in info.relations['installed-build-depends']:
...: print("Installed %s/%s" % (dep[0]['name'], dep[0]['version'][1]))
...:
Installed autoconf/2.69-11.1
Installed automake/1:1.16.2-4
Installed autopoint/0.19.8.1-10
Installed autotools-dev/20180224.1
Installed base-files/11
...(trimmed)...
The standard format for the list of package relationships contains features
that the buildinfo format doesn't need ("foo | bar", architecture and build-
profile restrictions), but it seems better to use exactly the same format as is
used for Packages and Sources. That does however mean there are lots of
single-element lists used, as seen in the `dep[0]` usage above. A bit ugly,
but consistency wins here, I think.
How does this look to you?
Are there additional data that would be nice to extract and interpret in a
structured way?
The current code doesn't handle dequoting the environment values and will
react particularly badly to environment values with newlines in them.
The current work-in-progress code is at
https://salsa.debian.org/python-debian-team/python-debian/-/merge_requests/29
Comments, suggestions and encouragement gratefully accepted :)
thanks
Stuart
--
Stuart Prescotthttp://www.nanonanonano.net/ stu...@nanonanonano.net
Debian Developer http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7
___
Reproducible-builds mailing list
Reproducible-builds@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#875306: python-debian: include a type for buildinfo files
Hi Holger
Thanks for having a look and suggesting that additional extension!
On Wednesday, 28 October 2020 06:58:55 AEDT Holger Levsen wrote:
> I believe there is a third place: changelog stanzas (aka
> Binary-Only-Changes:) from binNMUs, like the one from
> https://buildinfos.debian.net/ftp-master.debian.org/buildinfo/2020/10/27/dql
> ite_1.6.0-1+b1_amd64.buildinfo
>
> Binary-Only-Changes:
> dqlite (1.6.0-1+b1) sid; urgency=low, binary-only=yes
> .
>* Binary-only non-maintainer upload for amd64; no source changes.
>* Rebuild on buildd
> .
> -- amd64 Build Daemon (x86-grnet-01)
> Tue, 27 Oct 2020 16:00:36
> +
ah, I'd not seen one of these in action. I should go find some additional
buildinfo files to
play with.
Something like:
In [1]: from debian import deb822
In [2]: info = deb822.BuildInfo(open("debian/tests/test_BuildInfo"))
In [3]: changes = info.get_changelog()
where changes is a debian.changelog.ChangeLog object containing 1 ChangeBlock.
https://python-debian-team.pages.debian.net/python-debian/api/
debian.changelog.html#debian.changelog.Changelog
Thinking about the steps:
* If there is no Binary-Only-Changes field, it would just return None
* It would first remove the initial space indent and dots, raising a ValueError
if they
weren't there.
* Creating the ChangeBlock object might raise
debian.changelog.ChangelogParseError or
debian.changelog.VersionError should the changelog be bad in some way.
Like the accessors for environment and Installed-Build-Depends, this would be a
read-
only method, not providing a simple way to edit/insert the changelog into an
existing
buildinfo file.
Is that what you imagined?
cheers
Stuart
--
Stuart Prescotthttp://www.nanonanonano.net/ stu...@nanonanonano.net
Debian Developer http://www.debian.org/ stu...@debian.org
GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7
___
Reproducible-builds mailing list
Reproducible-builds@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#875306: python-debian: include a type for buildinfo files
Hi Chris Thanks for having a look and offering some feedback! On Tuesday, 27 October 2020 00:04:03 AEDT Chris Lamb wrote: > Thanks for working on this. I can't think of any additional data that > would be useful right this second. However, I tend to have to use the > library in the 'real world' before I can discover that kind of thing. I completely understand and I'm happy to expand the API once real use tests it out a bit. I might mark it as "experimental" in the documentation just in case real use suggests that some breaking changes are needed too. > > The current code doesn't handle dequoting the environment values and will > > react particularly badly to environment values with newlines in them. > > Do you plan to address this? Would be nice if callsites would be able > to rely on the quoting, as you might imagine. Yes, I think it should do so. I will need to recompletely rewrite that bit of code to accommodate some of the weirder possibilities that are permitted, also checking the code in dpkg that generates the file. > Glancing at the parsed data structures, it would seem like the code is > collapsing duplicate environment keys in the returned value of > get_environment() as well as throw away the original ordering. I would > be okay with this, but we don't do the same for the > installed-build-depends relation. Is this deliberate? It is deliberate, although inconsistent as you note. I'm happy to be told me reasoning is not sound and that different structures would be better: * I chose a dict for the environment as order doesn't matter for the environment and there can't be duplicates in the environment anyway. Python's precedent of using a dict for os.environ felt compelling. We could use an OrderedDict here to explicitly preserve order if that desirable (the dict will accidentally preserve order of course). * I chose a list for the dependencies as that is what is already used for all other package relations within deb822._PkgRelationMixin/deb822.PkgRelation. Ordering of a Depends or Build-Depends may or may not actually matter there as that's down in the weeds of the implementation details of how apt would resolve alternate dependencies. In the case of Installed-Build-Depends, the package list is all-encompassing and should be installable without additional resolution (although I expect that might be simpler to say than do); order shouldn't be an issue there, but I prioritised code reuse and consistent data structures within Deb822 so that existing consumers of the structures from _PkgRelationMixin are usable. The code also only currently consumes these data structures with no provision to edit them via the parsed versions (although the Deb822 super-class would let you edit the raw text and reserialise that to make changes). I've written it on the assumption that dpkg would always be the generator of the file and that python-debian is only providing tools to support analysis. cheers Stuart -- Stuart Prescotthttp://www.nanonanonano.net/ stu...@nanonanonano.net Debian Developer http://www.debian.org/ stu...@debian.org GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7 ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#875306: python-debian: include a type for buildinfo files
Hi again I've updated the implementation for consuming buildinfo files: * include a get_changelog() method for the Binary-Only-Changes field * deserialise the environment correctly; dequote \" and handle environment with \n in it https://salsa.debian.org/python-debian-team/python-debian/-/merge_requests/29 (and I realised that I had previously pushed an empty branch to salsa so the actual code was not visible... it is now!) Once again comments, suggestions and encouragement gratefully accepted :) regards Stuart -- Stuart Prescotthttp://www.nanonanonano.net/ stu...@nanonanonano.net Debian Developer http://www.debian.org/ stu...@debian.org GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7 ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#875306: python-debian: include a type for buildinfo files
Hi Chris On Tuesday, 3 November 2020 00:07:03 AEDT Chris Lamb wrote: > Hi Stuart, > > > > Glancing at the parsed data structures, it would seem like the code is > > > collapsing duplicate environment keys in the returned value of > > > get_environment() as well as throw away the original ordering. > > > > It is deliberate, although inconsistent as you note. I'm happy to be told > > me reasoning is not sound and that different structures would be better > > No, I was mostly just checking; you've clearly thought this through. I > think your various solutions are more than adequate, especially as we > don't really define anywhere what happens if any of our fields contain > duplicates anyway. All good :) I'll get this merged and released soonish. Hopefully people will start using this class, thinking about more things that it could do, reporting bugs and offering patches :) regards Stuart -- Stuart Prescotthttp://www.nanonanonano.net/ stu...@nanonanonano.net Debian Developer http://www.debian.org/ stu...@debian.org GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7 ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: Re: Updating dpkg-buildflags to enable reproducible=+fixfilepath by default
Sune Stolborg Vuorela wrote: > Can you provide some kind of hook in the environment so we at least can work > around it in the users, so that the internal functions (where the output of > __FILE__ is forwarded to) can glue e.g. the content of an environment > variable in front of the usage of the __FILE__ content to try to > reconstruct it at runtime, so that all packages doesn't need to do export > SOURCE_BASE_DIR=`pwd`; make test, but it will just work if we patch Qt and > similar libraries ? +1 for the request for the build system to provide for a reference point for where the source is to be found. SOURCE_BASE_DIR has a delightful symmetry to SOURCE_DATE_EPOCH and the algorithm for use is similar: if it is in the environment then use it, if not, fall back to the previous behaviour. At its crudest level, a package's debian/rules could include SOURCE_BASE_DIR= $(shell pwd); however, putting this into the build environment saves source changes in multiple packages, sets us up for future use across the entire ecosystem, signals to upstreams that this is a broad standard and not a mere hack in d/rules, and also saves maintainers from thinking about nasty corner cases to do with current directories with makefile invocation. Looking to the future, tests using SOURCE_BASE_DIR to locate test data would allow build-time tests to be repurposed as autopkgtest tests too, as the autopkgtest could tell the tests where the test input data is to be found. This would be a substantial improvement on the current situation where the tests can only be run at build time and not on ci.debian.net. By dpkg making SOURCE_BASE_DIR 'real', there is a distinct prospect of Debian carrying Qt5 patches to use it (thanks Sune!) and for Qt6 to include them upstream. regards Stuart -- Stuart Prescotthttp://www.nanonanonano.net/ stu...@nanonanonano.net Debian Developer http://www.debian.org/ stu...@debian.org GPG fingerprint90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7 ___ Reproducible-builds mailing list Reproducible-builds@alioth-lists.debian.net https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
Re: python-ansible-pygments: please make the build reproducible
tldr: smells like a dh-python bug - I'll look at it more and reassign
etc later.
Staring at some build logs some more:
* the dirs are generated always
* they get copied from .../.pybuild to ../debian/$package/ always
* they are supposed to get removed by dh_python3
* that removal is not always working
A common theme of the failures is that there are _two_
/usr/lib/python3.11/dist-packages/.foo directories to remove and only
one of them is being removed. For python-ansible-pygments, .pytest_cache
was being removed by dh-python3 but .test-results was not.
Adding in PYBUILD_VERBOSE=1 and some breakpoints into dh-python
(specifically /usr/share/dh-python/dhpython/fs.py), I think there's a
subtle bug about altering `dirs` while inside a loop from `os.walk`:
for name in dirs:
if name in ('test', 'tests') or name.startswith('.'):
log.debug('removing dist-packages/%s', name)
rmtree(join(root, name))
dirs.remove(name)
Removing `name` from `dirs` means that the next item is accidentally
skipped. A classic "don't change the object you're iterating through
while you are iterating through it" issue:
In [1]: L = list(range(0, 10))
In [2]: for i in L:
...: print(i)
...: L.remove(i)
...:
0
2
4
6
8
Which item is not processed in the next iteration by dh_python3 now
depends on the order in which `os.walk` lists the directories. That is
presumably dependent on all manner of things (filesystem? collation?
luck?). On the r-b setup and building by hand I get different results to
within sbuild (and on the buildd).
In sbuild on ext4, `find -type d` (I have memory that this reflects disk
order?) has an order in
./debian/python3-ansible-pygments/usr/lib/python3.11/dist-packages of:
.test-results ansible_pygments .pytest_cache
while building by hand on tmpfs, I get
ansible_pygments .test-results .pytest_cache
For the former, accidentally skipping the directory after the one that
gets removed isn't an issue, but for the latter it is.
--
Stuart Prescott http://www.nanonanonano.net/ stu...@nanonanonano.net
Debian Developer http://www.debian.org/ stu...@debian.org
GPG fingerprint 90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7
___
Reproducible-builds mailing list
Reproducible-builds@alioth-lists.debian.net
https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/reproducible-builds
