* Guillem Jover <guil...@debian.org> [2024-09-25 01:55]: > Hi! > > On Wed, 2024-09-25 at 00:39:10 +0200, Fay Stegerman wrote: > > * Guillem Jover <guil...@debian.org> [2024-09-24 17:45]: > > > Personally, I think fully migrating from zlib to zlib-ng would sound > > > great (even for trixie), but I guess we can take it slow if you do not > > > feel confident or have concerns over this. > > > > As using an alternative zlib implementation could impact Reproducible Builds > > [1], I would recommend taking that into consideration before deciding on > > this > > kind of change. > > Ah, this is related to something I wanted to mention too and forgot. > > I don't think the specific case you mention is in itself a concern for > Debian, because we only guarantee reproducibility given the same inputs, > which includes the set of packages and their versions that were used > when building the binaries. So if there was a switch, those would end up > being recorded as well, and used when reproducing the outputs. And this > could also happen with a newer version of zlib itself. > > The problem though is, that because the compressed stream is going to > change, that can make certain test suites fail if we perform this > switch, which I think would be the main fallout that we'd see from > this and would need manual fixing, although I assume Fedora has probably > handled most of these already. For example when I added explicit > zlib-ng support to dpkg, I had to fix its test suite to parametrize > sizes for test artifacts.
Whilst it indeed may not affect the reproducibility guarantees for Debian packages themselves, it does affect being able to use a Debian system for Reproducible Builds of other software for which the reference artefacts were built with regular zlib and thus can no longer be reproduced on Debian if that uses a different zlib implementation (so far I've only encountered the reverse, which seems relatively rare -- for now). For example, ZIP files or Android APKs built on a Debian system will have a different compressed stream, like the test files you mention. Which will likely break Reproducible Builds tooling like apksigcopier [1] and reproducible-apk-tools [2]. AFAIK all rebuilders (including my own [3]) for Android APKs use Debian base systems, so this could cause quite a bit of breakage for Reproducible Builds within that ecosystem, which is something I would like to avoid (or at least have a decent workaround for -- e.g. being able to easily choose between multiple zlib implementations during runtime in my Python tooling would be great). As you point out, we've been lucky that zlib has remained backwards-compatible for a long time (even though it doesn't provide any guarantees of that AFAIK). Which also makes me wonder how much more likely zlib-ng might be to produce different compressed streams between different versions or using different hardware (configurations). There might also be issues with reproducibility of Debian packages themselves if e.g. zlib-ng output can differ on different hardware (e.g. number of cores) even with an otherwise identical build environment. At the very least I think it would be good to know how all this could be affected (and how likely things are to remain as stable as zlib has been so far) before making a decision to switch. > I think it would be pretty easy to at least see the extent of this > fallout by performing a mass rebuild for packages build-depending > on zlib1g-dev with a zlib-ng version. - Fay [1] https://tracker.debian.org/pkg/apksigcopier [2] https://github.com/obfusk/reproducible-apk-tools [3] https://github.com/obfusk/rbtlog
