Congratulations Jarkek :-) I was glad to see a reference to https://maven.apache.org/guides/mini/guide-reproducible-builds.html
I hope we can keep that Maven page up-to-date with whatever comes up so we only have to look in one place ; -) Gary On Sun, Jan 14, 2024, 2:06 PM Jarek Potiuk <ja...@potiuk.com> wrote: > Hey everyone, > > I just wanted to share a little accomplishment I (mostly) implemented in > Airflow - I just merged the last PR to get fully reproducible builds for > all thePython artifacts we produce and publish in downloads.apache.org > (python whl, sdist packages, source tarballs). > > All our 90 or so artifacts are now fully reproducible and we check > reproducibility of them as a mandatory step of PMC verification when voting > the releases. Initially I thought it's not THAT needed for us in the Python > world, but I got the "let's be reproducible" bug implanted at the > "reproducible builds" talk at the ApacheCon in Halifax by Hervé Boutemy and > it stuck - until I got it completed. > > And yeah. It simplified quite a lot our artifact verification and made our > process much more robust. > > Arnout just created this page > https://cwiki.apache.org/confluence/display/SECURITY/Reproducible+Builds > where we might gather some notes and guidelines around reproducibility - > and I added some Python notes and links to our small snippets of making the > packages "nicer" reproducible. For example we have now pretty accurate - > yet reproducible - dates in the packages as we store source-date-epoch in > our repository and bump it automatically as part of our release preparation > process. Might be a good idea if others keep their notes there as well > to share experiences. > > As a side note (and maybe sparking a bit of a Java/Maven vs. Python > battle), I used to have a bit of an inferiority complex when comparing the > build system state in Python - where we did not have so good standards and > tooling was proliferated and complex. But as part of preparation there I > had to basically move Airflow to the modern packaging world of Python (we > accumulated quite some tech debt and had custom solutions there) but I > learned that with the approach of separate build frontend and build > backends, based on multiple PEP-standards, Python quite leapfrogged the > Java world IMHO. It's a bit of a marvel of what the Python Packaging team > accomplished in the last few years when it comes to standard adoption and > tooling. I am honestly quite impressed with it, and it's just the beginning > to be honest. And it makes our contributor's life so much easier - where > they can stick to whatever frontend they like (Hatch, poetry, flit, pip, > ....) and it seamlessly works with a nicely integrated and customizable > build backend of our choice (hatchling in this case). > > Looking forward to other stories there in the future. > > J. >
