Hey everyone, I just wanted to share a little accomplishment I (mostly) implemented in Airflow - I just merged the last PR to get fully reproducible builds for all thePython artifacts we produce and publish in downloads.apache.org (python whl, sdist packages, source tarballs).
All our 90 or so artifacts are now fully reproducible and we check reproducibility of them as a mandatory step of PMC verification when voting the releases. Initially I thought it's not THAT needed for us in the Python world, but I got the "let's be reproducible" bug implanted at the "reproducible builds" talk at the ApacheCon in Halifax by Hervé Boutemy and it stuck - until I got it completed. And yeah. It simplified quite a lot our artifact verification and made our process much more robust. Arnout just created this page https://cwiki.apache.org/confluence/display/SECURITY/Reproducible+Builds where we might gather some notes and guidelines around reproducibility - and I added some Python notes and links to our small snippets of making the packages "nicer" reproducible. For example we have now pretty accurate - yet reproducible - dates in the packages as we store source-date-epoch in our repository and bump it automatically as part of our release preparation process. Might be a good idea if others keep their notes there as well to share experiences. As a side note (and maybe sparking a bit of a Java/Maven vs. Python battle), I used to have a bit of an inferiority complex when comparing the build system state in Python - where we did not have so good standards and tooling was proliferated and complex. But as part of preparation there I had to basically move Airflow to the modern packaging world of Python (we accumulated quite some tech debt and had custom solutions there) but I learned that with the approach of separate build frontend and build backends, based on multiple PEP-standards, Python quite leapfrogged the Java world IMHO. It's a bit of a marvel of what the Python Packaging team accomplished in the last few years when it comes to standard adoption and tooling. I am honestly quite impressed with it, and it's just the beginning to be honest. And it makes our contributor's life so much easier - where they can stick to whatever frontend they like (Hatch, poetry, flit, pip, ....) and it seamlessly works with a nicely integrated and customizable build backend of our choice (hatchling in this case). Looking forward to other stories there in the future. J.
