Hello everyone, TL;DR; I would like to hear what you think about using GitHub Actions as a Trusted Publisher to publish Python packages to PyPI?
A bit more context: I've been involved in a few discussions recently about Python / PyPI trusted publisher [1] integration for Apache Airflow and other Python projects we have. Trusted Publisher - in short - is the mechanism PyPI introduced where packages can be published to PyPI not via individual's API keys, but via sigstore-signed OIDC provider generated release automation mechanisms. This is all part of securing the supply chain ecosystem, and mostly what it does is that it exchanges the authentication from those of individual API keys to short living keys generated by some identity provider. Currently it does not do much more - there is no signing or verification of artifacts, it merely allows publishing them via a trusted publisher that can prove the identity of the person uploading it. Initially we thought about turing the ASF into such trusted publisher - we already have OIDC need to do so so theoretically it is possible, but after some discussions (including Python's security developers in residence - Seth Larson and PyPI safety and security engineer Mike Fiedler - and we all agree that this is a bit red-herring to chase. The main assumption about "trusted publisher" is that it provides identity services to anyone that is on their platform and that it is open to the public. The current Trusted Publishers are GitHub Actions, GitLab, Google and Active state.. List of trusted publishers to choose from is available when you are signing your project to use Trusted Publisher of your choice [2]. And the list of Trusted Publishers to choose from is visible to everyone who maintains any project. On the other hand - the ASF is more of a "private" organisation - where you cannot randomly choose the ASF to be your trusted publisher. Only the existing PMCs could do so, and - as a general public - you cannot ask for an account where your project will be hosted. So it would be awkward to see ASF in the list of trusted publishers for all PyPI projects [2]. So in order to turn the ASF into a trusted publisher, a new entity type and workflows would have to be implemented by the PyPI team. That will take years. But.... Instead - we could actually use Github Actions as a Trusted Publisher for Python projects like **now** (for projects that use GitHub). It's just a matter of configuration and plugging in a GitHub Action to publish the artifacts in Github "release" workflows. Especially in cases that our releases are binary reproducible, such Github Action could even verify if the artifact is exactly the same as the one published via " downloads.apache.org" (or even download it from there) and publish it in PyPI. In this case we would exchange the long-lived API keys of release managers with short lived tokens generated for GitHub for maintainers of the project. IMHO it does not have any real danger nor it does not bind ASF with GitHub any more that we already are bound to. The configuration is not permanent (we can always change it in the future - either back to use the API keys or - swap it to use trusted publishing into Artifact Distribution Platform and **really** turn ASF into a Trusted Publisher and add Trusted Publishing capability. In the meantime - we could get the increased safety of publishing - because the trust would be based on the security of GitHub accounts of the maintainers (which we already trust to submit and merge code). And the PMC can limit who can actually run the publishing workflow, to only limit it to release managers. We already do that when we are publishing our Docker Container images in dockerhub. The publishing workflow for PyPI will be very similar to that one: https://github.com/apache/airflow/blob/main/.github/workflows/release_dockerhub_image.yml#L81 The added (surprising) benefit of it is that it actually allows INFRA to manage access to PyPI (via managing Github projects which are actually already managed by INFRA). For example it will give infra more control in case of any "bad actors" - removing someone from the project's maintainer list in GitHub will automatically revoke their ability to publish the packages. Currently, access in PyPI is managed independently from INFRA - each PyPI project is managed independently by the PMC. I wonder what you think of that? Are there any drawbacks or blockers I have not taken into account? J. [1] https://docs.pypi.org/trusted-publishers/ [2] https://docs.pypi.org/trusted-publishers/adding-a-publisher/#github-actions
