Jacob and Jarek, thanks very much for the pointers! I'll read through and see what else we might need help with.
On Fri, Mar 7, 2025 at 12:12 AM Jarek Potiuk <ja...@potiuk.com> wrote: > Yep. Very much what Jacob wrote. > > If you need any concrete help and examples - I simply think looking at > examples is one of the best ways to learn. > > You might simply look at our https://downloads.apache.org/airflow/ . Also > > https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-sources.html#released-packages > is the description of what kind of artifacts we publish there. Also - if it > might help - here is a description on how we build and release airflow: > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md > , > and 90 providers of ours > > https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDER_PACKAGES.md > . > > As Jacob wrote - how you map your build and release process into what's > really "required" by the ASF is up to you, whatever automation, tooling is > convenient for your team. And ASF requirements are basically about "source > packages" https://www.apache.org/legal/release-policy.html#publication , > having them signed, verified and voted by at least 3 PMC members every time > your release, there is a good deal (and see the recent discussions at > members) about - by default - at least 72 HR voting process for those. This > is the minimum that is required. > > I think we have a strong group of people here - including the tooling team > proficient in Python and packaging that - if you have concrete questions > and suggestions - will be able to share their experience and help you to > work out the right ways - but ultimately it's up to you to decide what's > best for you. > > From last years of working with Python, I would strongly recommend > investing a bit into having a "truly modern" way of packaging. Many past > ways describing how to do packaging and build your Python project, had been > almost turned on its head over the last few years, adn "traditional" ways > of doing things are now replaced by modern standards and tooling and they > provide excellent developer's experience. There are many modern tools like > `uv` (this is my personal recommendation), `poetry` , "hatch", "meson" (the > latter especially popular for projects that need a lot of c extensions) are > building on top of packaging standards > https://peps.python.org/topic/packaging/ that Python Packaging team worked > on for the last 5 or so years. And with `pyproject.toml`, describing your > project metadata and build process as "build backend" (of your choice) and > making your pyproject.toml following the standards and being usable by a > variety of "build frontends" is the way to go. Once you follow those > standards - attracting new contributors (who can choose the frontend of > their choice) while having a standardized build process with backend of > "maintainer choice" is the way to go - especially if you start your python > project from scratch. > > And once you have that - and follow the standards - there are a variety of > integrations (GitHub actions, Jenkins custom builds) and frontend/build > tools that you can very easily plugin into your build process and automate > release preparation and publishing. > > J. > > On Fri, Mar 7, 2025 at 1:27 AM Jacob Wujciak <assignu...@apache.org> > wrote: > > > Hey Andrew, > > > > What exactly are you looking for? Guidance on how to build, release and > > publish a python package with wheels? > > > > Currently the release policy only knows convenience binaries and the only > > requirement is to build them from the tarball. They don't need to be > voted > > on. > > > > So the minimalist python release would be just a tar ball with building > > and publishing of the wheels after the release. Some projects like > Airflow > > and Arrow for example goe a step further and make (some) binaries part of > > the release voluntarily, signing them in the same way the tar ball is > > signed. > > > > The actual build and the difficulty thereof depends on your package, if > > it's pure python it's easy, if you native code I'd recommend > > https://cibuildwheel.pypa.io/en/stable/ but you likely have things like > > that setup for ci. > > > > That should cover everything a little bit :D > > > > Best Jacob > > > > > > On 2025/03/06 23:03:32 Andrew Musselman wrote: > > > Hi Builds, we're cutting a release of Mahout based on python instead of > > > Scala; can someone point me to docs? I see this section on infra for GH > > > actions (https://infra.apache.org/build-supported-services.html#gha) > > but am > > > still missing steps and things to do, like what artifacts need to be > put > > in > > > the release, where does it need to go for voting and checksum > > verification, > > > what signing needs to happen, etc. > > > > > > Thanks in advance! > > > > > > Best > > > Andrew > > > > > >
