alamb opened a new pull request, #26828: URL: https://github.com/apache/datafusion/pull/26828
## Which issue does this PR close? - Part of #17071 - Follow on to #24792, see https://github.com/apache/datafusion/pull/24792#issuecomment-6446457683 ## Rationale for this change #24792 added a version picker, but publishing a release's documentation is still a manual process: build the tag locally with the release overlay, open a PR against `asf-site` with the result, then open a PR against `main` to add the release to `versions.json`. This automates it so that adding a release to the version picker manifest is all that is needed. It also gives us a way to rebuild already published versions so they pick up theme upgrades and changes to the shared configuration, and to backfill older releases. ## What changes are included in this PR? - `docs/scripts/build_versions.sh`: builds the docs for the releases listed in `docs/source/_static/versions.json` (or the versions passed on the command line) from their git tags, fetching the tags from `apache/datafusion` if the checkout does not have them. Each tag is checked out into a temporary worktree and built with `docs/scripts/release/conf.py` using the Python environment of the current checkout, so every version is rendered with the same theme. Output defaults to `docs/build/html/versions/<version>/` so a local preview of the whole site works. - `.github/workflows/docs_versions.yaml`: runs the script on pushes to `main` that change `versions.json`, the overlay, the script, the workflow itself, or the Python dependencies (which supply the theme), and mirrors the result into `versions/` on `asf-site`. `versions/` therefore matches `versions.json` exactly: a release removed from the manifest is removed from the site. - `.github/actions/setup-docs-builder`: the builder setup (Rust, uv, Graphviz, cargo-depgraph) shared by `docs.yaml`, `docs_pr.yaml` and the new workflow, which now all run on `ubuntu-slim`. - `docs.yaml` and the new workflow share a `publish-asf-site` concurrency group, since both push to `asf-site` and `docs.yaml` falls back to a force push. - `DATAFUSION_DOCS_BASE_PATH` (default `/`): the version picker uses absolute paths, which 404 when a fork serves the site at `https://<user>.github.io/datafusion/`. `conf.py` applies it to the switcher URL and to the `versions.json` written into the build, and the workflows pass it from a repository variable (unset here, so no change for apache). This is what made testing the picker on a fork possible. - `dev/release/README.md`: the "Publish the versioned documentation" step is now "open a PR adding the release to `versions.json`". `docs/README.md`: describe how versioned docs are published and how to test the site on a fork. ## Are these changes tested? Tested end to end on my fork, which serves its `asf-site` branch via GitHub Pages at https://alamb.github.io/datafusion/: - alamb/datafusion#38, alamb/datafusion#39: these changes. Merging rebuilt the two already published versions (55.0.0 and 55.1.0) so their pickers started working. - alamb/datafusion#40: adds 55.2.0 and 54.1.0 to `versions.json`. Merging rebuilt all four versions in about six minutes on `ubuntu-slim` and published them under https://alamb.github.io/datafusion/versions/. A corresponding PR adding 55.2.0 and 54.1.0 to `versions.json` here will follow once this merges. ## Are there any user-facing changes? No changes to the published site contents. Release managers publish versioned docs with a single PR to `main`. <details> <summary>Original request</summary> > We are going to make a draft PR to implement this: https://github.com/apache/datafusion/pull/24792#issuecomment-6446457683 > The end user visible goal is to reduce the manual work required for release > I want a new github action that will rebuild released docs versions so they pick up any changes to the theme and so we have a way to backfill old versions > The action should be triggered on changes to docs/source/_static/versions.json > > I want to test this using my fork of datafusion. First setup my fork with a mirror of what the apache fork does on commit: rebuild the docs. But instead of letting the asf publishing mechanism work, instead configure github pages to serve the new documentation > > I want the jobs to be as close as possible to the current jobs on the apache fork so we can test proposed changes to the apache fork easily there. > > Also disable CI on PRs to the main branch of my fork. No reason to waste CI time. > > This way if I made a PR to my fork that changes something about the docs, it should redeploy to github pages > > Please setup my fork this way. We'll make sure it works, and then we'll make the draft PR to auto rebuild old versions on changes to versions </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
