Good find, that will certainly help with the most occurring problem. However, I think the concurrency goups are scoped per repository, so pushing from multiple repositories at the same time will still be problematic?
Den tors 27 aug. 2026 02:00James Daugherty via dev <[email protected]> skrev: > I've waited to respond to this email until the next release was > performed. Looking at the latest rounds, our real problem is the > queue is set to terminate. > https://github.com/apache/grails-core/pull/16231/ will fix the doc > publish for any future staged release. Please see the associated > documentation. > > I am still against changing the github action to retry instead of > using a mutex to lock the values. Retrying will only let those > actions run for longer periods of time, and the right solution is to > have the mutex exist outside of the action of work. I'm happy to > propose solutions for this mutex, but if we've fixed the queue, I > don't see the need to proceed. > > On Fri, Aug 7, 2026 at 9:02 AM James Fredley <[email protected]> > wrote: > > > > Technical Architecture: How Content Publishes to grails.apache.org > > > > Core Model > > ---------- > > > > apache/grails-website (specifically the asf-site-production branch) is > > the single aggregation and publication point for the live site. > > > > Apache Infrastructure watches this branch via the standard ASF site > > publishing mechanism controlled by .asf.yaml: > > > > publish: > > whoami: asf-site-production > > > > Any fast-forward commit that lands on asf-site-production is mirrored by > > Apache Infra to https://grails.apache.org/ (with the usual CDN > > soft-purge behaviour). The repository itself contains no build logic for > > the site; it is purely the destination that multiple independent > > producers push into. > > > > Content Producers > > ----------------- > > > > There are currently two primary independent publishers (with more > expected): > > > > 1. Main website + guides aggregate – apache/grails-static-website > > > > - Gradle-based static site generator (pages, blog, assets, > > templates, guides registry, etc.). > > - Workflow: .github/workflows/publish.yml > > - Triggers: push to master, every 2 hours (cron: '0 */2 * * *'), > > or workflow_dispatch. > > - Has its own repository-scoped concurrency group (publish-${{ > > github.ref }}, cancel-in-progress: false) so only one of its own > > publishes runs at a time. > > - Runs ./gradlew clean publishMainSite. > > - The Gradle task uses environment variables > > (GITHUB_SLUG=apache/grails-website, GH_BRANCH=asf-site-production, > > GH_TOKEN) to push the generated content (root of the site) into the > > shared branch. > > - This is responsible for the majority of the top-level site content. > > > > 2. Versioned documentation – apache/grails-core (via the grails-doc > build) > > > > - Snapshot and release documentation jobs in grails-core (and > > related workflows) build the docs, then invoke the shared action: > > uses: apache/grails-github-actions/deploy-github-pages@asf > > - Key environment variables: > > - TARGET_REPOSITORY: apache/grails-website > > - DOCUMENTATION_BRANCH: asf-site-production > > - TARGET_FOLDER: docs > > - SOURCE_FOLDER: grails-doc/build/docs > > - GRADLE_PUBLISH_RELEASE: true|false > > - The action checks out the target branch, overlays the > > documentation under /docs/ (creating versioned folders such as latest, > > specific release versions, and snapshot), commits a single deployment > > commit, and pushes. > > > > Legacy documentation already lives under docs-legacy-* directories in > > the same branch. Guides are in the process of being fully consolidated > > into the static-website publisher (previously split across several > > locations). Grails Forge is expected to become another publisher > > targeting the same destination. > > > > Shared Publishing Action > > ------------------------ > > > > apache/grails-github-actions (branch asf) provides the reusable > > deploy-github-pages action used by the documentation publishers. It is > > deliberately a thin Git-based overlay: > > > > - Checkout of the target branch into a working tree. > > - Copy of the generated content into the configured folder. > > - Single commit + push. > > > > Because multiple independent workflows (from different repositories) can > > target the same branch tip concurrently, non-fast-forward rejections are > > a normal occurrence. The current work (PR #110) makes the action robust > > to this by: > > > > - Detecting a genuine non-fast-forward rejection. > > - Fetching the new tip into the existing shallow checkout. > > - Confirming the fetched tip is a descendant of the originally observed > tip. > > - Rebasing the unpublished local deployment commit. > > - Retrying a normal (non-force) push (maximum five attempts). > > - Failing closed on true content conflicts or non-descendant history. > > > > No force-push is ever performed. > > > > Why Cross-Repository Coordination Is Required > > --------------------------------------------- > > > > GitHub Actions concurrency groups are scoped to a single repository. A > > group defined in grails-core (grails-docs-publish) cannot coordinate > > with jobs running in grails-static-website (or future publishers such as > > Forge). Serialising everything inside one repository simply creates > > artificial queues and measurable release delays (observed 11–18 minute > > waits, plus manual re-runs and runner contention). > > > > The fundamental problem is concurrent independent commits to a shared > > branch. The correct coordination mechanism is ordinary Git fast-forward > > semantics (fetch + rebase + push), which is exactly what the improved > > action implements. > > > > Resulting Architecture > > ---------------------- > > > > grails-static-website ──publishMainSite──┐ > > │ > > grails-core (docs jobs) ──deploy-github-pages──┼──► asf-site-production > > ──ASF Infra──► grails.apache.org > > │ > > (future: Grails Forge, etc.) ──────────────┘ > > > > The live site is the pure aggregation of whatever the various generators > > last successfully pushed. There is no central “build the whole site” > > job; each producer owns its own content generation and is responsible > > for landing a clean, rebaseable commit on the shared branch. > > > > This is the architecture that the current pair of PRs > > (grails-github-actions#110 + grails-core#16110) is designed to make > > reliable under concurrent load without introducing additional artifact > > storage, coordinating workflows, or force-pushes. > > > > On 8/6/2026 4:12 PM, James Fredley wrote: > > > We have an ongoing issue with concurrent publishing to the grails- > > > website repository. This has affected multiple releases over the past > > > several months in addition to grails-website publishing. Any time two > > > publishing actions are running at the same time. > > > > > > I previously opened https://github.com/apache/grails-github-actions/ > > > pull/98 to make concurrent documentation pushes safe. It was closed in > > > favor of a job-level concurrency group (`grails-docs-publish`) added in > > > grails-core#15988 and then closed again, after re-opening, when it > > > became clear that did not work. That approach serializes independent > > > documentation jobs within Grails Core. Because GitHub Actions > > > concurrency groups are scoped to a single repository, it cannot > > > coordinate publishers from other repositories (grails-static-website, > > > and eventually Grails Forge) that also target the same destination > branch. > > > > > > In practice making this serial has introduced measurable delays during > > > release sequences (for example, an 18-minute wait observed on the each > > > release documentation job, plus manual synchronization, plus manual re- > > > run on grail-website action which failed, plus waiting on github runner > > > availability). This burned 3 hours this morning. > > > > > > The current pair of pull requests restores the ability to publish in > > > parallel while remaining conservative: > > > > > > > > > - https://github.com/apache/grails-github-actions/pull/110 – Safely > > > retry concurrent documentation pushes > > > When another publisher advances the destination branch first, the > > > action recognizes a genuine non-fast-forward rejection, fetches the new > > > tip into the existing shallow checkout, confirms the fetched tip > > > descends from the originally observed tip, rebases the unpublished > local > > > deployment commit, and retries a normal push (maximum five attempts). > > > There is no force-push. Conflicts, non-descendant history, and > unrelated > > > failures fail closed without modifying the remote. > > > > > > - https://github.com/apache/grails-core/pull/16110 – Allow concurrent > > > documentation publishing > > > Removes the repository-local `grails-docs-publish` queue so > > > independent documentation jobs can proceed once the action-level retry > > > is available. Git’s normal fast-forward rules then coordinate updates > to > > > the shared branch. > > > > > > This is a deliberately narrow solution. It covers the cross-repository > > > case, avoids serializing unrelated release work, and fails safely. I am > > > open to any alternative that fully addresses the same constraints > > > (multiple independent publishers, no force-push, safe handling of true > > > content conflicts, and no artificial serialization of independent > jobs). > > > Until a better complete solution is available and verified, these > > > changes allow the project to move forward without further release > delays. > > > > > > An alternate solution has also been proposed of uploading artifacts and > > > then having a separate workflow process them: > > > That approach would introduce additional moving parts (artifact > storage, > > > a new coordinating workflow, hand-off between producers and a consumer, > > > extra latency, and new failure modes) for what is fundamentally a > > > concurrent commit problem on a shared branch. Multiple independent > > > publishers are simply generating documentation and pushing it to the > > > same destination. When another push lands first, the required response > > > is the standard Git sequence: fetch the new tip, rebase the unpublished > > > local commit, and push again. That is exactly what the retry logic in > > > apache/grails-github-actions#110 does, using ordinary fast-forward > rules > > > and failing closed on true conflicts. > > > > > > Let's debate this and get it fixed before the next release cycle, so we > > > can get back to faster releases. > > > > > > James > > >
