[
https://issues.apache.org/jira/browse/FLINK-40248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40248:
-----------------------------------
Labels: github-actions pull-request-available (was: github-actions)
> Move publishing nightly snapshots from AZP to GHA
> -------------------------------------------------
>
> Key: FLINK-40248
> URL: https://issues.apache.org/jira/browse/FLINK-40248
> Project: Flink
> Issue Type: Sub-task
> Components: Build System / CI
> Reporter: Dale Lane
> Priority: Minor
> Labels: github-actions, pull-request-available
>
> Today, the nightly scheduled Azure Pipelines pipeline:
> 1) uploads release binaries to S3
> 2) pushes snapshot jars to Maven
> Before we could migrate fully off of AZP, we need an equivalent nightly
> scheduled job that does this from Github Actions.
> We'll do this in a phased way, one release branch at a time.
> h3. Approach
> Add the equivalent snapshot publishing to the GHA nightly workflow, covering
> both the binary release upload and the Maven snapshot deploy.
> Which pipeline publishes a given branch will be configuration held in the GHA
> nightly workflow itself.
> Because that workflow is versioned per branch, each branch can be switched
> over independently, and a branch that hasn't been switched over yet still
> *builds* its snapshots on GHA (it just keeps them as build artifacts of the
> workflow run rather than publishing them).
> The aim of this is to make sure that the new GHA jobs get exercised on every
> branch well before they become the canonical source of anything we publish.
> The safety property we want throughout is that exactly one pipeline is
> publishing a given branch at any time. Two pipelines publishing the same
> branch would overwrite each other (neither publishing would arguably be
> worse, as a branch's snapshots would silently go stale).
> For each branch, enabling GHA and disabling AZP should be done together, as
> one reviewable change.
> h3. Rollout
> The first branch that we migrate should have an extra safety net step that
> let's us compare GHA's output against AZP's output on a few real nightly runs
> before anything is depending on GHA output.
> # *Both pipelines publish the branch.* GHA publishes to a separate location
> that nothing consumes, so it can't overwrite what AZP is publishing. We
> compare the two sets of output over a series of nightly runs to confirm
> parity.
> # *Hand the branch over.* We make the switch described above, so GHA starts
> publishing to the real location and AZP stops publishing the branch. We then
> verify the published snapshots directly.
> # *Migrate the remaining branches.* With the approach proven end to end, each
> subsequent branch is switched over in a single step - no shadow phase needed.
> h3. Not in scope
> I'm not talking about migrating the nightly *test* jobs here, or the manual
> release process. This is only about the nightly snapshot builds currently
> produced from AZP
> h3. Done when
> This subtask will be complete once every active release branch publishes its
> nightly snapshots from GHA, and AZP no longer does. At that point, the
> nightly snapshot deployment configuration can be deleted from the AZP pipeline
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
