Hi Matthias, Thanks for the write-up and for the efforts on this. I really hope that we can move away from Azure towards GHA for a better integration as well (directly seeing if a PR can be merged due to CI passing for example).
The one thing I'm missing in the FLIP is how we would setup the secrets for the nightly runs (for the S3 tests, potential tests with external services etc). My guess is we need to provide the secret to ASF Infra and then we would be able to refer to them in a pipeline? Best regards, Martijn On Tue, Nov 21, 2023 at 3:05 PM Matthias Pohl <matthias.p...@aiven.io.invalid> wrote: > > I realized that I mixed up FLIP IDs. FLIP-395 is already reserved [1]. I > switched to FLIP-396 [2] for the sake of consistency. 8) > > [1] https://lists.apache.org/thread/wjd3nbvg6nt93lb0sd52f0lzls6559tv > [2] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-396%3A+Migration+to+GitHub+Actions > > On Tue, Nov 21, 2023 at 2:58 PM Matthias Pohl <matthias.p...@aiven.io> > wrote: > > > Hi everyone, > > > > The Flink community discussed migrating from Azure CI to GitHub Actions > > quite some time ago [1]. The efforts around that stalled due to limitations > > around self-hosted runner support from Apache Infra’s side. There were some > > recent developments on that topic. Apache Infra is experimenting with > > ephemeral runners now which might enable us to move ahead with GitHub > > Actions. > > > > The goal is to join the trial phase for ephemeral runners and experiment > > with our CI workflows in terms of stability and performance. At the end we > > can decide whether we want to abandon Azure CI and move to GitHub Actions > > or stick to the former one. > > > > Nico Weidner and Chesnay laid the groundwork on this topic in the past. I > > picked up the work they did and continued experimenting with it in my own > > fork XComp/flink [2] the past few weeks. The workflows are in a state where > > I think that we start moving the relevant code into Flink’s repository. > > Example runs for the basic workflow [3] and the extended (nightly) workflow > > [4] are provided. > > > > This will bring a few more changes to the Flink contributors. That is why > > I wanted to bring this discussion to the mailing list first. I did a write > > up on (hopefully) all related topics in FLIP-395 [5]. > > > > I’m looking forward to your feedback. > > > > Matthias > > > > [1] https://lists.apache.org/thread/vcyx2nx0mhklqwm827vgykv8pc54gg3k > > > > [2] https://github.com/XComp/flink/actions > > > > [3] https://github.com/XComp/flink/actions/runs/6926309782 > > > > [4] https://github.com/XComp/flink/actions/runs/6927443941 > > > > [5] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-395%3A+Migration+to+GitHub+Actions > > > > > > -- > > > > [image: Aiven] <https://www.aiven.io> > > > > *Matthias Pohl* > > Opensource Software Engineer, *Aiven* > > matthias.p...@aiven.io <i...@aiven.io> | +49 170 9869525 > > aiven.io <https://www.aiven.io> | > > <https://www.facebook.com/aivencloud> > > <https://www.linkedin.com/company/aiven/> <https://twitter.com/aiven_io> > > *Aiven Deutschland GmbH* > > Alexanderufer 3-7, 10117 Berlin > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > Amtsgericht Charlottenburg, HRB 209739 B > >
