Btw. even though we've been focusing on GitHub Actions with this FLIP, I'm curious whether somebody has experience with Apache Infra's Jenkins deployment. The discussion I found about Jenkins [1] is quite out-dated (2014). I haven't worked with it myself but could imagine that there are some features provided through plugins which are missing in GitHub Actions.
[1] https://lists.apache.org/thread/vs81xdhn3q777r7x9k7wd4dyl9kvoqn4 On Tue, Nov 21, 2023 at 4:19 PM Matthias Pohl <matthias.p...@aiven.io> wrote: > That's a valid point. I updated the FLIP accordingly: > >> Currently, the secrets (e.g. for S3 access tokens) are maintained by >> certain PMC members with access to the corresponding configuration in the >> Azure CI project. This responsibility will be moved to Apache Infra. They >> are in charge of handling secrets in the Apache organization. As a >> consequence, updating secrets is becoming a bit more complicated. This can >> be still considered an improvement from a legal standpoint because the >> responsibility is transferred from an individual company (i.e. Ververica >> who's the maintainer of the Azure CI project) to the Apache Foundation. > > > On Tue, Nov 21, 2023 at 3:37 PM Martijn Visser <martijnvis...@apache.org> > wrote: > >> Hi Matthias, >> >> Thanks for the write-up and for the efforts on this. I really hope >> that we can move away from Azure towards GHA for a better integration >> as well (directly seeing if a PR can be merged due to CI passing for >> example). >> >> The one thing I'm missing in the FLIP is how we would setup the >> secrets for the nightly runs (for the S3 tests, potential tests with >> external services etc). My guess is we need to provide the secret to >> ASF Infra and then we would be able to refer to them in a pipeline? >> >> Best regards, >> >> Martijn >> >> On Tue, Nov 21, 2023 at 3:05 PM Matthias Pohl >> <matthias.p...@aiven.io.invalid> wrote: >> > >> > I realized that I mixed up FLIP IDs. FLIP-395 is already reserved [1]. I >> > switched to FLIP-396 [2] for the sake of consistency. 8) >> > >> > [1] https://lists.apache.org/thread/wjd3nbvg6nt93lb0sd52f0lzls6559tv >> > [2] >> > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-396%3A+Migration+to+GitHub+Actions >> > >> > On Tue, Nov 21, 2023 at 2:58 PM Matthias Pohl <matthias.p...@aiven.io> >> > wrote: >> > >> > > Hi everyone, >> > > >> > > The Flink community discussed migrating from Azure CI to GitHub >> Actions >> > > quite some time ago [1]. The efforts around that stalled due to >> limitations >> > > around self-hosted runner support from Apache Infra’s side. There >> were some >> > > recent developments on that topic. Apache Infra is experimenting with >> > > ephemeral runners now which might enable us to move ahead with GitHub >> > > Actions. >> > > >> > > The goal is to join the trial phase for ephemeral runners and >> experiment >> > > with our CI workflows in terms of stability and performance. At the >> end we >> > > can decide whether we want to abandon Azure CI and move to GitHub >> Actions >> > > or stick to the former one. >> > > >> > > Nico Weidner and Chesnay laid the groundwork on this topic in the >> past. I >> > > picked up the work they did and continued experimenting with it in my >> own >> > > fork XComp/flink [2] the past few weeks. The workflows are in a state >> where >> > > I think that we start moving the relevant code into Flink’s >> repository. >> > > Example runs for the basic workflow [3] and the extended (nightly) >> workflow >> > > [4] are provided. >> > > >> > > This will bring a few more changes to the Flink contributors. That is >> why >> > > I wanted to bring this discussion to the mailing list first. I did a >> write >> > > up on (hopefully) all related topics in FLIP-395 [5]. >> > > >> > > I’m looking forward to your feedback. >> > > >> > > Matthias >> > > >> > > [1] https://lists.apache.org/thread/vcyx2nx0mhklqwm827vgykv8pc54gg3k >> > > >> > > [2] https://github.com/XComp/flink/actions >> > > >> > > [3] https://github.com/XComp/flink/actions/runs/6926309782 >> > > >> > > [4] https://github.com/XComp/flink/actions/runs/6927443941 >> > > >> > > [5] >> > > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-395%3A+Migration+to+GitHub+Actions >> > > >> > > >> > > -- >> > > >> > > [image: Aiven] <https://www.aiven.io> >> > > >> > > *Matthias Pohl* >> > > Opensource Software Engineer, *Aiven* >> > > matthias.p...@aiven.io <i...@aiven.io> | +49 170 9869525 >> > > aiven.io <https://www.aiven.io> | >> > > <https://www.facebook.com/aivencloud> >> > > <https://www.linkedin.com/company/aiven/> < >> https://twitter.com/aiven_io> >> > > *Aiven Deutschland GmbH* >> > > Alexanderufer 3-7, 10117 Berlin >> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen >> > > Amtsgericht Charlottenburg, HRB 209739 B >> > > >> >
