Hey Sander, Happy to help.
Side comment: I love the way GitHub actions evolve and I have quite an experience with running and optimizing our builds for Apache Airflow. We are really happy with it especially comparing to Travis CI we used previously. We've been waiting > 1 year (!!!) for GitLab to add running CI jobs for fork pull requests ( https://gitlab.com/gitlab-org/gitlab/-/issues/5667 - the issue is already 2 years old and keeps on being pushed to the next release with every release so we stopped waiting for it. The speed with which Github Actions adds new features is impressive, to say the least, and their public roadmap https://github.com/github/roadmap/projects/1 is even more impressive). I am happy to provide help with my experience (I will subscribe to your devlist now). I am right now in the process of testing a big overhaul of the way how Apache Airflow uses Github Actions. Mostly it is about consistency and optimizations of waiting time + decreasing the pressure on the Github Actions workers we use (we have rather long and comprehensive tests and big matrix of jobs). You can see some (almost completed) work in progress here https://github.com/potiuk/airflow/tree/master/.github/workflows. Along the way I am also finishing up a Github Action that is going to make full use of recently released "workflow_run" type of triggers - that enabled automated canceling of duplicated Pull Request builds from forks, which was a major pain for us (and so far we only workaround it with cancel run scheduled every 5 minutes - which was pretty nasty :(. With the new "workflow_run" it seems possible, though a bit complex, that's why I took an existing cancel-workflow action written by someone else and added full support for workflow_runs (workflow_runs have been added ~ 1 week ago or so so they are rather fresh). The action is here: https://github.com/potiuk/cancel-workflow-runs#cancel-duplicate-jobs-for-triggered-workflow (master version is what I am testing right now and will release v2 shortly with full support for workflow_run but the documentation is already rather comprehensive and contains examples and explanation how things work I am also helping the Apache Beam team in their Github Action migration, so happy to help you as well :). I am also working on handling flaky tests (and sharing experiences with BEam team as well). We've introduced concept of "Quarantined tests" and I am looking into using https://github.com/GoogleCloudPlatform/flaky-service - a service developed by Google Cloud Platform to manage flaky tests (It has Gihub Action integration out-of-the box). You might also find useful the architecture of our integration between Github Actions and DockerHub and the approach we've taken so far (I contributed it to infra some time ago https://cwiki.apache.org/confluence/display/INFRA/Github+Actions+to+DockerHub). It will be slightly updated with the Overhaul I am working on, so I am going to update it shortly, the in-progress version of it is in https://github.com/potiuk/airflow/blob/master/CI.rst J. On Fri, Aug 14, 2020 at 12:00 PM Sander Striker <s.stri...@striker.nl> wrote: > Hi, > > For anyone interested in following along, or providing any guidance or > help: we're looking to migrate BuildStream (see: > https://petri.apache.org/projects) from GitLab to GitHub, including > moving > from GitLab CI to GitHub Actions (https://s.apache.org/3tp4q > ). > > Cheers, > > Sander > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
