On Fri, Jan 8, 2021 at 10:45 PM Zach Hoffman <zrhoff...@apache.org> wrote:
> One of the biggest ways that Traffic Control benefits from GH Actions is > that workflows run on forks, which only counts towards that user's 20 > runner limit. As a result, our contributors have less cleanup work to do by > the time they open a PR. > How come? Most people in Airlfow simply open PR to Airflow and it counts against the Airflow (thus Apache) queue. How do you do that in Traffic Control ? I'd love to learn that. > > Since running on all forks is not an option with Jenkins, that's where my > preference comes from. Jenkins is still useful for jobs that don't need to > run on forks, (e.g., periodically checking for Go version updates and > opening a PR if a minor version update is found) VAST majority of our traffic comes from PRs from forks. > > -Zach > > On Fri, Jan 8, 2021 at 2:34 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > > Let me just answer from my side. > > > > Personally - I hate and love Jenkins at the same time. I am a Jenkins > user > > and developer for 15 years or so. I even developed mobile app plugins > for > > jenkins and we run our own jenkins farm for mobile app CI. but some 5 > years > > ago we switched to GitLab which was so much modern then. It has not > changed > > since even after jenkins introduced pipelines. > > > > To be perfectly honest - Java and even (once beloved by me) Gradle simply > > sucks for python and (yes!) javascript developers when it comes to > > productivity. > > > > But It has nothing to do with love/hate. > > > > We moved to GA when we had exactly the same troubles with > > Apache-organisation run Travis. more than a year ago we got in very much > > the same situation and GA seemed like an easy win for us. > > > > So, we pioneered GA use in Apache > 1 year ago and it worked well for a > > year or so. > > > > We have a rather complex ( > > https://github.com/apache/airflow/blob/master/CI.rst) system built on > top > > of it that has a great integration with GitHub repo and allows us to do a > > number of things: > > - images are build and share in Github Packages during the build > > - we can directly make comments and push changes to Github Branches > > - we do security analysis with CodeQL > > - we read and update label on PRs that serve us to determine scope of > tests > > ("full tests neeed" = all tests, otherwise only one matrix combination) > > - we push cache images to Github Packages so that our builds run faster > > - we update and modify content of Github Issues to keep track of > > "quarantined" issues - flaky issues that cannot be fixed immediately > > - we implemented a sophisticated 'cancel workflow" (based on my GH > Action) > > that cancels duplicate workflows but also workflows that failed important > > jobs > > - we implemented "selective" checks that determine which of the multitude > > kind of tests (unit, cli, www, heisentests, integration, core, API) > should > > be tested depending on which files changed. > > > > > > All this uses GA features and we implemented it over last year or so. > > Reimplementing it for Jenkins/whatever is probably months of engineering > > time. > > > > And a lot of that requires a lot of custom java/gradle plugins which we > > have pretty much 0 capacity at Airflow. We are Python devs and we can do > > some javascript. I personally did the whole set of gradle plugins for > > mobile app development ( > > https://mvnrepository.com/artifact/com.apphance.ameba/Ameba/0.99.4) but > > that was > 6 years ago and I moved on to Python/Javascript and never look > > back. > > > > J. > > > > > > > > > > On Fri, Jan 8, 2021 at 9:49 PM Chris Lambertus <c...@apache.org> wrote: > > > > > > > > > > > > On Jan 8, 2021, at 12:11 PM, Kamil Breguła < > kamil.breg...@polidea.com> > > > wrote: > > > > > > > I would be happy if we could get a tip on what we can do next to > > improve > > > > our CI as well as keep our community happy. Is there any solution > that > > > > complies with the Apache policy and is more stable? In Airflow, > we've > > > > already tried several solutions, starting with the Travis CI, then > > trying > > > > to use GitLab Ci, and now we're trying to use Github Action. Each > > > solution > > > > has some kinds of problems and prevents us from working freely. Does > > > > anyone have a hint on what we can use now? > > > > > > Have you considered the internal and fully supported ASF Jenkins and/or > > > Buildbot infrastructure? Infra has little control over the free open > > source > > > offerings, but we have significantly more resources we can bring to > bear > > on > > > own on CI systems. > > > > > > What are the gaps in the ASF CI systems that are pushing people onto > less > > > viable platforms such as GA? > > > > > > > > > -Chris > > > ASF Infra > > > > > > > > -- > > +48 660 796 129 > > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
