Let me just answer from my side. Personally - I hate and love Jenkins at the same time. I am a Jenkins user and developer for 15 years or so. I even developed mobile app plugins for jenkins and we run our own jenkins farm for mobile app CI. but some 5 years ago we switched to GitLab which was so much modern then. It has not changed since even after jenkins introduced pipelines.
To be perfectly honest - Java and even (once beloved by me) Gradle simply sucks for python and (yes!) javascript developers when it comes to productivity. But It has nothing to do with love/hate. We moved to GA when we had exactly the same troubles with Apache-organisation run Travis. more than a year ago we got in very much the same situation and GA seemed like an easy win for us. So, we pioneered GA use in Apache > 1 year ago and it worked well for a year or so. We have a rather complex ( https://github.com/apache/airflow/blob/master/CI.rst) system built on top of it that has a great integration with GitHub repo and allows us to do a number of things: - images are build and share in Github Packages during the build - we can directly make comments and push changes to Github Branches - we do security analysis with CodeQL - we read and update label on PRs that serve us to determine scope of tests ("full tests neeed" = all tests, otherwise only one matrix combination) - we push cache images to Github Packages so that our builds run faster - we update and modify content of Github Issues to keep track of "quarantined" issues - flaky issues that cannot be fixed immediately - we implemented a sophisticated 'cancel workflow" (based on my GH Action) that cancels duplicate workflows but also workflows that failed important jobs - we implemented "selective" checks that determine which of the multitude kind of tests (unit, cli, www, heisentests, integration, core, API) should be tested depending on which files changed. All this uses GA features and we implemented it over last year or so. Reimplementing it for Jenkins/whatever is probably months of engineering time. And a lot of that requires a lot of custom java/gradle plugins which we have pretty much 0 capacity at Airflow. We are Python devs and we can do some javascript. I personally did the whole set of gradle plugins for mobile app development ( https://mvnrepository.com/artifact/com.apphance.ameba/Ameba/0.99.4) but that was > 6 years ago and I moved on to Python/Javascript and never look back. J. On Fri, Jan 8, 2021 at 9:49 PM Chris Lambertus <c...@apache.org> wrote: > > > > On Jan 8, 2021, at 12:11 PM, Kamil Breguła <kamil.breg...@polidea.com> > wrote: > > > I would be happy if we could get a tip on what we can do next to improve > > our CI as well as keep our community happy. Is there any solution that > > complies with the Apache policy and is more stable? In Airflow, we've > > already tried several solutions, starting with the Travis CI, then trying > > to use GitLab Ci, and now we're trying to use Github Action. Each > solution > > has some kinds of problems and prevents us from working freely. Does > > anyone have a hint on what we can use now? > > Have you considered the internal and fully supported ASF Jenkins and/or > Buildbot infrastructure? Infra has little control over the free open source > offerings, but we have significantly more resources we can bring to bear on > own on CI systems. > > What are the gaps in the ASF CI systems that are pushing people onto less > viable platforms such as GA? > > > -Chris > ASF Infra -- +48 660 796 129
