On Fri, 8 Jan 2021 at 16:04, Jarek Potiuk <ja...@potiuk.com> wrote: > > Github Actions, GitLab, TravisCI. even Cloud Build are soooo much easier > to work with and get your stuff done.
The multiple threads about how shitty those are in practice for your needs seem to indicate otherwise. Security and easy learning curves don't seem to get along too well, do they? > Repos is great. Rewriting all the logic we implemented in our actions ( > https://github.com/apache/airflow/blob/master/CI.rst) where we use not only > the "tasks" but also: > * the image registry > * updating Github issues > * adding labels > * PR status integration > * pushing changes to dedicated branches in our repo > * cancelling duplicated workflows (of 3 kinds) > * selective builds based on what files changed, etc. > > And it is also nicely integrated with 'PR from forks' workflow. > > It would take a major engineering effort to rewrite to Jenkins (if at all > possible). That would all be possible in Jenkins, some of it would be fairly simple to integrate, others would indeed be non-trivial rewrites. > And even if we do so - we have no certainty whatsoever that the number of > our builds will be ok with the capacity. > I was involved in Apache Beam CI infra project where Beam has like 16 > dedicated workers just for them. You can have your own Jenkins controller for your PMC. This is vastly simpler for you to administer than a super time-shared environment like GA. CI systems seem to be a dime a dozen nowadays, yet not a single one seems to implement job scheduling in a sufficiently customizable way that scales. > Even if we discount the months of engineering effort needed, I have no idea > if shared jenkins is enough for us. we run ~50 jobs, some of them 25 > minute or more - for each full build (thanks to selective checks we have a > lot of builds with shorter/smaller number of jobs). It would take a LOT of > time to switch. Definitely valid points. Any CI migrations are non-trivial, especially once you've set up nice workflows. Perhaps there are some alternatives that can help bridge the gap if GA still can't meet your needs. I've seen prow [1] used in various projects in the Kubernetes communities, and I'm sure there must be plenty of others.
