> > > > > The multiple threads about how shitty those are in practice for your > > needs seem to indicate otherwise. Security and easy learning curves > > don't seem to get along too well, do they? >
The usabilty, integration level (especially GitHub Actions), maintenance effort needed - thi is far, far superior. If only we could solve one simple problem - securely running the self-hosted runners for GA - all our problems are solved INSTANTLY. Security issues happen everywhere, at least if they happen in such services you can mitigate (we just did it in Airflow- we mitigated all the security issues we found), open bounty requests (I did - I opened two bounty requests) and then escalate. If I do not hear about my 2 security bounties from GitHub shortly, I am going to start a hell of a social media campaign about it using all the means I can. I tried to responsibly disclose it but I am going to write a nice blog post about "How to exploit Github Actions" and I am going to tell them that before I publish it and give them a chance to fix it. So you have many ways to influence the security of public services like that. I think it's much better than when you have to manage security yourselves. > > > > That would all be possible in Jenkins, some of it would be fairly > > simple to integrate, others would indeed be non-trivial rewrites. > > > Yep. The non-trivial ones I am afraid of. It took me a year to perfect and optimise a number of steps in our CI and the problem was - it worked really well until it stopped because of uncontrolled increase of usage from other projects and no secure way to add extra resources needed (even if we have all the funds - we now have 8000 USD secured from our stakeholders - with outlook for more) to run those. But if you add the engineering effort needed to migrate, the engineering time for that costs FAR more than just that - enabling compute resources to use all the engineering efforts you've already spent. This is no brainer which way is simpler, cheaper and can be done faster. We just need to have a secure way of doing it. > > > > > > You can have your own Jenkins controller for your PMC. This is vastly > > simpler for you to administer than a super time-shared environment > > like GA. CI systems seem to be a dime a dozen nowadays, yet not a > > single one seems to implement job scheduling in a sufficiently > > customizable way that scales. > I do not want nor need to administer my CI. And I've done that many times in the past - Jenkins, Bamboo, GitLab - you name it. Heck - I built and maintained my first custom CI framework for my company some 20 years ago when the "CI" was just being coined. With CI as a service, I do not want to do it anymore. At all. CI for me should just 'be there'. Great CI is one that you are not aware of its existence until your test fail - and even then you just want to see the logs of your failed tests and figure out the reason This is what you want from CI system. I do not want to learn how to manage Jenkins, install plugins, configure that etc. This is not my job, nor any one in our project. This requires far more than just setting it up - it is making sure that it is secure, that it runs 24/7, that it gets updated etc. etc. This is far more complex than 'just use CI'. We have enough trouble with setting up and maintaining runners (once we get them securely connected). I know it looks differently from the infrastructure person point of view - running Jenkins is pretty much core part of what you do. But for project developers, CI should just 'work'. This is what I get from GitHub Actions. It just 'works'. I have to spend 0 effort to maintain it. Sometimes when it does not work, it pains, but then it's their problem to fix - and they have to fix it eventually because they get pressure from all their customers. In case I run my own jenkins install and administer it - all those problems fall on us. I do not want that. This moves us away from doing what we should - develop our product. > > > > Definitely valid points. Any CI migrations are non-trivial, especially > > once you've set up nice workflows. Perhaps there are some alternatives > > that can help bridge the gap if GA still can't meet your needs. I've > > seen prow [1] used in various projects in the Kubernetes communities, > > and I'm sure there must be plenty of others. > GA meets all my needs. Except one that I am asking ASF to help with - make GitHub focus on making a secure way of working with self-hosted runners. That's it . We even (In November) opened a PR to Github Actions Runner to enable it: https://github.com/actions/runner/pull/783 But we have not heard anything since. This is what I ask INFRA to help with - put pressure on GitHub to make it happen. I need nothing more - no money, no Jenkins, nothing like that. I just want to be able to spend the money we managed to secure. J. -- +48 660 796 129
