Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-10 Thread Jarek Potiuk
Happy to close the thread. However, Is there any way we can track the progress of what's happening? I am happy to help and do whatever needed to move it forward but the number of "I do not know" makes it difficult, especially at the time like that when it hits us hard and we do not have any other

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-10 Thread Gavin McDonald
Hi All, 1. Infra has internal contacts. Infra is in contact with GH. 2. Github is a Sponsor, kindly donating their services for free, let's not abuse that fact. 3. This is a PUBLIC mailing list and quite frankly some of this discussion should not be taking place here, please move to

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-10 Thread Dave Fisher
Jarek, I would suggest you have a direct chat with Greg Stein. Best Regards, Dave Sent from my iPhone > On Jan 10, 2021, at 9:08 AM, Jarek Potiuk wrote: > > On Sun, Jan 10, 2021 at 5:28 PM Matt Sicker wrote: > >> If we can get GA to handle our use case properly, that would be awesome. >> B

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-10 Thread Matt Sicker
Do we have anyone here who works at GitHub that could potentially help? An internal contact would be the easiest (e.g., I was that initial contact point to get us the CloudBees CI version of Jenkins here to ease our own Jenkins usage problems). Otherwise, I'd imagine that INFRA are likely to be imp

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-10 Thread Jarek Potiuk
On Sun, Jan 10, 2021 at 5:28 PM Matt Sicker wrote: > If we can get GA to handle our use case properly, that would be awesome. > Being in the security engineering domain, though, I’m generally pessimistic > about security, so please excuse my cynicism. > I totally understand. i am a bit more opti

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-10 Thread Matt Sicker
If we can get GA to handle our use case properly, that would be awesome. Being in the security engineering domain, though, I’m generally pessimistic about security, so please excuse my cynicism. On Sun, Jan 10, 2021 at 03:43 Jarek Potiuk wrote: > I have a feeling (though I cannot know for sure)

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-10 Thread Jarek Potiuk
I have a feeling (though I cannot know for sure) that you are underestimating the power of an organization like ASF in actually 'stating' their requirements and 'expectations' towards GitHub. I am now an engineer, but I used to be CTO, CEO, Head of IT, Head of Technology and I know that a lot can

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-09 Thread Matt Sicker
I work on the Jenkins security team. We don’t have embarrassing security failures like this anymore, but part of that is due to the added complexity of a secure configuration. By the time GA meets your security standards, it’ll likely either require non-trivial changes to your CI scripts, or it’ll

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-09 Thread Jarek Potiuk
And BTW. The change we (Ash) proposed in November is very, very reasonable one. It does not require GitHub to change any of the architecture for security, nor any of the components besides runner configuration. I will quote Ash here as I would not be able to put it better. Antoine, please take a l

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-09 Thread Antoine Pitrou
Le 09/01/2021 à 12:01, Jarek Potiuk a écrit : > > So if only we had 'approved', "secure" and easy way of running our own > self-hosted runners + a way from Github to distribute the free resources in > a fair way among the project. - the problem would be immediately solved. > This is really what

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-09 Thread Jarek Potiuk
> > > > > The multiple threads about how shitty those are in practice for your > > needs seem to indicate otherwise. Security and easy learning curves > > don't seem to get along too well, do they? > The usabilty, integration level (especially GitHub Actions), maintenance effort needed - thi is fa

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-09 Thread Jarek Potiuk
> > > > I presume you are referring to the fact that external non-committers > cannot force a build on a forked repo PR on the ASF Jenkins without > whitelisting [1]. This is by design because it is a huge security problem > to run unvetted 3rd party code on our build infrastructure. This is the >

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-09 Thread Jarek Potiuk
> > > > The ASF does not run Travis. We pay a large amount of money TO Travis for > additional workers on top of the free tier. The reason I ask for discussion > regarding Jenkins usability is because throwing infinite money at solutions > like Travis is not an option. Now you find yourself in the

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-08 Thread Chris Lambertus
> On Jan 8, 2021, at 1:45 PM, Zach Hoffman wrote:\ > Since running on all forks is not an option with Jenkins, that's where my > preference comes from. Jenkins is still useful for jobs that don't need to > run on forks, (e.g., periodically checking for Go version updates and > opening a PR if

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-08 Thread Chris Lambertus
> On Jan 8, 2021, at 1:33 PM, Jarek Potiuk wrote: > > We moved to GA when we had exactly the same troubles with > Apache-organisation run Travis. more than a year ago we got in very much > the same situation and GA seemed like an easy win for us. The ASF does not run Travis. We pay a large

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-08 Thread Zach Hoffman
> One of the biggest ways that Traffic Control benefits from GH Actions is > > that workflows run on forks, which only counts towards that user's 20 > > runner limit. As a result, our contributors have less cleanup work to do > by > > the time they open a PR. > > > > How come? Most people in Airlfo

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-08 Thread Matt Sicker
Oops, forgot the footnote: [1]: https://github.com/kubernetes/test-infra/tree/master/prow On Fri, 8 Jan 2021 at 16:15, Matt Sicker wrote: > > On Fri, 8 Jan 2021 at 16:04, Jarek Potiuk wrote: > > > > Github Actions, GitLab, TravisCI. even Cloud Build are s much easier > > to work with and g

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-08 Thread Matt Sicker
On Fri, 8 Jan 2021 at 16:04, Jarek Potiuk wrote: > > Github Actions, GitLab, TravisCI. even Cloud Build are s much easier > to work with and get your stuff done. The multiple threads about how shitty those are in practice for your needs seem to indicate otherwise. Security and easy learning

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-08 Thread Jarek Potiuk
On Fri, Jan 8, 2021 at 10:45 PM Zach Hoffman wrote: > One of the biggest ways that Traffic Control benefits from GH Actions is > that workflows run on forks, which only counts towards that user's 20 > runner limit. As a result, our contributors have less cleanup work to do by > the time they open

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-08 Thread Jarek Potiuk
I used to love gradle and Jenkins and started to hate it. Once you move to python/javascript world, suddenly stuff that take you days in Java/Gradle take hours in Python/Javascript. And it is amplified in case of CI where you do not need to have an enterprise-grade system but a bunch of scripts to

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-08 Thread Zach Hoffman
One of the biggest ways that Traffic Control benefits from GH Actions is that workflows run on forks, which only counts towards that user's 20 runner limit. As a result, our contributors have less cleanup work to do by the time they open a PR. Since running on all forks is not an option with Jenki

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-08 Thread Jarek Potiuk
Let me just answer from my side. Personally - I hate and love Jenkins at the same time. I am a Jenkins user and developer for 15 years or so. I even developed mobile app plugins for jenkins and we run our own jenkins farm for mobile app CI. but some 5 years ago we switched to GitLab which was so

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-08 Thread Antoine Pitrou
Le 08/01/2021 à 22:29, P. Ottlinger a écrit : > Hi Antoine, > > Am 08.01.21 um 22:17 schrieb Antoine Pitrou: >>> What are the gaps in the ASF CI systems that are pushing people onto >> less viable platforms such as GA? >> >> While being a PMC and core developer for Apache Arrow, I'm going to >>

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-08 Thread P. Ottlinger
Hi Antoine, Am 08.01.21 um 22:17 schrieb Antoine Pitrou: >> What are the gaps in the ASF CI systems that are pushing people onto > less viable platforms such as GA? > > While being a PMC and core developer for Apache Arrow, I'm going to > give a personal opinion here: > > - Jenkins I think many p

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-08 Thread Matt Sicker
It must have been many, many years since you last looked at Jenkins. They've supported pipelines (stored in your code repository) since at least 2015 or so. https://www.jenkins.io/doc/book/pipeline/ On Fri, 8 Jan 2021 at 15:17, Antoine Pitrou wrote: > > > Hi, > > On Fri, 8 Jan 2021 12:49:03 -080

Re: ASF Jenkins usability [Was: Re: GA again unreasonably slow (again)]

2021-01-08 Thread Antoine Pitrou
Hi, On Fri, 8 Jan 2021 12:49:03 -0800 Chris Lambertus wrote: > > Have you considered the internal and fully supported ASF Jenkins and/or Buildbot infrastructure? Infra has little control over the free open source offerings, but we have significantly more resources we can bring to bear on own on