> 1. Tune parallelism levels per job (David and Ekaterina have insight on this) > Question for David, do you tune only parallelism and use only xlarge? If yes, > we need to talk :D
Yes, and this is 100% because I am lazy. Too high parallel jobs are a problem for circle as 100% of resources need to be free to start a job; so if you ask for 100 resources and 99 are free, everyone is blocked until 1 resource frees up for that job… :sadpanda:. Now, do we need xlarge? Nope, but I don’t change as that doesn’t impact me… I am 100% cool getting rid of LOW/MID/HIGH and tuning our jobs to what actually is needed… I hate that we all do something different (MID, HIGH, and custom HIGH) > On Oct 21, 2022, at 10:39 AM, David Capwell <dcapw...@apple.com> wrote: > > I am cool with removing circle if apache CI is stable and works, we do need > to solve the non-committer issue but would argue that partially exists in > circle today (you can be a non-commuter with a paid account, but you can’t be > a non-committer with a free account) > > > >> On Oct 20, 2022, at 2:20 PM, Josh McKenzie <jmcken...@apache.org >> <mailto:jmcken...@apache.org>> wrote: >> >>> I believe it's original intention to be just about CircleCI. >> It was but fwiw I'm good w/us exploring adjacent things regarding CI here. >> I'm planning on deep diving on the thread tomorrow and distilling a snapshot >> of the work we have a consensus on for circle and summarizing here so we >> don't lose that. Seems like it's fairly non-controversial. >> >> On Thu, Oct 20, 2022, at 5:14 PM, Mick Semb Wever wrote: >>> >>> >>> On Thu, 20 Oct 2022 at 22:07, Derek Chen-Becker <de...@chen-becker.org >>> <mailto:de...@chen-becker.org>> wrote: >>> Would the preclusion of non-committers also prevent us from configuring >>> Jenkins to auto-test on PR independent of who opens it? >>> >>> One of my current concerns is that we're maintaining 2x the CI for 1x the >>> benefit, and I don't currently see an easy way to unify them (perhaps a >>> lack of imagination?). I know there's a long history behind the choice of >>> CircleCI, so I'm not trying to be hand-wavy about all of the thought that >>> went into that decision, but that decision has costs beyond just a paid >>> CircleCI account. My long term, probably naive, goals for CI would be to: >>> >>> 1. Have a CI system that is *fully* available to *any* contributor, modulo >>> safeguards to prevent abuse >>> >>> >>> This thread is going off-topic, as I believe it's original intention to be >>> just about CircleCI. >>> >>> But on your point… our community CI won't be allowed (by ASF), nor have >>> capacity (limited donated resources), to run pre-commit testing by anyone >>> and everyone. >>> >>> Today, trusted contributors can be handed tokens to ci-cassandra.a.o (make >>> sure to label them so they can be revoked easily), but we still face the >>> issue that too many pre-commit runs impacts the throughput and quality of >>> the post-commit runs (though this has improved recently). >>> >>> It's on my wishlist to be able to: with a single command line; spin up the >>> ci-cassandra.a.o stack on any k8s cluster, run any git sha through it and >>> collect results, and tear it down. Variations on this would solve >>> non-committers being able to repeat, use, and work on their own (or a >>> separately donated) CI system, and folk/companies with money to be able to >>> run their own ci-cassandra.a.o stacks for faster pre-commit turnaround >>> time. Having this reproducibility of the CI system would make testing >>> changes to it easier as well, so I'd expect a positive feedback loop here. >>> >>> I have some rough ideas on how to get started on this, if anyone would like >>> to buddy up on it. >
