It seemed like a lot of other changes were happening around the CircleCI config, so I was holding off on the parameterization. I would be happy to work with Claude for the changes if that’s already in progress, though.
Cheers, Derek From: "Claude Warren, Jr via dev" <dev@cassandra.apache.org> Reply-To: "dev@cassandra.apache.org" <dev@cassandra.apache.org>, "Claude Warren, Jr" <claude.war...@aiven.io> Date: Friday, November 11, 2022 at 1:06 AM To: "dev@cassandra.apache.org" <dev@cassandra.apache.org> Subject: RE: [EXTERNAL]A proposal for refactoring the CircleCI config CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. I have been working on https://issues.apache.org/jira/projects/CASSANDRA/issues/CASSANDRA-18012 which modifies the generate.sh script for the circleci configurations. Perhaps all of this should be rolled into one change? On Fri, Nov 11, 2022 at 3:47 AM Ekaterina Dimitrova <e.dimitr...@gmail.com<mailto:e.dimitr...@gmail.com>> wrote: Hey Derek, Thanks for looking into this. As we spoke in Slack, probably an easy way to show people how things will look like is to have a prototype with some minimal config. Could be even not Cassandra one but something that will show how things will look like and improve the current model. Thanks, Ekaterina On Wed, 2 Nov 2022 at 17:08, David Capwell <dcapw...@apple.com<mailto:dcapw...@apple.com>> wrote: Here is the ticket I was talking about https://issues.apache.org/jira/browse/CASSANDRA-17600 On Nov 2, 2022, at 1:29 PM, Derek Chen-Becker <de...@chen-becker.org<mailto:de...@chen-becker.org>> wrote: For the parallel param logic, sounds fine to me. Not sure if this would also work for resource_type, but I still argue that xlarge isn’t needed in 90% of the cases its used… so fixing this may be better than param there…. So yes, I would be cool with this change if it basically removes the patching logic… I had another JIRA to have a python script rewrite the YAML, but this method may solve in a cleaner way. Almost any part of a CircleCI definition can be replaced with a parameter, so basically we want config-2_1.yml to be a template, and we plug different values in as desired. Would you mind sending a link to that JIRA so I can understand that use case? About matrix jobs; I don’t know them in circle but have used in other places, this sounds good to me. I would also enhance to argue that JVM is just 1 config and we sadly have many more: JVM: [8, 11, 17] VNODE: [true, false] CDC: [true, false] COMPRESSION: [true, false] MEMTABLE: [skiplist, shardedskiplist, trie] My understanding is that we could parameterize all of these such that we could use a matrix as long as all combinations are valid. Let me get parameterization of basic configuration reviewed first, and then we can take a look at how to matricize things. Cheers, Derek -- +---------------------------------------------------------------+ | Derek Chen-Becker | | GPG Key available at https://keybase.io/dchenbecker and | | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org | | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7 7F42 AFC5 AFEE 96E4 6ACC | +---------------------------------------------------------------+
