> You don’t have to run every suite on every commit since as folks have pointed > out for the most part the JVM isn’t culprit. Need to run it enough times to > catch when it is for some assumption of “enough”. So riffing on this. We could move to something like: • For each given supported C* branch, confirm it **builds **on all supported JDKs (pre-commit verification, post-commit reactive runs) • Constrain language level on any given C* branch to **lowest supported JDK** • Run all reactive post-commit CI pipelines against *the *highest supported JDK only** • Once a N (day, week, month?), run all pipelines against all supported JDKs on all branches • Augment notification mechanisms so it squawks to dev list and slack on failure of non-highest JDK pipelines That approach would tweak our balance towards our perception of the infrequency of per-JDK failures while allowing us to "scale up" the matrix of tests that we perform.
i.e. once a week we could have a heavy 9x run (3 branches, 3 JDK's) which we could then plan around and space out in terms of resource allocation, but otherwise we run a single set of pipelines per branch post-commit. That'd give us the confidence to say "we tested the upgrade path we're recommending for you" without having to pay the tax of doing it on every commit or allowing potential defects to pile up to a once-a-year JDK-specific bug-bash. In terms of JDK support when bumping (mapping of relative C* version and relative JDK version): • N-2: JDK-2, JDK-3, JDK-4 • N-1: JDK-1, JDK-2, JDK-3 • N: JDK, JDK-1, JDK-2 So we'd have 3 supported LTS per branch, be able to adhere to "you can upgrade from N-2 to N using the same JDK", and allow us to balance our CI coverage to our expected surfacing of defects. Then if we rev JDK we support on any given N+1, we end up with (keeping with N above as reference): • N-1: JDK-1, JDK-2, JDK-3 • N: JDK, JDK-1, JDK-2 • N+1: JDK+1, JDK, JDK-1 So shared JDK across all 3 on that rev is JDK-1. I think 3 LTS per branch gives us the ability to both add / drop a JDK per major and test / provide for upgrades from N-2 to N w/out requiring a new JDK cert too. On Wed, May 21, 2025, at 3:27 AM, Mick Semb Wever wrote: > . > >>> So yeah. I think we'll need to figure out how much coverage is reasonable >>> to call something "tested". I don't think it's sustainable for us to have, >>> at any given time, 3 branches we test across 3 JDK's each with all our >>> in-jvm test suites is it? >>> > > > Correct. > For non-upgrade tests, where testing against more than one jdk exists, we > should start the conversation of the value of running more than one JDK for > all tests per-commit CI, before we go adding a third. > > I'm not against weekly/fortnightly CI runs, just that it deserves the > discussion of cost (it's not necessarily cheaper due to saturation, nor are > we a team that has assigned build barons). The actual change is relatively > easy, just adding a profile and a jdk element here: > https://github.com/apache/cassandra/blob/trunk/.jenkins/Jenkinsfile#L126-L135
