Hello all, The changes in CircleCI proposed in the previous messages are implemented in CASSANDRA-16882. They try to include the feedback from both the reviewers and the Slack discussion at https://the-asf.slack.com/archives/CK23JSY2K/p1631627458109000.
The patch modifies the CircleCI config to have two pairs of j8/j11 workflows: * The javaX_pre-commit_tests workflows are meant to be used when a patch is definitively ready for final review and/or commit. They have a single approval step that starts all the basic tests, including unit tests, dtests, etc. Additionally, it has approval steps for those tests that are not generally required in every ticket, such as upgrade tests and the multiplexer. This pair of workflows is quite similar to what we currently have, and the main difference is that there is an approval step to start the build. * The javaX_separate_tests workflows are meant to be used in intermediate steps and special cases such as fixing flaky tests. All the jobs in these workflows have an individual approval step, so one can run any test job without wasting resources in the others. For example, it makes possible to repeatedly run a unit test without firing the entire JVM dtests. Here is an example of how the workflows would look like: https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-trunk-v04 Hopefully this approach would help us to save resources in intermediate development steps and special cases, while keeping the current concept of mandatory tests. If no one disagrees with this approach I'll commit the changes in a few days. On Fri, 27 Aug 2021 at 11:54, Andrés de la Peña <adelap...@apache.org> wrote: > Hi, > > CASSANDRA-16882 has patches for any of the mentioned configurations aimed > to save CircleCI resources. > > There are CircleCI runs showing how each approach would look like, plus an > additional fourth option: > > 1. Make the entire workflow optional: > https://app.circleci.com/pipelines/github/adelapena/cassandra/800/workflows/9cb8ca7b-ab57-431e-a22b-643d61c92c29 > 2. Make all the test jobs optional: > https://app.circleci.com/pipelines/github/adelapena/cassandra/798/workflows/a859cfbc-fdf8-4468-beb9-b2ee17dc1ae3 > 3. Make all the mandatory test jobs depend on a single optional job: > https://app.circleci.com/pipelines/github/adelapena/cassandra/802/workflows/0372f5d6-d1f0-4f0e-91a3-aa75a2712bae > 4. Combine 2 and 3 to have approval jobs for both individual tests and the > grouped mandatory tests: > https://app.circleci.com/pipelines/github/adelapena/cassandra/803/workflows/08ae07d5-6a1e-4e5b-bc0c-32bdc9b9f190 > > I think that the fourth option gives us more flexibility, because it > allows us to start any combination of tests we want while it keeps the > concept of required tests. > > On Thu, 12 Aug 2021 at 17:47, Andrés de la Peña <a.penya.gar...@gmail.com> > wrote: > >> Hello all, >> >> The current CircleCI configuration automatically runs the unit tests, JVM >> dtests and cqhshlib tests. This is done by default for every commit or, >> with some configuration, for every push. >> >> Along the lifecycle of a ticket it is quite frequent to have multiple >> commits and pushes, all running these test jobs. I'd say that frequently it >> is not necessary to run the tests for some of those intermediate commits >> and pushes. For example, one can show proofs of concept, or have multiple >> rounds of review before actually running the tests. Running the tests for >> every change can produce an unnecessary expense of CircleCI resources. >> >> I think we could make running those tests optional, as well as clearly >> specifying in the documentation what are the tests runs that are mandatory >> before actually committing. We could do this in different ways: >> >> 1. Make the entire CircleCI workflow optional, so the build job requires >> manual approval. Once the build is approved the mandatory test jobs would >> be run without any further approval, exactly as it's currently done. >> >> 2. Make all the test jobs optional, so every test job requires manual >> approval, and the documentation specifies which tests are mandatory in the >> final steps of a ticket. >> >> 3. Make all the mandatory test jobs depend on a single optional job, so >> we have a single button to optionally run all the mandatory tests. >> >> I think any of these changes, or a combination of them, would >> significantly reduce the usage of resources without making things less >> tested. The only downside I can think of is that we would need some >> additional clicks on the CircleCI GUI. What do you think? >> >
