I thought this discussion was still ongoing, but it looks like CASSANDRA-16882 is now committed.
Could you give some context on why at least compilation is not done on every commit now? On Fri, Oct 8, 2021 at 4:12 PM Oleksandr Petrov <oleksandr.pet...@gmail.com> wrote: > I personally rarely push tickets/post a patch in an raw state, but since I > almost always have to approve jobs when it gets close to commit, I don't > mind also confirming utest runs. I'd say it'd be good to run at very least > a compilation step on every commit. I think it's fine if dtests/utests/jvm > tests require approval. > > On Wed, Oct 6, 2021 at 4:22 PM Andrés de la Peña <adelap...@apache.org> > wrote: > >> Hello all, >> >> The changes in CircleCI proposed in the previous messages are implemented >> in CASSANDRA-16882. They try to include the feedback from both the >> reviewers and the Slack discussion at >> https://the-asf.slack.com/archives/CK23JSY2K/p1631627458109000. >> >> The patch modifies the CircleCI config to have two pairs of j8/j11 >> workflows: >> >> * The javaX_pre-commit_tests workflows are meant to be used when a patch >> is >> definitively ready for final review and/or commit. They have a single >> approval step that starts all the basic tests, including unit tests, >> dtests, etc. Additionally, it has approval steps for those tests that are >> not generally required in every ticket, such as upgrade tests and the >> multiplexer. This pair of workflows is quite similar to what we currently >> have, and the main difference is that there is an approval step to start >> the build. >> >> * The javaX_separate_tests workflows are meant to be used in intermediate >> steps and special cases such as fixing flaky tests. All the jobs in these >> workflows have an individual approval step, so one can run any test job >> without wasting resources in the others. For example, it makes possible to >> repeatedly run a unit test without firing the entire JVM dtests. >> >> Here is an example of how the workflows would look like: >> >> https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-trunk-v04 >> >> Hopefully this approach would help us to save resources in intermediate >> development steps and special cases, while keeping the current concept of >> mandatory tests. >> >> If no one disagrees with this approach I'll commit the changes in a few >> days. >> >> On Fri, 27 Aug 2021 at 11:54, Andrés de la Peña <adelap...@apache.org> >> wrote: >> >> > Hi, >> > >> > CASSANDRA-16882 has patches for any of the mentioned configurations >> aimed >> > to save CircleCI resources. >> > >> > There are CircleCI runs showing how each approach would look like, plus >> an >> > additional fourth option: >> > >> > 1. Make the entire workflow optional: >> > >> https://app.circleci.com/pipelines/github/adelapena/cassandra/800/workflows/9cb8ca7b-ab57-431e-a22b-643d61c92c29 >> > 2. Make all the test jobs optional: >> > >> https://app.circleci.com/pipelines/github/adelapena/cassandra/798/workflows/a859cfbc-fdf8-4468-beb9-b2ee17dc1ae3 >> > 3. Make all the mandatory test jobs depend on a single optional job: >> > >> https://app.circleci.com/pipelines/github/adelapena/cassandra/802/workflows/0372f5d6-d1f0-4f0e-91a3-aa75a2712bae >> > 4. Combine 2 and 3 to have approval jobs for both individual tests and >> the >> > grouped mandatory tests: >> > >> https://app.circleci.com/pipelines/github/adelapena/cassandra/803/workflows/08ae07d5-6a1e-4e5b-bc0c-32bdc9b9f190 >> > >> > I think that the fourth option gives us more flexibility, because it >> > allows us to start any combination of tests we want while it keeps the >> > concept of required tests. >> > >> > On Thu, 12 Aug 2021 at 17:47, Andrés de la Peña < >> a.penya.gar...@gmail.com> >> > wrote: >> > >> >> Hello all, >> >> >> >> The current CircleCI configuration automatically runs the unit tests, >> JVM >> >> dtests and cqhshlib tests. This is done by default for every commit or, >> >> with some configuration, for every push. >> >> >> >> Along the lifecycle of a ticket it is quite frequent to have multiple >> >> commits and pushes, all running these test jobs. I'd say that >> frequently it >> >> is not necessary to run the tests for some of those intermediate >> commits >> >> and pushes. For example, one can show proofs of concept, or have >> multiple >> >> rounds of review before actually running the tests. Running the tests >> for >> >> every change can produce an unnecessary expense of CircleCI resources. >> >> >> >> I think we could make running those tests optional, as well as clearly >> >> specifying in the documentation what are the tests runs that are >> mandatory >> >> before actually committing. We could do this in different ways: >> >> >> >> 1. Make the entire CircleCI workflow optional, so the build job >> requires >> >> manual approval. Once the build is approved the mandatory test jobs >> would >> >> be run without any further approval, exactly as it's currently done. >> >> >> >> 2. Make all the test jobs optional, so every test job requires manual >> >> approval, and the documentation specifies which tests are mandatory in >> the >> >> final steps of a ticket. >> >> >> >> 3. Make all the mandatory test jobs depend on a single optional job, so >> >> we have a single button to optionally run all the mandatory tests. >> >> >> >> I think any of these changes, or a combination of them, would >> >> significantly reduce the usage of resources without making things less >> >> tested. The only downside I can think of is that we would need some >> >> additional clicks on the CircleCI GUI. What do you think? >> >> >> > >> > > > -- > alex p > -- alex p
