I personally rarely push tickets/post a patch in an raw state, but since I almost always have to approve jobs when it gets close to commit, I don't mind also confirming utest runs. I'd say it'd be good to run at very least a compilation step on every commit. I think it's fine if dtests/utests/jvm tests require approval.
On Wed, Oct 6, 2021 at 4:22 PM Andrés de la Peña <adelap...@apache.org> wrote: > Hello all, > > The changes in CircleCI proposed in the previous messages are implemented > in CASSANDRA-16882. They try to include the feedback from both the > reviewers and the Slack discussion at > https://the-asf.slack.com/archives/CK23JSY2K/p1631627458109000. > > The patch modifies the CircleCI config to have two pairs of j8/j11 > workflows: > > * The javaX_pre-commit_tests workflows are meant to be used when a patch is > definitively ready for final review and/or commit. They have a single > approval step that starts all the basic tests, including unit tests, > dtests, etc. Additionally, it has approval steps for those tests that are > not generally required in every ticket, such as upgrade tests and the > multiplexer. This pair of workflows is quite similar to what we currently > have, and the main difference is that there is an approval step to start > the build. > > * The javaX_separate_tests workflows are meant to be used in intermediate > steps and special cases such as fixing flaky tests. All the jobs in these > workflows have an individual approval step, so one can run any test job > without wasting resources in the others. For example, it makes possible to > repeatedly run a unit test without firing the entire JVM dtests. > > Here is an example of how the workflows would look like: > > https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-trunk-v04 > > Hopefully this approach would help us to save resources in intermediate > development steps and special cases, while keeping the current concept of > mandatory tests. > > If no one disagrees with this approach I'll commit the changes in a few > days. > > On Fri, 27 Aug 2021 at 11:54, Andrés de la Peña <adelap...@apache.org> > wrote: > > > Hi, > > > > CASSANDRA-16882 has patches for any of the mentioned configurations aimed > > to save CircleCI resources. > > > > There are CircleCI runs showing how each approach would look like, plus > an > > additional fourth option: > > > > 1. Make the entire workflow optional: > > > https://app.circleci.com/pipelines/github/adelapena/cassandra/800/workflows/9cb8ca7b-ab57-431e-a22b-643d61c92c29 > > 2. Make all the test jobs optional: > > > https://app.circleci.com/pipelines/github/adelapena/cassandra/798/workflows/a859cfbc-fdf8-4468-beb9-b2ee17dc1ae3 > > 3. Make all the mandatory test jobs depend on a single optional job: > > > https://app.circleci.com/pipelines/github/adelapena/cassandra/802/workflows/0372f5d6-d1f0-4f0e-91a3-aa75a2712bae > > 4. Combine 2 and 3 to have approval jobs for both individual tests and > the > > grouped mandatory tests: > > > https://app.circleci.com/pipelines/github/adelapena/cassandra/803/workflows/08ae07d5-6a1e-4e5b-bc0c-32bdc9b9f190 > > > > I think that the fourth option gives us more flexibility, because it > > allows us to start any combination of tests we want while it keeps the > > concept of required tests. > > > > On Thu, 12 Aug 2021 at 17:47, Andrés de la Peña < > a.penya.gar...@gmail.com> > > wrote: > > > >> Hello all, > >> > >> The current CircleCI configuration automatically runs the unit tests, > JVM > >> dtests and cqhshlib tests. This is done by default for every commit or, > >> with some configuration, for every push. > >> > >> Along the lifecycle of a ticket it is quite frequent to have multiple > >> commits and pushes, all running these test jobs. I'd say that > frequently it > >> is not necessary to run the tests for some of those intermediate commits > >> and pushes. For example, one can show proofs of concept, or have > multiple > >> rounds of review before actually running the tests. Running the tests > for > >> every change can produce an unnecessary expense of CircleCI resources. > >> > >> I think we could make running those tests optional, as well as clearly > >> specifying in the documentation what are the tests runs that are > mandatory > >> before actually committing. We could do this in different ways: > >> > >> 1. Make the entire CircleCI workflow optional, so the build job requires > >> manual approval. Once the build is approved the mandatory test jobs > would > >> be run without any further approval, exactly as it's currently done. > >> > >> 2. Make all the test jobs optional, so every test job requires manual > >> approval, and the documentation specifies which tests are mandatory in > the > >> final steps of a ticket. > >> > >> 3. Make all the mandatory test jobs depend on a single optional job, so > >> we have a single button to optionally run all the mandatory tests. > >> > >> I think any of these changes, or a combination of them, would > >> significantly reduce the usage of resources without making things less > >> tested. The only downside I can think of is that we would need some > >> additional clicks on the CircleCI GUI. What do you think? > >> > > > -- alex p
