On Mon, Sep 26, 2022 at 12:20 PM Josh McKenzie <jmcken...@apache.org> wrote: > For CI on a patch, run the pre-commit suite and also run multiplexer with 250 > runs on new, changed, or related tests to ensure not flaky
250 iterations isn't enough; I use 500 as a low water mark. > I think it's worth calling out: relying on circleci test runs along with ASF > CI as an option implies we have the same confidence in a green circleci run > as we have in a green ASF CI run. This is also assuming that circle and ASF CI run the same tests, which is not entirely true. We know that circle isn't running some upgrade tests from https://issues.apache.org/jira/browse/CASSANDRA-17912 but I think we need to specifically investigate parity between the two systems for further discrepancies before we can begin to compare confidence in runs between them.
