[
https://issues.apache.org/jira/browse/CASSANDRA-15520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023170#comment-17023170
]
David Capwell commented on CASSANDRA-15520:
-------------------------------------------
bq. I suspect doing this will also give us a chunk of the work required to
parallelise the dtests in Jenkins
That is my hope as well. My setup is modeled after Circle and I see that there
is very very little in the scripts which actually are CircleCI specific
(https://github.com/apache/cassandra/blob/trunk/.circleci/config-2_1.yml#L512
calls circleci tests split --split-by=timings --timings-type=filename
--index=${CIRCLE_NODE_INDEX} --total=${CIRCLE_NODE_TOTAL}
/tmp/all_java_unit_tests.txt which can stay in CircleCI, everything else is
generic). There is also test behavior that assumes CircleCI (see
https://github.com/apache/cassandra/search?l=Java&q=CIRCLECI).
As long as the "split" file being text works for Jenkins, that seems like a
simple way to work cross many platforms. For those which can't run the tests
in parallel, we could offer a run.sh which just calls split.sh and
run_partition.sh
> split circle ci commands into reusable scripts which can be used outside of
> circle ci
> -------------------------------------------------------------------------------------
>
> Key: CASSANDRA-15520
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15520
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
> Reporter: David Capwell
> Assignee: David Capwell
> Priority: Normal
>
> CircleCI is one of the main tools we use for build and test, but there is
> also ASF Jenkins and many people run builds in their own companies as well.
> It would be nice to refactor the existing CircleCI yaml to delegate to a set
> of scripts which could be reused by other build systems.
> I feel that we could do the following directory layout
> {code}
> ci - top level directory containing all scripts
> - <test type : unit, distributed> - directory containing the different build
> steps
> - <test name : eg. unit, jvm-dtest, long, etc.> - a single build with
> the required steps to run it
> - split.sh - script which takes in a output file to write to and
> dumps out all test cases (not partitioned)
> - run_partition.sh - script which takes a partitioned list of tests
> and executes the build (does not move around artifacts)
> {code}
> This would allow CircleCI and Jenkins to run the same way, but also acts as
> documentation for how to run some of the tests (jvm-dtest and python dtest
> upgrade tests take more time to figure out how to run).
> CircleCI would also be simpler as it would mostly be the circle ci specific
> logic (partition tests, move code/test results around, etc.) calling these
> scripts.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
