Hello Cassandra development community! I've been putting together these nightly emails tracking test failures on the upcoming release branch and kicking them out to a small list of committers, but I was thinking there may be value in opening up to the dev community at large now that I've gotten a somewhat established format for it.
Assuming no major protest, I plan on sending these out once a day at EoD EST, listing test failures on the upcoming release branch that would block release, a Jira associated with them, and their current status. It's a great place for us to have some discussion about the code-base and the state of the current release branch as well as introduce ideas to improve our testing infrastructure. As we didn't discuss blocking a release for flaky tests until 3.9, 3.8 is already delayed, and 3.8 is a feature release, I think it's reasonable that we continue tracking 3.9 in an effort to get 3.8 out the door and give us some runway to catch up with some flaky debt. So - on to the report: 7/25/16 Jira Status cdc_test.TestCDC.test_cdc_data_available_in_cdc_raw <http://cassci.datastax.com/job/cassandra-3.9_dtest/lastCompletedBuild/testReport/cdc_test/TestCDC/test_cdc_data_available_in_cdc_raw> 12286 <https://issues.apache.org/jira/browse/CASSANDRA-12286> jwitschey offline_tools_test.TestOfflineTools.sstablelevelreset_test <http://cassci.datastax.com/job/cassandra-3.9_dtest/lastCompletedBuild/testReport/offline_tools_test/TestOfflineTools/sstablelevelreset_test> None Invalid Jira link. Needs Triage org.apache.cassandra.cql3.validation.miscellaneous.SSTablesIteratedTest.testDeletionOnOverlappingIndexedSSTable-compression <http://cassci.datastax.com/job/cassandra-3.9_testall/lastCompletedBuild/testReport/org.apache.cassandra.cql3.validation.miscellaneous/SSTablesIteratedTest/testDeletionOnOverlappingIndexedSSTable-compression> 12293 stefania, flaky test rebuild_test.TestRebuild.simple_rebuild_test <http://cassci.datastax.com/job/cassandra-3.9_dtest/lastCompletedBuild/testReport/rebuild_test/TestRebuild/simple_rebuild_test> 11687 jwitschey cassandra-3.9_novnode_dtest -> last run failed <https://cassci.datastax.com/view/cassandra-3.9/job/cassandra-3.9_novnode_dtest/18/> None Needs Triage cassandra-3.9_dtest_upgrade -> 74 failures <https://cassci.datastax.com/view/cassandra-3.9/job/cassandra-3.9_dtest_upgrade/lastCompletedBuild/testReport/> 12236 Won't fix all, probably most. Need to run against branch Some observations: - 12293 is a new ticket for a pretty consistently flaky test. - sstablelevelreset_test appears to have had a regression around build #15 from some code last Wednesday perhaps (having not looked into it beyond that). Consistently fails, not flaky. - The upgrade tests are incredibly noisy due to post 3.0 mixed-version schema issues uncovered by CASSANDRA-8844 and being addressed in CASSANDRA-12236. As I mentioned before in Sankalp's thread, please let me know if you're interested in being included in a list of contributors to help out with test failures and whether you'd prefer unit test only or are ok with dtests and python work. The more contributors we have engaged in this process the more stable our code-base can become. Thanks. ~Josh McKenzie
