[
https://issues.apache.org/jira/browse/CASSJAVA-138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bret McGuire updated CASSJAVA-138:
----------------------------------
Description:
While working on CASSJAVA-136 I started seeing random test failures on Jenkins
builds of my pull request. The change implemented on the pull request (a
modification of equality for DSE geometric types) has absolutely no relation to
continuous paging or anything in the test. Yet after my modification I saw the
following:
First run:
{noformat}
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=1,
sizeInBytes=false, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=100]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=10, maxPagesPerSecond=0, expectedRows=100,
expectedPages=10]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=9, maxPagesPerSecond=0, expectedRows=90,
expectedPages=9]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=0, maxPagesPerSecond=2, expectedRows=100,
expectedPages=10]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=8,
sizeInBytes=true, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=100]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=16,
sizeInBytes=true, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=50]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=32,
sizeInBytes=true, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=25]
{noformat}
Second run (with identical code):
{noformat}
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=99,
sizeInBytes=false, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=2]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=50,
sizeInBytes=false, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=2]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=1,
sizeInBytes=false, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=100]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=10, maxPagesPerSecond=0, expectedRows=100,
expectedPages=10]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=9, maxPagesPerSecond=0, expectedRows=90,
expectedPages=9]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=0, maxPagesPerSecond=2, expectedRows=100,
expectedPages=10]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=9, maxPagesPerSecond=0, expectedRows=90,
expectedPages=9]
{noformat}
A couple things are of interest here:
* -The tests only fail on Java17. The Jenkins build runs tests for Java8,
Java11, Java17 and Java21 but only Java17 seems to fail-
* I do not see any equivalent failure on test runs of the 4.x branch
** In fairness the 4.x branch would still have the geometry issue referenced in
CASSJAVA-136 but as mentioned above there's no correlation between that issue
and this test
* The [test provider in
question|https://github.com/apache/cassandra-java-driver/blob/e15da70e9fe2856bbcb27ab141b755440ca39ca9/integration-tests/src/test/java/com/datastax/dse/driver/api/core/cql/continuous/ContinuousPagingITBase.java#L84-L108]
has 10 distinct options and no test is failing for all 10 options
* Tests for a given version of DSE are not consistent between runs
All this evidence together seems to argue for a flaky integration test... but I
don't see anything obviously wrong with it
was:
While working on CASSJAVA-136 I started seeing random test failures on Jenkins
builds of my pull request. The change implemented on the pull request (a
modification of equality for DSE geometric types) has absolutely no relation to
continuous paging or anything in the test. Yet after my modification I saw the
following:
First run:
{noformat}
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=1,
sizeInBytes=false, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=100]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=10, maxPagesPerSecond=0, expectedRows=100,
expectedPages=10]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=9, maxPagesPerSecond=0, expectedRows=90,
expectedPages=9]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=0, maxPagesPerSecond=2, expectedRows=100,
expectedPages=10]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=8,
sizeInBytes=true, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=100]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=16,
sizeInBytes=true, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=50]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=32,
sizeInBytes=true, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=25]
{noformat}
Second run (with identical code):
{noformat}
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=99,
sizeInBytes=false, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=2]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=50,
sizeInBytes=false, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=2]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=1,
sizeInBytes=false, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
expectedPages=100]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=10, maxPagesPerSecond=0, expectedRows=100,
expectedPages=10]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=9, maxPagesPerSecond=0, expectedRows=90,
expectedPages=9]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=0, maxPagesPerSecond=2, expectedRows=100,
expectedPages=10]
Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
'[email protected]' / Execute-Tests /
com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
sizeInBytes=false, maxPages=9, maxPagesPerSecond=0, expectedRows=90,
expectedPages=9]
{noformat}
A couple things are of interest here:
* The tests only fail on Java17. The Jenkins build runs tests for Java8,
Java11, Java17 and Java21 but only Java17 seems to fail
* I do not see any equivalent failure on test runs of the 4.x branch
** In fairness the 4.x branch would still have the geometry issue referenced in
CASSJAVA-136 but as mentioned above there's no correlation between that issue
and this test
* The [test provider in
question|https://github.com/apache/cassandra-java-driver/blob/e15da70e9fe2856bbcb27ab141b755440ca39ca9/integration-tests/src/test/java/com/datastax/dse/driver/api/core/cql/continuous/ContinuousPagingITBase.java#L84-L108]
has 10 distinct options and no test is failing for all 10 options
* Tests for a given version of DSE are not consistent between runs
All this evidence together seems to argue for a flaky integration test... but I
don't see anything obviously wrong with it
> Intermittent test failures with ContinuousPagingIT and Java17 (likely flaky
> test)
> ---------------------------------------------------------------------------------
>
> Key: CASSJAVA-138
> URL: https://issues.apache.org/jira/browse/CASSJAVA-138
> Project: Apache Cassandra Java driver
> Issue Type: Improvement
> Reporter: Bret McGuire
> Priority: Normal
>
> While working on CASSJAVA-136 I started seeing random test failures on
> Jenkins builds of my pull request. The change implemented on the pull
> request (a modification of equality for DSE geometric types) has absolutely
> no relation to continuous paging or anything in the test. Yet after my
> modification I saw the following:
> First run:
> {noformat}
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=1,
> sizeInBytes=false, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
> expectedPages=100]
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
> sizeInBytes=false, maxPages=10, maxPagesPerSecond=0, expectedRows=100,
> expectedPages=10]
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
> sizeInBytes=false, maxPages=9, maxPagesPerSecond=0, expectedRows=90,
> expectedPages=9]
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
> sizeInBytes=false, maxPages=0, maxPagesPerSecond=2, expectedRows=100,
> expectedPages=10]
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=8,
> sizeInBytes=true, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
> expectedPages=100]
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=16,
> sizeInBytes=true, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
> expectedPages=50]
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=32,
> sizeInBytes=true, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
> expectedPages=25]
> {noformat}
> Second run (with identical code):
> {noformat}
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=99,
> sizeInBytes=false, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
> expectedPages=2]
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=50,
> sizeInBytes=false, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
> expectedPages=2]
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=1,
> sizeInBytes=false, maxPages=0, maxPagesPerSecond=0, expectedRows=100,
> expectedPages=100]
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
> sizeInBytes=false, maxPages=10, maxPagesPerSecond=0, expectedRows=100,
> expectedPages=10]
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
> sizeInBytes=false, maxPages=9, maxPagesPerSecond=0, expectedRows=90,
> expectedPages=9]
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.9.0', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
> sizeInBytes=false, maxPages=0, maxPagesPerSecond=2, expectedRows=100,
> expectedPages=10]
> Per-Commit / Matrix - SERVER_VERSION = 'dse-6.8.30', JABBA_VERSION =
> '[email protected]' / Execute-Tests /
> com.datastax.dse.driver.api.core.cql.continuous.ContinuousPagingIT.should_execute_prepared_statement_synchronously[pageSize=10,
> sizeInBytes=false, maxPages=9, maxPagesPerSecond=0, expectedRows=90,
> expectedPages=9]
> {noformat}
> A couple things are of interest here:
> * -The tests only fail on Java17. The Jenkins build runs tests for Java8,
> Java11, Java17 and Java21 but only Java17 seems to fail-
> * I do not see any equivalent failure on test runs of the 4.x branch
> ** In fairness the 4.x branch would still have the geometry issue referenced
> in CASSJAVA-136 but as mentioned above there's no correlation between that
> issue and this test
> * The [test provider in
> question|https://github.com/apache/cassandra-java-driver/blob/e15da70e9fe2856bbcb27ab141b755440ca39ca9/integration-tests/src/test/java/com/datastax/dse/driver/api/core/cql/continuous/ContinuousPagingITBase.java#L84-L108]
> has 10 distinct options and no test is failing for all 10 options
> * Tests for a given version of DSE are not consistent between runs
> All this evidence together seems to argue for a flaky integration test... but
> I don't see anything obviously wrong with it
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
