[
https://issues.apache.org/jira/browse/CASSANDRA-15953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161611#comment-17161611
]
Yifan Cai commented on CASSANDRA-15953:
---------------------------------------
Log from the manual spark job testing.
{code:java}
➜ docker run --name cas-src -d -p 9042:9042 cassandra:3.0.18
➜ docker run --name cas-tgt -d -p 9043:9042 cassandra:latest
➜ docker exec cas-src cassandra-stress write n=1k -schema keyspace="keyspace1"
➜ docker exec cas-tgt cassandra-stress write n=1k -schema keyspace="keyspace1"
➜ docker exec cas-src cassandra-stress write n=1k -schema keyspace="keyspace2"
➜ docker exec cas-tgt cassandra-stress write n=1k -schema keyspace="keyspace2"
➜ spark-submit --files ./spark-job/localconfig-auto-discover.yaml --class
org.apache.cassandra.diff.DiffJob
spark-uberjar/target/spark-uberjar-0.2-SNAPSHOT.jar
localconfig-auto-discover.yaml
// The diff job yields the following result.
INFO DiffJob: FINISHED: {KeyspaceTablePair{keyspace=keyspace1,
table=counter1}=Matched Partitions - 0, Mismatched Partitions - 0, Partition
Errors - 0, Partitions Only In Source - 0, Partitions Only In Target - 0,
Skipped Partitions - 0, Matched Rows - 0, Matched Values - 0, Mismatched Values
- 0 , KeyspaceTablePair{keyspace=keyspace1, table=standard1}=Matched Partitions
- 1000, Mismatched Partitions - 0, Partition Errors - 0, Partitions Only In
Source - 0, Partitions Only In Target - 0, Skipped Partitions - 0, Matched Rows
- 1000, Matched Values - 6000, Mismatched Values - 0 ,
KeyspaceTablePair{keyspace=keyspace2, table=standard1}=Matched Partitions -
1000, Mismatched Partitions - 0, Partition Errors - 0, Partitions Only In
Source - 0, Partitions Only In Target - 0, Skipped Partitions - 0, Matched Rows
- 1000, Matched Values - 6000, Mismatched Values - 0 ,
KeyspaceTablePair{keyspace=keyspace2, table=counter1}=Matched Partitions - 0,
Mismatched Partitions - 0, Partition Errors - 0, Partitions Only In Source - 0,
Partitions Only In Target - 0, Skipped Partitions - 0, Matched Rows - 0,
Matched Values - 0, Mismatched Values - 0 }
{code}
> Support fetching all user tables to compare in Cassandra-diff
> -------------------------------------------------------------
>
> Key: CASSANDRA-15953
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15953
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/diff
> Reporter: Yifan Cai
> Assignee: Yifan Cai
> Priority: Normal
>
> The spark diff job may fail to launch with kernel error "E2BIG: Argument list
> too long", when passing a large list of keyspace table list to compare.
> Proposing a mode to fetch all user tables from the clusters to be compared.
> When the mode is on, the spark job ignores the parameter "keyspace_tables".
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
