Hi all,
I am working on a Hadoop InputFormat implementation that uses only the native protocol Java driver and not the Thrift API. I am currently trying to replicate some of the behavior of *Cassandra.client.describe_ring(myKeyspace)* from the Thrift API. I would like to do the following: - Get a list of all of the token ranges for a cluster - For every token range, determine the replica nodes on which the data in the token range resides - Estimate the number of rows for every range of tokens - Groups ranges of tokens on common replica nodes such that we can create a set of input splits for Hadoop with total estimated line counts that are reasonably close to the requested split size Last week I received some much-appreciated help on this list that pointed me to using the system.peers table to get the list of token ranges for the cluster and the corresponding hosts. Today I created a three-node C* cluster in Vagrant (https://github.com/dholbrook/vagrant-cassandra) and tried inspecting some of the system tables. I have a couple of questions now: 1. *How many total unique tokens should I expect to see in my cluster?* If I have three nodes, and each node has a cassandra.yaml with num_tokens = 256, then should I expect a total of 256*3 = 768 distinct vnodes? 2. *How does the creation of vnodes and their assignment to nodes relate to the replication factor for a given keyspace?* I never thought about this until today, and I tried to reread the documentation on virtual nodes, replication in Cassandra, etc., and now I am sadly still confused. Here is what I think I understand. :) - Given a row with a partition key, any client request for an operation on that row will go to a coordinator node in the cluster. - The coordinator node will compute the token value for the row and from that determine a set of replica nodes for that token. - One of the replica nodes I assume is the node that "owns" the vnode with the token range that encompasses the token - The identity of the "owner" of this virtual node is a cross-keyspace property - And the other replicas were originally chosen based on the replica-placement strategy - And therefore the other replicas will be different for each keyspace (because replication factors and replica-placement strategy are properties of a keyspace) 3. What do the values in the "token" column in system.peers and system.local refer to then? - Since these tables appear to be global, and not per-keyspace properties, I assume that they don't have any information about replication in them, is that correct? - If I have three nodes in my cluster, 256 vnodes per node, and I'm using the Murmur3 partitioner, should I then expect to see the values of "tokens" in system.peers and system.local be 768 evenly-distributed values between -2^63 and 2^63? 4. Is there any other way, without using Thift, to get as much information as possible about what nodes contain replicas of data for all of the token ranges in a given cluster? I really appreciate any help, thanks! Best regards, Clint
