Hi, the documentation says that '...The recommended initial value for num_tokens is 256...' and this is what we did with our cluster which is running Cassandra Community 2.0.9, has 3 physical nodes with replication factor 3 for all keyspaces, each with 256 vnodes, each physical node has about 96 GB data. We noticed that doing a repair for some keyspaces can take up to 37 hours. We did some testing and reduced the number of vnodes from 256 to 32 for each physical node, and we noticed that this does reduce the amount of time to do repair quite a lot, as indicated in the following:
nodetool repair command Cassandra version Number of vnodes/physical node Repair time nodetool repair courseassociation associations 2.0.9 256 26 hours 4 minutes 32 21 hours 46 minutes nodetool repair userassociation associations 2.0.9 256 37 hours 2 minutes 32 26 hours 29 minutes nodetool repair orguserassociation associations 2.0.9 256 13 hours 35 minutes 32 6 hrs 27 minutes nodetool repair userorgassociation associations 2.0.9 256 3 hours 26 minutes 32 1 hour 39 minutes So using a smaller number of vnodes does reduce the repair time, but what are other implications by doing so, performance? system resource consumptions? Is there a general guideline on the number of vnodes we should configure to? Thanks. George
