256 was chosen because the original vnode allocation algorithm was random and fewer than 256 could lead to unbalanced nodes
In 3.0 there’s a less naive algorithm to ensure more balanced distribution, and there 16 or 32 is probably preferable -- Jeff Jirsa > On Oct 9, 2017, at 7:38 AM, Li, Guangxing <guangxing...@pearson.com> wrote: > > Hi, > > the documentation says that '...The recommended initial value for > num_tokens is 256...' and this is what we did with our cluster which is > running Cassandra Community 2.0.9, has 3 physical nodes with replication > factor 3 for all keyspaces, each with 256 vnodes, each physical node has > about 96 GB data. We noticed that doing a repair for some keyspaces can > take up to 37 hours. We did some testing and reduced the number of vnodes > from 256 to 32 for each physical node, and we noticed that this does reduce > the amount of time to do repair quite a lot, as indicated in the following: > > nodetool repair command Cassandra version Number of vnodes/physical node > Repair > time > > nodetool repair courseassociation associations > 2.0.9 > 256 26 hours 4 minutes > 32 21 hours 46 minutes > > nodetool repair userassociation associations > 2.0.9 > 256 37 hours 2 minutes > 32 26 hours 29 minutes > > nodetool repair orguserassociation associations > 2.0.9 > 256 13 hours 35 minutes > 32 6 hrs 27 minutes > > nodetool repair userorgassociation associations > 2.0.9 > 256 3 hours 26 minutes > 32 1 hour 39 minutes > > So using a smaller number of vnodes does reduce the repair time, but what > are other implications by doing so, performance? system resource > consumptions? Is there a general guideline on the number of vnodes we > should configure to? > > Thanks. > > George --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
