+1 to reducing the number of tokens as low as possible for availability issues. 4 lgtm
On Wed, Jan 29, 2020 at 1:14 AM Dinesh Joshi <djo...@apache.org> wrote: > Thanks for restarting this discussion Jeremy. I personally think 4 is a > good number as a default. I think whatever we pick, we should have enough > documentation for operators to make sense of the new defaults in 4.0. > > Dinesh > > > On Jan 28, 2020, at 9:25 PM, Jeremy Hanna <jeremy.hanna1...@gmail.com> > wrote: > > > > I wanted to start a discussion about the default for num_tokens that > we'd like for people starting in Cassandra 4.0. This is for ticket > CASSANDRA-13701 <https://issues.apache.org/jira/browse/CASSANDRA-13701> > (which has been duplicated a number of times, most recently by me). > > > > TLDR, based on availability concerns, skew concerns, operational > concerns, and based on the fact that the new allocation algorithm can be > configured fairly simply now, this is a proposal to go with 4 as the new > default and the allocate_tokens_for_local_replication_factor set to 3. > That gives a good experience out of the box for people and is the most > conservative. It does assume that racks and DCs have been configured > correctly. We would, of course, go into some detail in the NEWS.txt. > > > > Joey Lynch and Josh Snyder did an extensive analysis of availability > concerns with high num_tokens/virtual nodes in their paper < > http://mail-archives.apache.org/mod_mbox/cassandra-dev/201804.mbox/%3CCALShVHcz5PixXFO_4bZZZNnKcrpph-=5QmCyb0M=w-mhdyl...@mail.gmail.com%3E>. > This worsens as clusters grow larger. I won't quote the paper here but in > order to have a conservative default and with the accompanying new > allocation algorithm, I think it makes sense as a default. > > > > The difficulties have always been that virtual nodes have been > beneficial for operations but that 256 is too high for the purposes of > repair and as Joey and Josh cover, for availability. Going lower with the > original allocation algorithm has produced skew in allocation in its naive > distribution. Enter CASSANDRA-7032 < > https://issues.apache.org/jira/browse/CASSANDRA-7032> and the new token > allocation algorithm. CASSANDRA-15260 < > https://issues.apache.org/jira/browse/CASSANDRA-15260> makes the new > algorithm operationally simpler. > > > > One other item of note - since Joey and Josh's analysis, there have been > improvements in streaming and other considerations that can reduce the > probability of more than one node representing some token range being > unavailable, but it would still be good to be conservative. > > > > Please chime in with any concerns with having num_tokens=4 and > allocate_tokens_for_local_replication_factor=3 and the accompanying > rationale so we can improve the experience for all users. > > > > Other resources: > > > https://thelastpickle.com/blog/2019/02/21/set-up-a-cluster-with-even-token-distribution.html > > > https://docs.datastax.com/en/dse/6.7/dse-admin/datastax_enterprise/config/configVnodes.html > > > https://www.datastax.com/blog/2016/01/new-token-allocation-algorithm-cassandra-30 > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > > -- Ben Bromhead Instaclustr | www.instaclustr.com | @instaclustr <http://twitter.com/instaclustr> | (650) 284 9692
