Re: [EXTERNAL] Re: adding multiple node to a cluster, cleanup and num_tokens

2018-09-09 Thread Jonathan Haddad
Your example only really applies if someone is using a 20 node cluster at RF=1, something I've never seen, but I'm sure exists somewhere. Realistically, RF=3 using racks (or AWS regions) and 21 nodes, means you'll have 3 racks with 7 nodes per rack. Adding a single node is an unlikely operation, y

Re: [EXTERNAL] Re: adding multiple node to a cluster, cleanup and num_tokens

2018-09-08 Thread Oleksandr Shulgin
On Sat, 8 Sep 2018, 19:00 Jeff Jirsa, wrote: > Virtual nodes accomplish two primary goals > > 1) it makes it easier to gradually add/remove capacity to your cluster by > distributing the new host capacity around the ring in smaller increments > > 2) it increases the number of sources for streamin

Re: [EXTERNAL] Re: adding multiple node to a cluster, cleanup and num_tokens

2018-09-08 Thread onmstester onmstester
Thanks Jeff, You mean that with RF=2, num_tokens = 256 and having less than 256 nodes i should not worry about data distribution? Sent using Zoho Mail On Sat, 08 Sep 2018 21:30:28 +0430 Jeff Jirsa wrote Virtual nodes accomplish two primary goals 1) it makes it easier to gradually add

Re: [EXTERNAL] Re: adding multiple node to a cluster, cleanup and num_tokens

2018-09-08 Thread Jeff Jirsa
Virtual nodes accomplish two primary goals 1) it makes it easier to gradually add/remove capacity to your cluster by distributing the new host capacity around the ring in smaller increments 2) it increases the number of sources for streaming, which speeds up bootstrap and decommission Whether

Re: [EXTERNAL] Re: adding multiple node to a cluster, cleanup and num_tokens

2018-09-08 Thread Jonathan Haddad
> I wonder why not setting it to all the way down to 1 then? What's the key difference once you have so few vnodes? 4 tokens lets you have balanced clusters when they're small and imposes very little overhead when they get big. Using multiple tokens let's multiple nodes stream data to the new nod

Re: [EXTERNAL] Re: adding multiple node to a cluster, cleanup and num_tokens

2018-09-08 Thread Jonathan Haddad
Keep using whatever settings you've been using. I'd still use allocate tokens for keyspace but it probably won't make much of a difference with 256 tokens. On Sat, Sep 8, 2018 at 10:40 AM onmstester onmstester wrote: > Thanks Jon, > But i never concerned about num_tokens config before, because

Re: [EXTERNAL] Re: adding multiple node to a cluster, cleanup and num_tokens

2018-09-08 Thread Oleksandr Shulgin
On Sat, 8 Sep 2018, 14:47 Jonathan Haddad, wrote: > 256 tokens is a pretty terrible default setting especially post 3.0. I > recommend folks use 4 tokens for new clusters, > I wonder why not setting it to all the way down to 1 then? What's the key difference once you have so few vnodes? with s

Re: [EXTERNAL] Re: adding multiple node to a cluster, cleanup and num_tokens

2018-09-08 Thread onmstester onmstester
Thanks Jon, But i never concerned about num_tokens config before, because no official cluster setup documents (on datastax:  https://docs.datastax.com/en/cassandra/3.0/cassandra/initialize/initSingleDS.html or other blogs) warned us-beginners to be concerned about it. I always setup my clusters

Re: [EXTERNAL] Re: adding multiple node to a cluster, cleanup and num_tokens

2018-09-08 Thread Jonathan Haddad
256 tokens is a pretty terrible default setting especially post 3.0. I recommend folks use 4 tokens for new clusters, with some caveats. When you fire up a cluster, there's no way to make the initial tokens be distributed evenly, you'll get random ones. You'll want to set them explicitly using:

RE: [EXTERNAL] Re: adding multiple node to a cluster, cleanup and num_tokens

2018-09-07 Thread onmstester onmstester
Why not setting default vnodes count to that recommendation in Cassandra installation files?  Sent using Zoho Mail On Tue, 04 Sep 2018 17:35:54 +0430 Durity, Sean R wrote   Longer term, I agree with Oleksandr, the recommendation for number of vnodes is now much smaller than 256. I am

RE: [EXTERNAL] Re: adding multiple node to a cluster, cleanup and num_tokens

2018-09-04 Thread Durity, Sean R
I would only run the clean-up (on all nodes) after all new nodes are added. I would also look at increasing RF to 3 (and running repair) once there are plenty of nodes. (This is assuming that availability matters and that your queries use QUORUM or LOCAL_QUORUM for consistency level. Longer ter