Hello Alain , Good Morning & thank you for these details , very interesting your calcul for the node and vnodes thanks for your example for calculate the tokens in the cluster and i gonna read & take my class of maths for understand the formule and document of mr Joey Lynch i thin so very interesting !!!
best regards & we rest in contact Gracias y buen dia Cordialement Victor Ibarra Le lun. 21 janv. 2019 à 15:58, Alain RODRIGUEZ <arodr...@gmail.com> a écrit : > Sure, it's called "Cassandra Availability with Virtual Nodes”, by Joey > Lynch and Josh Snyder. > > I found it in the mailing list archives: > https://github.com/jolynch/python_performance_toolkit/blob/master/notebooks/cassandra_availability/whitepaper/cassandra-availability-virtual.pdf > > There are some maths in there to explain impacts of the number of vnodes > on availability. > > Using the formula "1d", and considering a datacenter of 3 balanced racks > with RF = 3, we have: > > Np*(1-(1-(1/Np))^(v*2*(R-1)) = 40*(1-(1-(1/40))^(256*2*(3-1)) = > 39.9999999998 > Thus if my calculation is accurate, with 60 nodes and 256 vnodes, we > expect a node to have 39.9999999998 neighbors. This means that with 60 > nodes, *each node* has 40 *possible* replicas (all the nodes in other > racks) and will be sharing a token range with all the other nodes. Thus 2 > nodes down in distinct racks and you have an outage almost ensured (still > needs 2 nodes down). > > Some other arbitrary numbers that show the evolution of this value > depending on the number of nodes and vnodes. > > - With 60 nodes and 256 vnodes, expect 39.9999999998 neighbors > - With 60 nodes and 16 vnodes, expect 32.0867407145 neighbors > - With 60 nodes and 4 vnodes, expect 13.323193263 neighbors > > - With 300 nodes and 256 vnodes, expect 198.8200470802 neighbors > - With 300 nodes and 16 vnodes, expect 54.8867183963 neighbors > - With 300 nodes and 4 vnodes, expect 15.4137752052 neighbors > > > Good reading :). > > C*heers, > ----------------------- > Alain Rodriguez - al...@thelastpickle.com > France / Spain > > The Last Pickle - Apache Cassandra Consulting > http://www.thelastpickle.com > > Le lun. 21 janv. 2019 à 13:30, VICTOR IBARRA <vic2...@gmail.com> a écrit : > >> Hi Alain , >> >> thank you very much for the explication and the points for the sujet of >> managing de vnodes >> >> you talk about the paper of netflix and the outage ? you have the link >> with this discution >> >> thank you for your help >> BEST REGARDS >> >> Le lun. 21 janv. 2019 à 13:53, Alain RODRIGUEZ <arodr...@gmail.com> a >> écrit : >> >>> There have been some discussion on this topic in this mailing list, >>> including a paper from Netflix with the impact of vnodes. I could not find >>> it quickly, but I invite you to check. >>> >>> To share some ideas: >>> >>> More vnodes: >>> + Better balance between nodes >>> + maximize the streaming throughput for operations as all nodes share a >>> small bit of the data of all the other nodes (according to the topology). >>> - When the cluster fails, there is more chance to lose availability as >>> we 256 vnodes for example, 2 nodes down in distinct racks would for sure >>> make data partially unavailable. >>> - Overheads / Operational issues (in practice, using 256 vnodes have >>> been a nightmare for multiple reasons, see below) >>> >>> >>> Less vnodes >>> - Imbalances can be big before C* 3.0. After, using >>> allocate_tokens_for_keyspace --> >>> http://cassandra.apache.org/doc/4.0/configuration/cassandra_config_file.html#allocate-tokens-for-keyspace, >>> you can mitigate this issue. With this and some technics*, you can have >>> good results in terms of balances. >>> * Off the top of my head: this involve bootstrapping the seeds first, >>> picking the tokens to use, create your keyspace then adding nodes with the >>> option above. You can test it quite easily. Then with "nodetool status >>> <keyspace" you can check the ownership balance. >>> - The streaming throughput is generally limited by the receiving host >>> when using vnodes, thus 16 vnodes is probably not worse than 256 in terms >>> of streaming >>> + The other way around, the overhead of having 256 vnodes makes >>> operations such as repair almost impossible, or at least way longer and >>> complex. Repairing tables almost empty can take up to minutes and repairing >>> big dataset might never end. >>> + In Netflix paper about this topic (very interesting, I recommend >>> reading), it is explained that reducing the number of vnodes reduces the >>> chances of an outage. >>> + There was a discussion in the dev mailing list. I believe the >>> community agreed on the need to reduce the number of vnodes by default. >>> Here again, you can have a quick look at the archive, Jira, github/trunk. >>> >>> I think that commonly accepted values would be 16/32. Values as low as 4 >>> are considered to improve availability, reduce overheads induced by vnodes. >>> I would suggest you test it and see if low values you still manage to keep >>> the balance between nodes. >>> >>> Also using "physical" nodes (initial_token, no vnodes) gives the >>> possibility to reason about token distribution. You can perform advanced >>> operations >>> where you bootstrap 1/3 of the cluster at once. This is very good >>> especially for big clusters, I would say. While with many vnodes you'll >>> have to go add a node at the time as each node is actually the 'neighbor' >>> of all the others (according to the topology again - ie racks/data >>> centers...). >>> >>> I would stay away from the default in this case (256 vnodes). I think >>> this value is way too high by default. >>> >>> Also, keep in mind that to change the number of vnodes cannot be changed >>> in a running cluster. The best way to change it is to add a new data center >>> I think. >>> >>> C*heers, >>> ----------------------- >>> Alain Rodriguez - al...@thelastpickle.com >>> France / Spain >>> >>> The Last Pickle - Apache Cassandra Consulting >>> http://www.thelastpickle.com >>> >>> >>> Le lun. 21 janv. 2019 à 11:08, VICTOR IBARRA <vic2...@gmail.com> a >>> écrit : >>> >>>> >>>> Good morning every one, >>>> >>>> I would like have a contact with the cassandra community for the >>>> questions of cluster configuration >>>> >>>> Today i have many questions and differents projets about the >>>> configuration of cluster cassandra and with the general problems of >>>> configuration migration and for the use of vnodes. >>>> >>>> and the principal question is what about the gain to use 256 vnodes vs >>>> 16 vnodes for example >>>> >>>> Best regards >>>> -- >>>> L'integrité de ce message n'étant pas assurée sur internet, VICTOR >>>> IBARRA ne peut être tenue responsable de son contenu en ce compris les >>>> pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. >>>> Si vous n'êtes pas destinataire de ce message, merci de le détruire et >>>> d'avertir l'expéditeur. >>>> >>>> The integrity of this message cannot be guaranteed on the Internet. >>>> VICTOR IBARRA can not therefore be considered liable for the contents >>>> including its attachments. Any unauthorized use or dissemination is >>>> prohibited. If you are not the intended recipient of this message, then >>>> please delete it and notify the sender. >>>> >>> >> >> -- >> L'integrité de ce message n'étant pas assurée sur internet, VICTOR >> IBARRA ne peut être tenue responsable de son contenu en ce compris les >> pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. >> Si vous n'êtes pas destinataire de ce message, merci de le détruire et >> d'avertir l'expéditeur. >> >> The integrity of this message cannot be guaranteed on the Internet. >> VICTOR IBARRA can not therefore be considered liable for the contents >> including its attachments. Any unauthorized use or dissemination is >> prohibited. If you are not the intended recipient of this message, then >> please delete it and notify the sender. >> > -- L'integrité de ce message n'étant pas assurée sur internet, VICTOR IBARRA ne peut être tenue responsable de son contenu en ce compris les pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur. The integrity of this message cannot be guaranteed on the Internet. VICTOR IBARRA can not therefore be considered liable for the contents including its attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.
