On Mon, 2013-06-03 at 17:20 -0700, Aiman Parvaiz wrote: > @Faraaz check out the comment by Aaron morton here : > http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Seed-Nodes-td6077958.html > Having same nodes is a good idea but it is not necessary. > > In your case, sure the nodes will be in the cluster for 10 > > minutes but what about sporadic failures that cause them to leave the ring > > and > > then re-enter it? At that point, you might reach the network fragmentation > > issue. > > I am not sure I understand this completely, if a node leaves the ring and re > enters it would use its seed nodes to know about the ring and these nodes > would be the ones which are part of ring so I don't see any information lag > happening here.
As I understand the risk: If you start with four nodes in the ring: 10.1.1.1 10.1.1.2 10.2.1.1 10.2.1.2 .. Then the connection between 10.1.0.0/16 and 10.2.0.0/16 is lost temporarily through some routing issue / firewall config etc. You run puppet to re-deploy on 10.1.1.1 and 10.1.1.2 . When they discover nodes on the current ring, they only find the 10.1.0.0 nodes. You do the same on 10.2.1.1 and 10.2.1.2, and they only find the nodes on that segment of the ring (I'm not used to puppet - if the config generation is run once on a single machine then you could still end up with that by chance assuming that machine can still reach both parts of the cluster) At this point your networking team spot the routing issue and fix it - but it's too late. So you now have two disjoint rings running, which don't have any of the other subring nodes in their seed nodes respectively. .. all that is fairly unlikely to happen, but it's possible - and the more switches and machines that are involved the more likely it becomes that it will happen. I'm not sure if that's the only risk - but if it is then IMHO it's a question of if that risk is acceptable, and if so then adding monitoring to continually check for that situation. Tim
