I believe what's going on here is this step:
Select Count (*) From MYTABLE;---> 15 rows Shut down Node B. Start Up Node B. Select Count (*) From MYTABLE;---> 15 rows To understand why this is an issue, consider the way that consistency is attempted within Cassandra. With RF=2, (You should really use an odd number RF and LOCAL_QUORUM so you can tolerate a node failure, but that's another thing), your write is hitting Node B, and being queued for writing to Node A via a process called hinted handoff. Normally, this handoff occurs when Node A returns to the cluster, up to max_hint_window_in_ms later, causing all writes it missed to be replayed and integrated. However, since Node B also goes down during this time period, it loses the queued hints and therefore Node A never gets that write. You may see this flip flopping due to your query hitting Node A and Node B alternately (you can use trace to verify this). Keep in mind that due to Cassandra's architecture, missing writes will result in inconsistent data. There are mechanisms to help mitigate this, for example the aforementioned hinted handoff, or read repair. However, at the end of the day the only way to ensure consistent data is a repair. These mechanisms cannot operate reliably if the entire cluster goes down- which happens in your scenario between the above steps. On Mon, Nov 2, 2015 at 12:46 PM, Luis Miguel <arb...@hotmail.com> wrote: > Thanks for your answer! > I thought that bootstrapping is executed only when you add a node to the > cluster the first time after that I thought tgat gossip is the method used > to discover the cluster members again....In my case I thought that it was > more about a read repair issue.., am I wrong? > > ------------------------------ > Date: Mon, 2 Nov 2015 21:12:20 +0100 > Subject: Re: FW: Two node cassandra cluster doubts > From: ichi.s...@gmail.com > To: user@cassandra.apache.org > > > I think that this is a normal behaviour as you shut down your seed and > then reboot it. You should know that when you start a seed node it doesn't > do the bootstrapping thing. Which means it doesn't look if there are > changes in the contents of the tables. In here in your tests, you shut down > node A before doing the inserts and started it after. So you node A doesn't > have the new rows you inserted. And yes it is normal to have different > values of your query each time. Because the coordinator node changes and > therfore the query is executed each time on a different node ( when node > B answers you've got 15 rows and WHE node A does you have 10 rows) > Le 2 nov. 2015 19:22, "Luis Miguel" <arb...@hotmail.com> a écrit : > > Hello! > > I have set a cassandra cluster with two nodes, Node A and Node B --> RF=2, > Read CL=1 and Write CL = 1; > > Node A is seed... > > > At first everything is working well, when I add/delete/update entries on > Node A, everything is replicated on Node B and vice-versa, even if I shut > down node A, and I made new insertions on Node B meanwhile, and After that > I start up node A again Cassandra recovers OK....BUT there is ONE case when > this situation fails.... I am going to describe the process: > > Node A and Node B are sync. > > Select Count (*) From MYTABLE;---> 10 rows > > Shut down Node A. > > Made some inserts on Node B. > > Select Count (*) From MYTABLE;---> 15 rows > > Shut down Node B. > > Start Up Node B. > > Select Count (*) From MYTABLE;---> 15 rows > > (Everything Ok, yet). > > Start Up Node A. > > Select Count (*) From MYTABLE;---> 10 rows (uhmmm...this is weird...check > it again) > Select Count (*) From MYTABLE;---> 15 rows (wow!..this is correct, lets > try again) > Select Count (*) From MYTABLE;---> 10 rows (Ok...values are dancing) > > If I made the same queries on NODE B it Behaves the same way.... and it > only is solved with a nodetool repair...but I would prefer an automatic > fail-over... > > is there any way to avoid this??? or a nodetool repair execution is > mandatory??? > > Thanks in advance!!! > >
