Thanks Mick.
We're on datastax java driver version: 2.1.10.2 and we aren't using the client side timestamps. Anyway, we went ahead and verified that all client machines and cassandra machines are in sync with regards to time. We've also verified that no reads and writes are going to the remote data center. Here's a bit more information: -Few rows in this column family can grow quite wide (> 100K columns) -But we keep seeing this behavior most frequently with rows with just 1 or two columns . The typical behavior is: Machine A adds a new row and a column. 30-60 seconds later Machine B tries to read this row. It doesn't find the row. So the application retries within 500ms. This time it finds the row. Thanks. ________________________________ From: Mick Semb Wever <m...@thelastpickle.com> Sent: Saturday, October 20, 2018 10:24:53 PM To: user@cassandra.apache.org Subject: Re: Cassandra: Inconsistent data on reads (LOCAL_QUORUM) [ This email has been sent from a source external to Epsilon. Please use caution when clicking links or opening attachments. ] > Thanks James. Yeah, we're using the datastax java driver. But we're on > version 2.1.10.2. And we are not using the client side timestamps. Just to check Ninad. If you are using Cassandra-2.1 (native protocol v3) and the java driver version 3.0 or above, then you would be using client-side timestamps by default. https://github.com/datastax/java-driver/tree/3.x/manual/query_timestamps With client-side timestamps all client servers and all C* nodes must be kept tightly in-sync, as Elliot said. Monitoring and alerting on any clock skew on any of these machines is important. Also worth checking that any local_quorum requests are not accidentally go to the wrong datacenter. regards, Mick --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org The information contained in this e-mail message and any attachments may be privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that any review, dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by replying to this e-mail and delete the message and any attachments from your computer.
