Hi another tip, make sure the OS doesn't come with pre-configured NTP synchronisation services. We had a proper NTP setup, but we missed a service that came with CentOS that synced to a low stratum NTP server. -- Brice
On Thu, Mar 31, 2016 at 10:00 AM -0700, "Eric Evans" <eev...@wikimedia.org> wrote: On Wed, Mar 30, 2016 at 8:07 PM, Mukil Kesavan wrote: > Are there any issues if this causes a huge time correction on the cassandra > cluster? I know that NTP gradually corrects the time on all the servers. I > just wanted to understand if there were any corner cases that will cause us > to lose data/schema updates when this happens. In particular, we seem to be > having some issues around missing secondary indices at the moment (not all > but some). As a thought experiment, imagine every scenario where it matters to have one write occur after another (an update followed by a delete is a good example). Now imagine having your clock yanked backward to correct for drift between the first such operation and the second. I would strongly recommend you come up with a stable NTP setup. -- Eric Evans eev...@wikimedia.org
