Re: high latency on one node after replacement

thanks for pointing that out, i just found it too :) i overlooked this On Tue, Mar 27, 2018 at 3:44 PM, Voytek Jarnot wrote: > Have you ruled out EBS snapshot initialization issues ( > https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html)? > > On Tue, Mar 27, 2018 at 2:24 PM,

Re: high latency on one node after replacement

Have you ruled out EBS snapshot initialization issues ( https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html)? On Tue, Mar 27, 2018 at 2:24 PM, Mike Torra wrote: > Hi There - > > I have noticed an issue where I consistently see high p999 read latency on > a node for a few hour

Re: high latency on one node after replacement

Hi Mike, Unlike normal EBS volumes for which you don't need to pre-warm, I think you need to pre-Warm your EBS volume restored from a snapshot Have a look at this AWS doc https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html It says that: However, storage blocks on volumes that

high latency on one node after replacement

Hi There - I have noticed an issue where I consistently see high p999 read latency on a node for a few hours after replacing the node. Before replacing the node, the p999 read latency is ~30ms, but after it increases to 1-5s. I am running C* 3.11.2 in EC2. I am testing out using EBS snapshots of