And also aws ec2 stop and start comes with new instance with same ip and all our file systems are in ebs mounted fine. Does coming new instance with same ip cause any gossip issues?
On Tue, Oct 29, 2019, 6:16 PM Rahul Reddy <rahulreddy1...@gmail.com> wrote: > Thanks Alex. We have 6 nodes in each DC with RF=3 with CL local qourum . > and we stopped and started only one instance at a time . Tough nodetool > status says all nodes UN and system.log says canssandra started and started > listening . Jmx explrter shows instance stayed down longer how do we > determine what caused the Cassandra unavialbe though log says its stared > and listening ? > > On Tue, Oct 29, 2019, 4:44 PM Oleksandr Shulgin < > oleksandr.shul...@zalando.de> wrote: > >> On Tue, Oct 29, 2019 at 9:34 PM Rahul Reddy <rahulreddy1...@gmail.com> >> wrote: >> >>> >>> We have our infrastructure on aws and we use ebs storage . And aws was >>> retiring on of the node. Since our storage was persistent we did nodetool >>> drain and stopped and start the instance . This caused 500 errors in the >>> service. We have local_quorum and rf=3 why does stopping one instance cause >>> application to have issues? >>> >> >> Can you still look up what was the underlying error from Cassandra driver >> in the application logs? Was it request timeout or not enough replicas? >> >> For example, if you only had 3 Cassandra nodes, restarting one of them >> reduces your cluster capacity by 33% temporarily. >> >> Cheers, >> -- >> Alex >> >>
