Could you share a bit your cluster setup? Do you use cloud for your deployment or dedicated firewalls in front of nodes?
If gossip shows that everything is up it doesn't mean that all nodes can communicate with each other. I have noticed situations when TCP connection was killed by firewall and Cassandra didn't reconnect automatically. It can be easily detected with nodetool describecluster command. Aleksandr shows - all nodes are up. > > But when we perform writes , coordinator stores the hints. It means - > coordinator was not able to deliver the writes to few nodes after meeting > consistency requirements. > > The nodes for which writes were failing, are in different DC. Those nodes > do not have any load. > > Gossips shows everything is up. I already set write timeout to 60 sec, > but no help. > > Can anyone encounter this scenario ? Network side everything is fine. > > Cassandra version is 2.1.13 > > -- > *Regards,* > *Anshu * > > >
