On Wed, Sep 23, 2015 at 7:28 AM, Björn Hachmann <bjoern.hachm...@metrigo.de> wrote:
> Today I realized that one of the nodes in our Cassandra cluster (2.1.7) is > storing a lot of hints (>80GB) and I fail to see a convincing way to deal > with them. > ... > We had a look into the table system.hints and from there we learnt that > most hints > are for one of the nodes in our 2nd datacenter and most of the mutations > are > increments to one of our counter tables which are very frequent. > This is probably timeouts on the increment creating your hints. > We have several questions: > - What could be the reason that only one of the nodes has hints for only > one target node, altough every other node should be coordinator for these > queries sometimes also? > That sounds unexpected, I don't have a good answer. > - Is there a way to turn of hinted handoff on a table level or on data > center level? > No. > - What could we do to investigate the cause of this issue deeper? > Are the hints being successfully delivered? It sounds like not.. =Rob