No dropped messages in tpstats on any of the nodes. On Mon, Mar 20, 2017 at 3:11 PM, Voytek Jarnot <voytek.jar...@gmail.com> wrote:
> Appreciate the reply, Kurt. > > I sanitized it out of the traces, but all trace outputs listed the same > node for all three queries (1 working, 2 not working). Read repair chance > set to 0.0 as recommended when using TWCS. > > I'll check tpstats - in this environment, load is not an issue, but > network issues may be. > > On Mon, Mar 20, 2017 at 2:42 PM, kurt greaves <k...@instaclustr.com> > wrote: > >> As secondary indexes are stored individually on each node what you're >> suggesting sounds exactly like a consistency issue. the fact that you read >> 0 cells on one query implies the node that got the query did not have any >> data for the row. The reason you would sometimes see different behaviours >> is likely because of read repairs. The fact that the repair guides the >> issue pretty much guarantees it's a consistency issue. >> >> You should check for dropped mutations in tpstats/logs and if they are >> occurring try and stop that from happening (probably load related). You >> could also try performing reads and writes at LOCAL_QUORUM for stronger >> consistency, however note this has a performance/latency impact. >> >> >> >
