Re: Lots of write timeouts and missing data during decomission/bootstrap

2015-07-01 Thread Kevin Burton
WOW.. nice. you rock!! On Wed, Jul 1, 2015 at 3:18 PM, Robert Coli wrote: > On Wed, Jul 1, 2015 at 2:58 PM, Kevin Burton wrote: > >> Looks like all of this is happening because we’re using CAS operations >> and the driver is going to SERIAL consistency level. >> ... >> This is Cassandra 2.0.9 b

Re: Lots of write timeouts and missing data during decomission/bootstrap

2015-07-01 Thread Robert Coli
On Wed, Jul 1, 2015 at 2:58 PM, Kevin Burton wrote: > Looks like all of this is happening because we’re using CAS operations and > the driver is going to SERIAL consistency level. > ... > This is Cassandra 2.0.9 btw. > > https://issues.apache.org/jira/browse/CASSANDRA-8640 =Rob (credit to iamal

Re: Lots of write timeouts and missing data during decomission/bootstrap

2015-07-01 Thread Kevin Burton
Looks like all of this is happening because we’re using CAS operations and the driver is going to SERIAL consistency level. SERIAL and LOCAL_SERIAL write failure scenarios¶ > >

Lots of write timeouts and missing data during decomission/bootstrap

2015-07-01 Thread Kevin Burton
We get lots of write timeouts when we decommission a node. About 80% of them are write timeout and just about 20% of them are read timeout. We’ve tried to adjust streamthroughput (and compaction throughput) for that matter and that doesn’t resolve the issue. We’ve increased write_request_timeout