Re: All subsequent CAS requests time out after heavy use of new CAS feature

2016-12-24 Thread horschi
Oh yes it is, like Couters :-) On Sat, Dec 24, 2016 at 4:02 AM, Edward Capriolo wrote: > Anecdotal CAS works differently than the typical cassandra workload. If > you run a stress instance 3 nodes one host, you find that you typically run > into CPU issues, but if you are doing a CAS workload y

Re: All subsequent CAS requests time out after heavy use of new CAS feature

2016-12-23 Thread Edward Capriolo
Anecdotal CAS works differently than the typical cassandra workload. If you run a stress instance 3 nodes one host, you find that you typically run into CPU issues, but if you are doing a CAS workload you see things timing out and before you hit 100% CPU. It is a strange beast. On Fri, Dec 23, 201

Re: All subsequent CAS requests time out after heavy use of new CAS feature

2016-12-23 Thread horschi
Update: I replace all quorum reads on that table with serial reads, and now these errors got less. Somehow quorum reads on CAS values cause most of these WTEs. Also I found two tickets on that topic: https://issues.apache.org/jira/browse/CASSANDRA-9328 https://issues.apache.org/jira/browse/CASSAND

Re: All subsequent CAS requests time out after heavy use of new CAS feature

2016-12-15 Thread horschi
Hi, I would like to warm up this old thread. I did some debugging and found out that the timeouts are coming from StorageProxy.proposePaxos() - callback.isFullyRefused() returns false and therefore triggers a WriteTimeout. Looking at my ccm cluster logs, I can see that two replica nodes return di

Re: All subsequent CAS requests time out after heavy use of new CAS feature

2016-04-15 Thread Denise Rogers
My thinking was that due to the size of the data that there maybe I/O issues. But it sounds more like you're competing for locks and hit a deadlock issue. Regards, Denise Cell - (860)989-3431 Sent from mi iPhone > On Apr 15, 2016, at 9:00 AM, horschi wrote: > > Hi Denise, > > in my case its

Re: All subsequent CAS requests time out after heavy use of new CAS feature

2016-04-15 Thread horschi
Hi Denise, in my case its a small blob I am writing (should be around 100 bytes): CREATE TABLE "Lock" ( lockname varchar, id varchar, value blob, PRIMARY KEY (lockname, id) ) WITH COMPACT STORAGE AND COMPRESSION = { 'sstable_compression' : 'S

Re: All subsequent CAS requests time out after heavy use of new CAS feature

2016-04-15 Thread Denise Rogers
Also, what type of data were you reading/writing? Regards, Denise Sent from mi iPad > On Apr 15, 2016, at 8:29 AM, horschi wrote: > > Hi Jan, > > were you able to resolve your Problem? > > We are trying the same and also see a lot of WriteTimeouts: > WriteTimeoutException: Cassandra timeout

Re: All subsequent CAS requests time out after heavy use of new CAS feature

2016-04-15 Thread horschi
Hi Jan, were you able to resolve your Problem? We are trying the same and also see a lot of WriteTimeouts: WriteTimeoutException: Cassandra timeout during write query at consistency SERIAL (2 replica were required but only 1 acknowledged the write) How many clients were competing for a lock in y

Re: All subsequent CAS requests time out after heavy use of new CAS feature

2013-09-23 Thread Robert Coli
On Mon, Sep 16, 2013 at 9:09 AM, Jan Algermissen wrote: > I am experimenting with C* 2.0 ( and today's java-driver 2.0 snapshot) for > implementing distributed locks. > [ and I'm experiencing the problem described in the subject ... ] > Any idea how to approach this problem? > 1) Upgrade to 2

All subsequent CAS requests time out after heavy use of new CAS feature

2013-09-16 Thread Jan Algermissen
Hi, I am experimenting with C* 2.0 ( and today's java-driver 2.0 snapshot) for implementing distributed locks. Basically, I have a table of 'states' I want to serialize access to: create table state ( id text , lock uuid , data text, primary key (id) ) (3 nodes, replication level 3) in