Re: Crash with TombstoneOverwhelmingException

2014-01-17 Thread Shao-Chuan Wang
Agree with Robert about the dogfood. http://www.datastax.com/docs/datastax_enterprise3.2/dse_release_notes#rn-3-2-4 It may be a good indicator when DSE starts using Cassandra 2.x.y in production. > From: Robert Coli > Date: Mon, Dec 30, 2013 at 2:58 PM > Subject: Re: C

Re: Crash with TombstoneOverwhelmingException

2014-01-16 Thread Cyril Scetbon
With cassandra an update is equivalent to an insert Cyril Scetbon > Le 14 janv. 2014 à 08:38, David Tinker a écrit : > > We never delete rows but we do a lot of updates. Is that where the > tombstones are coming from?

Re: Crash with TombstoneOverwhelmingException

2014-01-13 Thread David Tinker
We are seeing the exact same exception in our logs. Is there any workaround? We never delete rows but we do a lot of updates. Is that where the tombstones are coming from? On Wed, Dec 25, 2013 at 5:24 PM, Sanjeeth Kumar wrote: > Hi all, > One of my cassandra nodes crashes with the following ex

Re: Crash with TombstoneOverwhelmingException

2013-12-30 Thread Robert Coli
On Wed, Dec 25, 2013 at 10:01 AM, Edward Capriolo wrote: > I have to hijack this thread. There seem to be many problems with the > 2.0.3 release. > +1. There is no 2.0.x release I consider production ready, even after today's 2.0.4. Outside of passing all unit tests, factors into the release vo

Re: Crash with TombstoneOverwhelmingException

2013-12-27 Thread Kais Ahmed
You can read the comments about this new feature here : https://issues.apache.org/jira/browse/CASSANDRA-6117 2013/12/27 Kais Ahmed > This threshold is to prevent bad performance, you can increase the value > > > 2013/12/27 Sanjeeth Kumar > >> Thanks for the replies. >> I dont think this is ju

Re: Crash with TombstoneOverwhelmingException

2013-12-27 Thread Kais Ahmed
This threshold is to prevent bad performance, you can increase the value 2013/12/27 Sanjeeth Kumar > Thanks for the replies. > I dont think this is just a warning , incorrectly logged as an error. > Everytime there is a crash, this is the exact traceback I see in the logs. > I just browsed thro

Re: Crash with TombstoneOverwhelmingException

2013-12-27 Thread Sanjeeth Kumar
Thanks for the replies. I dont think this is just a warning , incorrectly logged as an error. Everytime there is a crash, this is the exact traceback I see in the logs. I just browsed through the code and the code throws a TombstoneOverwhelmingException exception in these situations and I did not s

Re: Crash with TombstoneOverwhelmingException

2013-12-26 Thread Edward Capriolo
I do not think the feature is supposed to crash the server. It could be that the message is the logs and the crash is not related to this message. WARN might be a better logging level for any message, even though the first threshold is WARN and the second is FAIL. ERROR is usually something more dr

Re: Crash with TombstoneOverwhelmingException

2013-12-25 Thread Laing, Michael
It's a feature: In the stock cassandra.yaml file for 2.03 see: # When executing a scan, within or across a partition, we need to keep the > # tombstones seen in memory so we can return them to the coordinator, which > # will use them to make sure other replicas also know about the deleted > rows.

Re: Crash with TombstoneOverwhelmingException

2013-12-25 Thread Edward Capriolo
I have to hijack this thread. There seem to be many problems with the 2.0.3 release. If this exception is being generated by hinted-handoff, I could understand where it is coming from. If you have many hints and many tombstones then this new feature interacts with the hint delivery process, in a b

Re: Crash with TombstoneOverwhelmingException

2013-12-25 Thread Rahul Menon
Sanjeeth, Looks like the error is being populated from the hintedhandoff, what is the size of your hints cf? Thanks Rahul On Wed, Dec 25, 2013 at 8:54 PM, Sanjeeth Kumar wrote: > Hi all, > One of my cassandra nodes crashes with the following exception > periodically - > ERROR [HintedHandoff

Crash with TombstoneOverwhelmingException

2013-12-25 Thread Sanjeeth Kumar
Hi all, One of my cassandra nodes crashes with the following exception periodically - ERROR [HintedHandoff:33] 2013-12-25 20:29:22,276 SliceQueryFilter.java (line 200) Scanned over 10 tombstones; query aborted (see tombstone_fail_thr eshold) ERROR [HintedHandoff:33] 2013-12-25 20:29:22,278 Ca