Re: Lots of Deleted Rows Came back after upgrade 1.1.6 to 1.1.10

2013-04-01 Thread Arya Goudarzi
nactive SSTables and some bug exposed itself >> and marked them as active? What are the possibilities that could lead to >> this? I am eager to find our as this is blocking our upgrade. >> >> On Tue, Mar 19, 2013 at 2:11 AM, wrote: >> >>> This obscure feature of

Re: Lots of Deleted Rows Came back after upgrade 1.1.6 to 1.1.10

2013-03-31 Thread aaron morton
gt;> and marked them as active? What are the possibilities that could lead to >>> this? I am eager to find our as this is blocking our upgrade. >>> >>> On Tue, Mar 19, 2013 at 2:11 AM, wrote: >>> This obscure feature of Cassandra is called “haunted handoff”. >>>

Re: Lots of Deleted Rows Came back after upgrade 1.1.6 to 1.1.10

2013-03-28 Thread Arya Goudarzi
? I am eager to find our as this is blocking our upgrade. > > On Tue, Mar 19, 2013 at 2:11 AM, wrote: > >> This obscure feature of Cassandra is called “haunted handoff”. >> >> ** ** >> >> Happy (early) April Fools J**** >> >> ** **

Re: Lots of Deleted Rows Came back after upgrade 1.1.6 to 1.1.10

2013-03-24 Thread aaron morton
Happy (early) April Fools J >> >> >> >> From: aaron morton [mailto:aa...@thelastpickle.com] >> Sent: Monday, March 18, 2013 7:45 PM >> To: user@cassandra.apache.org >> Subject: Re: Lots of Deleted Rows Came back after upgrade 1.1.6 to 1.1.10 >> >>

Re: Lots of Deleted Rows Came back after upgrade 1.1.6 to 1.1.10

2013-03-23 Thread aaron morton
April Fools J > > > > From: aaron morton [mailto:aa...@thelastpickle.com] > Sent: Monday, March 18, 2013 7:45 PM > To: user@cassandra.apache.org > Subject: Re: Lots of Deleted Rows Came back after upgrade 1.1.6 to 1.1.10 > > > > As you see, this node thin

Re: Lots of Deleted Rows Came back after upgrade 1.1.6 to 1.1.10

2013-03-21 Thread Arya Goudarzi
ted handoff”. > > ** ** > > Happy (early) April Fools J > > ** ** > > *From:* aaron morton [mailto:aa...@thelastpickle.com] > *Sent:* Monday, March 18, 2013 7:45 PM > *To:* user@cassandra.apache.org > *Subject:* Re: Lots of Deleted Rows Came

RE: Lots of Deleted Rows Came back after upgrade 1.1.6 to 1.1.10

2013-03-19 Thread moshe.kranc
This obscure feature of Cassandra is called "haunted handoff". Happy (early) April Fools :) From: aaron morton [mailto:aa...@thelastpickle.com] Sent: Monday, March 18, 2013 7:45 PM To: user@cassandra.apache.org Subject: Re: Lots of Deleted Rows Came back after upgrade 1.1.6 to 1.1.10

Re: Lots of Deleted Rows Came back after upgrade 1.1.6 to 1.1.10

2013-03-18 Thread Arya Goudarzi
Hi Aaron: Thanks for your attention. The cluster in question is a 4 node sandbox cluster we have that does not have much traffic. I was able to chase down this issue on a CF that doesn't change much. That bug was flagged as fixed on 1.1.10. They were row level deletes. We use the nanosecond pr

Re: Lots of Deleted Rows Came back after upgrade 1.1.6 to 1.1.10

2013-03-18 Thread aaron morton
> As you see, this node thinks lots of ranges are out of sync which shouldn't > be the case as successful repairs where done every night prior to the > upgrade. Could this be explained by writes occurring during the upgrade process ? > I found this bug which touches timestamp and tomstones whi

Lots of Deleted Rows Came back after upgrade 1.1.6 to 1.1.10

2013-03-15 Thread Arya Goudarzi
Hi, I have upgraded our test cluster from 1.1.6 to 1.1.10. Followed by running repairs. It appears that the repair task that I executed after upgrade, brought back lots of deleted rows into life. Here are some logistics: - The upgraded cluster started from 1.1.1 -> 1.1.2 -> 1.1.5 -> 1.1.6 - Old c