column with TTL of 10 seconds lives very long...

Hi! I have Cassandra cluster with 3 node running version 1.0.11. I am using Hector HLockManagerImpl, which creates a keyspace named HLockManagerImpl and CF HLocks. For some reason I have a row with single column that should have expired yesterday who is still there. I tried deleting it using cli,

Re: Creating namespace and column family from multiple nodes concurrently

I am assuming here you want to sync all the 100s of nodes once the application is airborne. I suspect this would flood the network and even potentially affect the machine itself memory-wise. How are you going to maintain the nodes (compaction+repair)? Regards, Arthur -Original

Re: Creating namespace and column family from multiple nodes concurrently

Others on this list may know better, but it might not be good to have all your clients create a schema on initialization. They can all use it once it has been created, but creating it should be done by a single entity. One of the issues that can come up if you make changes to schemas at the same ti

Creating namespace and column family from multiple nodes concurrently

Hi all, I am implementing a distributed application which runs on 100s of machines concurrently. This application is going to use Cassandra as underlaying storage. The application creates the schema (name space and column families) during initialization phase.  It seems I have two options to cr

exception causes streaming to hang forever

We had 3 nodes roll on good and the next 2, we see a remote node with this exception every time we start over and bootstrap the node ERROR [Streaming to /10.10.42.36:2] 2013-05-22 14:47:59,404 CassandraDaemon.java (line 132) Exception in thread Thread[Streaming to /10.10.42.36:2,5,main] java.la

Re: High performance disk io

without VNodes, during repair -pr, it will stream data for all the replicates and repair all of them. So it will impact RF number of nodes. In the case of VNodes, the streaming/compaction should happen to all the physical nodes. I heard the repair is even worse for VNodes Test it and see ho

Re: High performance disk io

If you are only running repair on one node, should it not skip that node? So there should be no performance hit except when doing CL_ALL of course. We had to make a change to cassandra or slow nodes did impact us previously. Dean From: Wei Zhu mailto:wz1...@yahoo.com>> Reply-To: "user@cassand

Re: High performance disk io

For us, the biggest killer is repair and compaction following repair. If you are running VNodes, you need to test the performance while running repair. - Original Message - From: "Igor" To: user@cassandra.apache.org Sent: Wednesday, May 22, 2013 7:48:34 AM Subject: Re: High performa

For those using Cassandra from .Net

NativeX is giving a talk about using Cassandra with .Net. Our firm created a port of Hector over to .Net late last year. Here is the abstract. *The Perils and Triumphs of using Cassandra at a .NET/Microsoft Shop* *Speakers:* Derek Bromenshenkel and Jeff Smoley, Infrastructure Architects at Nativ

Re: Cassandra 1.2 TTL histogram problem

Thanks for the answer. It means that if we use randompartioner it will be very difficult to find a sstable without any overlap. Let me give you an example from my test. I have ~50 sstables in total and an sstable with droppable ratio 0.9. I use GUID for key and only insert (no update -delete) s

Re: Upgrade Cassandra from version 1.2.3 to 1.2.5

On Wed, May 22, 2013 at 4:26 AM, Gabriel Ciuloaica wrote: > Is there any known issue of upgrading Cassandra from 1.2.3 directly to > 1.2.5? Consult NEWS.txt for the answer to this question. https://github.com/apache/cassandra/blob/trunk/NEWS.txt =Rob

Re: Cassandra 1.2 TTL histogram problem

> Can method calculate non-overlapping keys as overlapping? Yes. And randomized keys don't matter here since sstables are sorted by "token" calculated from key by your partitioner, and the method uses sstable's min/max token to estimate overlap. On Tue, May 21, 2013 at 4:43 PM, cem wrote: > Than

Re: High performance disk io

On 05/22/2013 05:41 PM, Christopher Wirt wrote: Hi Igor, Yea same here, 15ms for 99^th percentile is our max. Currently getting one or two ms for most CF. It goes up at peak times which is what we want to avoid. Our 99 percentile also goes up at peak times but stay at acceptable level. We

RE: High performance disk io

Hi Dean, Adding nodes is the easy way out. We can get three smaller SSDs for the same price as our current setup. How do we optimise performance for this? Is it worth the effort? To RAID or not to RAID, that is one of my questions. Currently I'm thinking it must be faster and given the same pr

RE: High performance disk io

Hi Igor, Yea same here, 15ms for 99th percentile is our max. Currently getting one or two ms for most CF. It goes up at peak times which is what we want to avoid. We're using Cass 1.2.4 w/vnodes and our own barebones driver on top of thrift. Needed to be .NET so Hector and Astyanax were no

Re: High performance disk io

Well, if you just want to lower your I/O util %, you could always just add more nodes to the cluster ;). Dean From: Igor mailto:i...@4friends.od.ua>> Reply-To: "user@cassandra.apache.org" mailto:user@cassandra.apache.org>> Date: Wednesday, May 22, 2013 8:06 AM

Re: High performance disk io

Hello What level of read performance do you expect? We have limit 15 ms for 99 percentile with average read latency near 0.9ms. For some CF 99 percentile actually equals to 2ms, for other - to 10ms, this depends on the data volume you read in each query. Tuning read performance involved clea

High performance disk io

Hello, We're looking at deploying a new ring where we want the best possible read performance. We've setup a cluster with 6 nodes, replication level 3, 32Gb of memory, 8Gb Heap, 800Mb keycache, each holding 40/50Gb of data on a 200Gb SSD and 500Gb SATA for OS and commitlog Three column fam

Re: Unreachable Nodes

I had to face this too, but precisely the "unsafeAssassinateEndpoint" removed the "UNREACHABLE" nodes (from describe cluster - CLI). After that, I had these ghost host marked as "STATUS:LEFT" on gossipinfo (nodetool) and my truncate could run properly. But this is only my own experience, and you mi

Re: Unreachable Nodes

Have to disagree with the "does no harm" comment just a tiny bit. I had a similar situation recently and coincidentally needed to do a CF truncate. The system rejected the request saying that not all nodes were up. Nodetool ring said everyone was up but nodetool gossipinfo said there were vestig

Upgrade Cassandra from version 1.2.3 to 1.2.5

Hi, Is there any known issue of upgrading Cassandra from 1.2.3 directly to 1.2.5? Thank you, Gabi

Re: Unreachable Nodes

Hello, Thanks for your fast response. That makes sense. I'll just keep an eye on it then. Many thanks, Vasilis On Wed, May 22, 2013 at 10:54 AM, Alain RODRIGUEZ wrote: > Hi. > > I think that the "unsafeAssassinateEndpoint" was the good solution here. I > was going to lead you to this solution

Re: Unreachable Nodes

Hi. I think that the "unsafeAssassinateEndpoint" was the good solution here. I was going to lead you to this solution after reading the first part of your message. "Does anyone know why the dead nodes still appear when we run "nodetool gossipinfo" but they don't when we run "describe cluster" fro

Unreachable Nodes

Hello All, A while ago we had 3 cassandra nodes on Amazon. At some point we decided to buy some servers and deploy cassandra there. The problem is that since then we have a list of dead IPs listed as UNREACHABLE nodes when we run describe cluster on cassandra-cli. I have seen other posts which de