Re: performance reaching plateau while the hardware is still idle

2011-12-15 Thread Peter Tillotson
May I suggest dstat, does cpu, memory, and io on one console dstat -vn 3 So possible causes  * Not sufficient parrallelism in the client  * Server has too few threads  * You are not CPU bound, network or disk may be the bottleneck  p - Original Message - From: Kent Tong To: "user@cassa

Re: data agility

2011-11-21 Thread Peter Tillotson
"Cassandra's write path is completely optimized. It will write as fast as the disk will allow it" I don't believe this is quite true, as posted recently out-the-box compaction is switched on, compaction is often necessary, and requires significant IO.  Cassandra's compaction is okay but there is

Re: Debian package jna bug workaroung

2011-11-07 Thread Peter Tillotson
lizingCacheProvider, it's off-heap.   Note that I think you need to create the columnfamily while JNA is present, not just have JNA present when cassandra starts.  Might be wrong on that. p On Thu, Nov 3, 2011 at 4:10 PM, Peter Tillotson wrote: Cassandra 1.0.1 and only seemed to happen

Re: Debian package jna bug workaroung

2011-11-03 Thread Peter Tillotson
va/jna.jar to the classpath, and that should be all you need. > > The failure you see with o.a.c.cache.FreeableMemory is not because the > jre can't find the class, it's just that it can't initialize the class > (because it needs JNA, and it can't find JNA). > >

Re: Second Cassandra users survey

2011-11-03 Thread Peter Tillotson
To: user@cassandra.apache.org; Peter Tillotson Sent: Thursday, 3 November 2011, 14:15 Subject: Re: Second Cassandra users survey On Thu, Nov 3, 2011 at 5:46 AM, Peter Tillotson wrote: > I'm using Cassandra as a big graph database, loading large volumes of data > live and linking on the fl

Re: Second Cassandra users survey

2011-11-03 Thread Peter Tillotson
I'm using Cassandra as a big graph database, loading large volumes of data live and linking on the fly.  The number of edges grow geometrically with data added, and need to be read to continue linking the graph on the fly.  Consequently, my problem is constrained by:  * Predominantly read - esp

Debian package jna bug workaroung

2011-11-02 Thread Peter Tillotson
see below  * JAVA_HOME=/usr/lib/jvm/java-6-openjdk  works -- Reading the documentation over at Datastax “The Debian and RPM packages of Cassandra install JNA automatically” http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-improved-memory-and-disk-space-management And indeed the Debia

Re: Using Cassandra for transaction logging, good idea?

2011-07-31 Thread Peter Tillotson
You could try the following: i:20110728 { tx1="va1", tx2="va1", tx3="va1", tx4="va1", tx5="va1", tx6="va1", } The value could either be a blob / json pojo, or a reference off to another row storing the columns representing the value. Taking it further

Re: Java bits: Horizontal Scalibility and DAO layer design

2011-07-27 Thread Peter Tillotson
I had a quick look at DAO and my common uses case is, row names appearing as column ID's elsewhere (indexes etc). I also wanted to track deltas at a column level and have a reasonable sized client side cache. I ended up with two abastract DAO's, one for Column Family, one for Super column factory,

Re: Equalizing nodes storage load

2011-07-22 Thread Peter Tillotson
I'm not sure if this is the answer, but major compaction on each node for each column family. I suspect the data shuffle has left quite a few deleted keys which may get cleaned out on major compaction. As I remember major compaction doesn't automatically in 7.x, I'm not sure if it is triggered by r

Re: b-tree

2011-07-22 Thread Peter Tillotson
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I've not tried this but speculative implementation schema probably something like the following: Super Col family for structure hash(nodeId): { root: { left="nodeId1", right="nodeId2" } nodeId1: { left="nodeId3", right="nodeId4" }