Re: Cassandra performance decreases drastically with increase in data size.

2013-05-30 Thread Aiman Parvaiz
I believe you should roll out more nodes as a temporary fix to your problem, 400GB on all nodes means (as correctly mentioned in other mails of this thread) you are spending more time on GC. Check out the second comment in this link by Aaron Morton, he says the more than 300GB can be problematic

Re: Getting error "Too many in flight hints"

2013-05-30 Thread Theo Hultberg
thanks a lot for the explanation. if I understand it correctly it basically back pressure from C*, it's telling me that it's overloaded and that I need to back off. I better start a few more nodes, I guess. T# On Thu, May 30, 2013 at 10:47 PM, Robert Coli wrote: > On Thu, May 30, 2013 at 8:24

java.lang.AssertionError on starting the node

2013-05-30 Thread himanshu.joshi
Hi, I have created a 2 node test cluster in Cassandra version 1.2.3 with Simple Strategy and Replication Factor 2. The Java version is "1.6.0_27" The seed node is working fine but when I am starting the second node it is showing the following error: ERROR 10:16:55,603 Exception in thread

Re: Cassandra performance decreases drastically with increase in data size.

2013-05-30 Thread Bryan Talbot
One or more of these might be effective depending on your particular usage - remove data (rows especially) - add nodes - add ram (has limitations) - reduce bloom filter space used by increasing fp chance - reduce row and key cache sizes - increase index sample ratio - reduce compaction concurrency

Re: Interview request on SaaS NoSQL databases

2013-05-30 Thread julien Campan
Hi Chistophe, I noticed your email just now. Do you still need some feedback for your thesis on NoSQL? Cheers, Julien 2013/4/8 Christophe Caron > Hi all, > > I'm currently preparing my master's thesis in IT sciences at Itescia > school and UPMC university in France. This thesis focuses on NoS

Re: Bulk loading into CQL3 Composite Columns

2013-05-30 Thread Keith Wright
StringSerializer and CompositeSerializer are actually from Astyanax for what's it worth. I would recommend you change your table definition so that only val1 is part of the primary key. There is no reason to include val2. Perhaps sending the IndexOutOfBoundsException would help. All the Stri

Re: 1.2 tuning

2013-05-30 Thread Robert Coli
On Wed, May 29, 2013 at 2:38 PM, Darren Smythe wrote: > Were using the latest JNA and separate ephemeral drives for commit log and > data directories. (as a note..) Per nickmbailey, testing shows that there is little/no benefit to separating commit log and data dirs on virtualized disk (or SSD),

Re: Cassandra performance decreases drastically with increase in data size.

2013-05-30 Thread srmore
You are right, it looks like I am doing a lot of GC. Is there any short-term solution for this other than bumping up the heap ? because, even if I increase the heap I will run into the same issue. Only the time before I hit OOM will be lengthened. It will be while before we go to latest and greate

Re: Getting error "Too many in flight hints"

2013-05-30 Thread Robert Coli
On Thu, May 30, 2013 at 8:24 AM, Theo Hultberg wrote: > I'm using Cassandra 1.2.4 on EC2 (3 x m1.large, this is a test cluster), and > my application is talking to it over the binary protocol (I'm using JRuby > and the cql-rb driver). I get this error quite frequently: "Too many in > flight hints:

Re: Bulk loading into CQL3 Composite Columns

2013-05-30 Thread Daniel Morton
Hi Edward... Thanks for the pointer. I will use that going forward. Daniel Morton On Thu, May 30, 2013 at 4:09 PM, Edward Capriolo wrote: > You should probably be using system.nanoTime() not > system.currentTimeInMillis(). The user is free to set the timestamp to > whatever they like but nano-

Re: Bulk loading into CQL3 Composite Columns

2013-05-30 Thread Daniel Morton
Hi Keith... Thanks for the help. I'm presently not importing the Hector library (Which is where classes like CompositeSerializer and StringSerializer come from, yes?), only the cassandra-all maven artifact. Is the behaviour of the CompositeSerializer much different than using a Builder from a Com

Re: Bulk loading into CQL3 Composite Columns

2013-05-30 Thread Edward Capriolo
You should probably be using system.nanoTime() not system.currentTimeInMillis(). The user is free to set the timestamp to whatever they like but nano-time is the standard (it is what the cli uses, and what cql will use) On Thu, May 30, 2013 at 3:33 PM, Keith Wright wrote: > Sorry, typo in code

Re: Bulk loading into CQL3 Composite Columns

2013-05-30 Thread Keith Wright
Sorry, typo in code sample, should be: ssTableWriter.newRow(StringSerializer.get().toByteBuffer("20101201")); Composite columnComposite = new Composite(); columnComposite.setComponent(0,5,IntegerSerializer.get()); columnComposite.setComponent(1,10,IntegerSerializer.get()); ssTableWriter.addColumn(

Re: Bulk loading into CQL3 Composite Columns

2013-05-30 Thread Keith Wright
You do not want to repeat the first item of your primary key again. If you recall, in CQL3 a primary key as defined below indicates that the row key is the first item (key) and then the column names are composites of val1,val2. Although I don't see why you need val2 as part of the primary key

Re: slice query

2013-05-30 Thread Edward Capriolo
In thrift an empty ByteBuffer (in some cases an empty string "") can mean both start and end Thus: start: "", end "" is the entire slice start: "c", end "" start at c inclusive rest of slice On Thu, May 30, 2013 at 2:37 PM, Kanwar Sangha wrote: > Hi – We gave a dynamic CF which has a key and

slice query

2013-05-30 Thread Kanwar Sangha
Hi - We gave a dynamic CF which has a key and multiple columns which get added dynamically. For example - Key_1 , Column1, Column2, Column3,... Key_2 , Column1, Column2, Column3,. Now I want to get all columns after Column3...how do we query that ? The ColumnSliceIterator in hector al

Cassandra 1.1.11 does not always show filename of corrupted files

2013-05-30 Thread horschi
Hi, we had some hard-disk issues this week, which caused some datafiles to get corrupt, which was reported by the compaction. My approach to fix this was to delete the corrupted files and run repair. That sounded easy at first, but unfortunetaly C* 1.1.11 sometimes does not show which datafile is

Re: Cassandra on a single (under-powered) instance?

2013-05-30 Thread Daniel Morton
Hi Tyler... Thank you very much for the response. It is nice to know that there is some possibility this might work. :) Regards, Daniel Morton On Wed, May 29, 2013 at 2:03 PM, Tyler Hobbs wrote: > You can get away with a 1 to 2GB heap if you don't put too much pressure > on it. I commonly

Bulk loading into CQL3 Composite Columns

2013-05-30 Thread Daniel Morton
Hi All. I am trying to bulk load some data into a CQL3 table using the sstableloader utility and I am having some difficulty figuring out how to use the SSTableSimpleUnsortedWriter with composite columns. I have created this simple contrived table for testing: create table test (key varchar, val

Getting error "Too many in flight hints"

2013-05-30 Thread Theo Hultberg
Hi, I'm using Cassandra 1.2.4 on EC2 (3 x m1.large, this is a test cluster), and my application is talking to it over the binary protocol (I'm using JRuby and the cql-rb driver). I get this error quite frequently: "Too many in flight hints: 2411" (the exact number varies) Has anyone any idea of w

Re: slow read for cassandra time series

2013-05-30 Thread Sylvain Lebresne
> I have a slow query that is making me think I don't understand the data model for > time series: > select asset, returns from marketData where date >= 20130101 and date <= 20130110 > allow filtering; > > create table marketData { > asset varchar, > returns double, > date timestamp, > PRIMARY KEY(