Re: BulkLoading sstables from v1.0.3 to v1.1.1

2012-07-10 Thread rubbish me
Thanks Ivo. We are quite close to releasing so we'd hope to understand what causing the error and may try to avoid it where possible. As said, it seems to work ok the first time round. The problem you referring in the last mail, was it restricted to bulk loading or otherwise? Thanks -A Iv

Re: Serious issue updating Cassandra version and topology

2012-07-10 Thread aaron morton
To be clear, this happened on a 1.1.2 node and it happened again *after* you had run a scrub ? Has this cluster been around for a while or was the data created with 1.1 ? Can you confirm that all sstables were re-written for the CF? Check the timestamp on the files. Also also files should have

Re: Effect of rangequeries with RandomPartitioner

2012-07-10 Thread aaron morton
Index files map keys (not tokens) to offsets in the data file. A range scan uses the index file to seek to the start position in the data file and then does a partial scan of the data file. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 9

Re: Setting the Memtable allocator on a per CF basis

2012-07-10 Thread aaron morton
> Would you guys consider adding this option to a future release? All improvements are considered :) Please create a ticket on https://issues.apache.org/jira/browse/CASSANDRA and reference CASSANDRA-3073 > If you want I can try to create a patch myself and submit it to you? Sounds like a plan. Th

Re: Composite Slice Query returning non-sliced data

2012-07-10 Thread aaron morton
Ah, it's a Hector query question. You may have bette luck on the Hector email list. Or if you can turn on debug logging on the server and grab the query that would be handy. The first thing that stands out is that (in cassandra) comparison operations are not used in a slice range. Cheers

Re: Dynamic CF

2012-07-10 Thread Sylvain Lebresne
On Fri, Jul 6, 2012 at 10:49 PM, Leonid Ilyevsky wrote: > At this point I am really confused about what direction Cassandra is going. > CQL 3 has the benefit of composite keys, but no dynamic columns. > I thought, the whole point of Cassandra was to provide dynamic tables. CQL3 absolutely provid

Re: cannot build 1.1.2 from source

2012-07-10 Thread Sylvain Lebresne
I would check if you don't have a version of antlr install on you system that takes precedence over the one distributed with C* and happens to not be compatible. Because I don't remember there having been much change to the Cli between 1.1.1 and 1.1.2 and the grammar nobody has had that problem so

Trigger and customized filter

2012-07-10 Thread Felipe Schmidt
Does anyone know something about the following questions? 1. Does Cassandra support customized filter? customized filter means programmer can define his desired filter to select the data. 2. Does Cassandra support trigger? trigger has the same meaning as in RDBMS. Thanks in advance. Regards, Fel

RE: Dynamic CF

2012-07-10 Thread Leonid Ilyevsky
Thanks Sylvain, this is useful. So I guess, in the batch_mutate call, in the map that I pass to it, only the first element of the composite key should be used as a key (because it is the real key), and the other parts of the key should be passed as regular columns? Is this correct? While I am wa

Re: Dynamic CF

2012-07-10 Thread Carlos Carrasco
I think he means something like having a fixed set of coiumns in the table definition, then in the actual rows having other columns not specified in the defintion, indepentent of the composited part of the PK. When I reviewed CQL3 for using in Gossie[1] I realized I couldn't have this, and that it

Re: Dynamic CF

2012-07-10 Thread Sylvain Lebresne
On Tue, Jul 10, 2012 at 4:19 PM, Carlos Carrasco < carlos.carra...@groupalia.com> wrote: > I think he means something like having a fixed set of coiumns in the table > definition, then in the actual rows having other columns not specified in > the defintion, indepentent of the composited part of t

Re: Dynamic CF

2012-07-10 Thread Sylvain Lebresne
On Tue, Jul 10, 2012 at 4:17 PM, Leonid Ilyevsky wrote: > So I guess, in the batch_mutate call, in the map that I pass to it, only the > first element of the composite key should be used as a key (because it is the > real key), and the other parts of the key should be passed as regular > column

Re: Dynamic CF

2012-07-10 Thread Carlos Carrasco
I am confused then. I remember reviewing the source for CQL3 and finding that the row reader used the column count in the CF definition in order to find how many columns it needed to read a single row. I guess I missed a filter over the composited part or that I reviewed an old version. On 10 July

Re: Trigger and customized filter

2012-07-10 Thread Brian O'Neill
While Jonathan and crew work on the infrastructure to support triggers: https://issues.apache.org/jira/browse/CASSANDRA-4285 We have a project going over here that provides a trigger-like capability: https://github.com/hmsonline/cassandra-triggers/ https://github.com/hmsonline/cassandra-triggers/w

Re: Serious issue updating Cassandra version and topology

2012-07-10 Thread Michael Theroux
Hello Aaron, Thank you for responding. Since the time of my original email, we noticed that in the process of performing this upgrade that data was lost. We have restored from backup and are now trying this again with two changes: 1) We will be using 1.1.2 throughout the cluster 2) We have sw

RE: Dynamic CF

2012-07-10 Thread Leonid Ilyevsky
I see. I actually tried it, and it consistently throws an exception. Below is my test code. I have two tests; test1 is for the composite key case, and test2 is for the simple key. The test2 works fine, while test1 gives me: Exception in thread "main" InvalidRequestException(why:Not enough bytes

reading deleted rows is super-slow

2012-07-10 Thread Thorsten von Eicken
We're finding that reading deleted columns can be very slow and I'm trying to get confirmation for our analysis of what happens. We wrote lots of data eons ago into fairly large rows (up to 1MB). We recently read those rows and then deleted them. After this, we ran a verification-type pass that att

Re: Composite Slice Query returning non-sliced data

2012-07-10 Thread Tyler Hobbs
I think in this case that's just Hector's way of setting the EOC byte for a component. My guess is that the composite isn't being structured correctly through Hector, as well. On Tue, Jul 10, 2012 at 4:40 AM, aaron morton wrote: > > The first thing that stands out is that (in cassandra) comparis

Re: reading deleted rows is super-slow

2012-07-10 Thread Tyler Hobbs
This is expected due to tombstones, which this explains pretty well: http://wiki.apache.org/cassandra/DistributedDeletes If you don't have any tombstones for the row, the bloom filter will let Cassandra avoid doing any disk reads at all 99% of the time. On Tue, Jul 10, 2012 at 10:50 AM, Thorsten

what is the best data model for time series of small data chunks...

2012-07-10 Thread Roland Hänel
Hi, I have an application that consists of multiple (possible 1000's) of measurement series, and each measurement series generates a small amount of data output (only about 500 bytes) every 10 seconds. This time series of data should be stored in Cassandra in a fashion that both read access is pos

Re: what is the best data model for time series of small data chunks...

2012-07-10 Thread Tyler Hobbs
On Tue, Jul 10, 2012 at 12:14 PM, Roland Hänel wrote: > Hi, > > I have an application that consists of multiple (possible 1000's) of > measurement series, and each measurement series generates a small amount of > data output (only about 500 bytes) every 10 seconds. This time series of > data shou

Using a node in separate cluster without decommissioning.

2012-07-10 Thread rohit bhatia
Hi I want to take out 2 nodes from a 8 node cluster and use in another cluster, but can't afford the overhead of streaming the data and rebalance cluster. Since replication factor is 2 in first cluster, I won't lose any data. I'm planning to save my commit_log and data directories and bootstrappi

RE: Dynamic CF

2012-07-10 Thread Leonid Ilyevsky
I see now there is a package org.apache.cassandra.cql3.statements, with BatchStatement class. Is this what I should use? -Original Message- From: Leonid Ilyevsky [mailto:lilyev...@mooncapital.com] Sent: Tuesday, July 10, 2012 11:45 AM To: user@cassandra.apache.org Subject: RE: Dynamic CF

Re: Composite Slice Query returning non-sliced data

2012-07-10 Thread Sunit Randhawa
I have tested this extensively and EOC has huge issue in terms of usability of CompositeTypes in Cassandra. As an example: If you have 2 Composite Columns such as A:B:C and A:D:C. And if you do search on A:B as start and end Composite Components, it will return D as well. Because it returns all t

Re: Composite Slice Query returning non-sliced data

2012-07-10 Thread Tyler Hobbs
On Tue, Jul 10, 2012 at 2:20 PM, Sunit Randhawa wrote: > I have tested this extensively and EOC has huge issue in terms of > usability of CompositeTypes in Cassandra. > > As an example: If you have 2 Composite Columns such as A:B:C and A:D:C. > > And if you do search on A:B as start and end Compos

help using org.apache.cassandra.cql3

2012-07-10 Thread Leonid Ilyevsky
I am trying to use the org.apache.cassandra.cql3 package. Having problem connecting to the server using ClientState. I was not sure what to put in the credentials map (I did not set any users/passwords on my server), so I tried setting empty strings for "username" and "password", setting them to

failed to delete commitlog, cassandra can't accept writes

2012-07-10 Thread Frank Hsueh
after reading the JIRA, I decided to use Java 6. with Casandra 1.1.2 on Java 6 x64 on Win7 sp1 x64 (all latest versions), after a several minutes of sustained writes, I see: from system.log: java.io.IOError: java.io.IOException: Failed to delete C:\var\lib\cassandra\commitlog\CommitLog-94869

Re: failed to delete commitlog, cassandra can't accept writes

2012-07-10 Thread Frank Hsueh
oops; I missed log line: ERROR [COMMIT-LOG-ALLOCATOR] 2012-07-10 14:19:39,776 AbstractCassandraDaemon.java (line 134) Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main] java.io.IOError: java.io.IOException: Failed to delete C:\var\lib\cassandra\commitlog\CommitLog-948695923996466.log at

Re: help using org.apache.cassandra.cql3

2012-07-10 Thread Derek Williams
On Tue, Jul 10, 2012 at 3:04 PM, Leonid Ilyevsky wrote: > I am trying to use the org.apache.cassandra.cql3 package. Having problem > connecting to the server using ClientState. > > I was not sure what to put in the credentials map (I did not set any > users/passwords on my server), so I tried

Re: Multiple keyspace question

2012-07-10 Thread Edward Capriolo
A problem of many keyspaces is clients are bound to a keyspace so connection pooling multiple keyspaces is an issue. Cql has support for some limited cross keyspace operations. On Sunday, July 8, 2012, aaron morton wrote: > I would do a test to see the latency difference under load between having

Cassandra take 100% CPU for 2~3 minutes every half an hour and mutation lost

2012-07-10 Thread Jason Tang
Hi I encounter the High CPU problem, Cassandra 1.0.3, happened on both sized and leveled compaction, 6G heap, 64bit Oracle java. For normal traffic, Cassandra will use 15% CPU. But every half a hour, Cassandra will use almost 100% total cpu (SUSE, 12 Core). And here is the top inform