Composite Column Question

2011-12-20 Thread Martin Arrowsmith
Dear Cassandra Experts, Are the number of composite attributes fixed for each column family ? I have been doing : "create column family MyCF with comparator = 'CompositeType(IntegerType, UTF8Type)' And this creates a composite { integer:string } Hector complains when I give a 3rd attribute. If

Can I slice on composite indexes?

2011-12-20 Thread Maxim Potekhin
Let's say I have rows with composite columns Like ("key1", {('xyz', 'abc'): 'colval1'}, {('xyz', 'def'): 'colval2'}) ("key2", {('ble', 'meh'): 'otherval'}) Is it possible to create a composite type index such that I can query on 'xyz' and get the first two columns? Thanks Maxim

Routine nodetool repair

2011-12-20 Thread Blake Starkenburg
I have been playing around with Cassandra for a few months now. Starting to explore more of the routine maintenance and backup strategies and I have a general question about nodetool repair. After reading the following page: http://www.datastax.com/docs/0.8/operations/cluster_management it has occu

Re: about decommissioned node that returns

2011-12-20 Thread aaron morton
A node will only request data from other nodes on start up if it is bootstrapping. Bootstrapping happens when there is no system data and the auto_bootstrap: yaml config is set true, normally nodes only bootstrap once in their life. If you bring a node back for whatever reason you will want to

Re: Doubts related to composite type column names/values

2011-12-20 Thread aaron morton
+1 use them if you can. Also you can reverse the sort order on components in the type, that can make some common queries faster. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 21/12/2011, at 9:49 AM, Guy Incognito wrote: > afaik composi

Counter read requests spread across replicas ?

2011-12-20 Thread Philippe
Hello, 5 nodes running 0.8.7/0.8.9, RF=3, BOP, counter columns inside super columns. Read queries are multigetslices of super columns inside of which I read every column for processing (20-30 at most), using Hector with default settings. Watching tpstat on the 3 nodes holding the data being most of

Re: setStrategy_options syntax in thrift

2011-12-20 Thread A J
Thanks, that worked. On Tue, Dec 20, 2011 at 4:08 PM, Dave Brosius wrote: > > KsDef ksDef = new KsDef(); > Map options = new HashMap(); > options.put("replication_factor", "2"); > ksDef.setStrategy_options(options); > > > > *- Original Message -* > *From:* "A J" > *Sent:* Tue, December 2

Re: java thrift error

2011-12-20 Thread A J
The following worked: import java.nio.ByteBuffer; import java.nio.CharBuffer; import java.nio.charset.Charset; import java.nio.charset.CharacterCodingException; import java.nio.charset.CharsetDecoder; import java.nio.charset.CharsetEncoder; import java.nio.charset.CodingErrorAction; . public st

Re: about decommissioned node that returns

2011-12-20 Thread Carlos Pérez Miguel
What happens with the data stored for that token range already in the cluster? will the restarted node actualize it from any of the replicas? I readed in the maillist archives that in case of token collision, the cluster may not work correctly. But if the restarted node takes always ownership of i

Re: setStrategy_options syntax in thrift

2011-12-20 Thread Dave Brosius
KsDef ksDef = new KsDef();Map>;String, String;String, String

Re: java thrift error

2011-12-20 Thread Dave Brosius
A ByteBuffer is not a byte[] to convert a String to a ByteBuffer do something likepublic static ByteBuffer toByteBuffer(String value) throws UnsupportedEncodingException { return ByteBuffer.wrap(value.getBytes("UTF-8")); } see http://wiki.apache.org/cassandra/ThriftExamples --

Re: setStrategy_options syntax in thrift

2011-12-20 Thread A J
I am new to java. Can you specify the exact syntax for replication_factor=2 ? Thanks. On Tue, Dec 20, 2011 at 1:50 PM, aaron morton wrote: > It looks like you tried to pass the string "{replication_factor:2}" > > You need to pas a Map type , where the the key is the option > and the value is the

Re: questions on datastax opscenter

2011-12-20 Thread Tyler Hobbs
On Tue, Dec 20, 2011 at 2:19 PM, Tyler Hobbs wrote: > On Tue, Dec 20, 2011 at 1:49 PM, Feng Qu wrote: > >> I have two questions for community version of opscenter >> >> 1) does it work with multiple cassandra cluster? >> 2) are agent ports 61620/61621 configurable? these ports are blocked in >>

java thrift error

2011-12-20 Thread A J
The following syntax : import org.apache.cassandra.thrift.*; . . ColumnOrSuperColumn col = client.get("count_key".getBytes("UTF-8"), cp, ConsistencyLevel.QUORUM); is giving the error: get(java.nio.ByteBuffer,org.apache.cassandra.thrift.ColumnPath,org.apache.cassandra.thrift.ConsistencyLev

Re: Doubts related to composite type column names/values

2011-12-20 Thread Guy Incognito
afaik composite lets you do sorting in a way that would be difficult/impossible with string concatenation. eg with the string ascending, and the integer descending. if i had composites available (which i don't b/c we are on 0.7), i would use them over string concatenation. string concatenati

Re: Doubts related to composite type column names/values

2011-12-20 Thread Maxim Potekhin
Thank you Aaron! As long as I have plain strings, would you say that I would do almost as well with catenation? Of course I realize that mixed types are a very different case where the composite is very useful. Thanks Maxim On 12/20/2011 2:44 PM, aaron morton wrote: Component values are co

Re: about decommissioned node that returns

2011-12-20 Thread Brandon Williams
On Tue, Dec 20, 2011 at 2:24 PM, aaron morton wrote: > Sorry, got that a little wrong. > > At startup the node will use the higher of the current seconds since epoch > or the stored generation number. Technically stored generation + 1 so it's always increased on a restart. -Brandon

Re: about decommissioned node that returns

2011-12-20 Thread aaron morton
Sorry, got that a little wrong. At startup the node will use the higher of the current seconds since epoch or the stored generation number. So if you restart the old node it should always take ownership of it's token. Cheers - Aaron Morton Freelance Developer @aaronmorton ht

Re: questions on datastax opscenter

2011-12-20 Thread Tyler Hobbs
On Tue, Dec 20, 2011 at 1:49 PM, Feng Qu wrote: > I have two questions for community version of opscenter > > 1) does it work with multiple cassandra cluster? > 2) are agent ports 61620/61621 configurable? these ports are blocked in > our env. > > Thanks in advance > > Feng Qu > They are configu

Re: Choosing a Partitioner Type for Random java.util.UUID Row Keys

2011-12-20 Thread Bryce Allen
I wasn't aware of CompositeColumns, thanks for the tip. However I think it still doesn't allow me to do the query I need - basically I need to do a timestamp range query, limiting only to certain file names at each timestamp. With BOP and a separate row for each timestamp, prefixed by a random UUID

questions on datastax opscenter

2011-12-20 Thread Feng Qu
I have two questions for community version of opscenter 1) does it work with multiple cassandra cluster? 2) are agent ports 61620/61621 configurable? these ports are blocked in our env.  Thanks in advance   Feng Qu

Re: Doubts related to composite type column names/values

2011-12-20 Thread aaron morton
Component values are compared in a type aware fashion, an Integer is an Integer. Not a 10 character zero padded string. You can also slice on the components. Just like with string concat, but nicer. . e.g. If you app is storing comments for a thing, and the column names have the form or yo

Re: about decommissioned node that returns

2011-12-20 Thread aaron morton
> - What happens in Cassandra with a failed node that has been > decommissioned if it returns to the cluster with old data and its old > token? If it starts normally it *may* take ownership of the token away from the previous node. The Gossip Generation is compared and the server with the higher

Re: memory estimate for each key in the key cache

2011-12-20 Thread aaron morton
If you get the sizing wrong there area couple of emergency pressure valves in the config… https://github.com/apache/cassandra/blob/trunk/conf/cassandra.yaml#L113 Watch for log messages with "Cassandra is now reducing cache sizes to free up memory." in them. Cheers - Aaron Mort

Re: Choosing a Partitioner Type for Random java.util.UUID Row Keys

2011-12-20 Thread aaron morton
Bryce, Have you considered using CompositeColumns and a standard CF? Row key is the UUID column name is (timestamp : dir_entry) you can then slice all columns with a particular time stamp. Even if you have a random key, I would use the RP unless you have an extreme use case.

Re: Counters != Counts

2011-12-20 Thread aaron morton
Yes you can do a rolling restart. For more info on the counters architecture see the presentation here http://www.datastax.com/events/cassandrasf2011/presentations Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 20/12/2011, at 10:10 PM, Al

Re: setStrategy_options syntax in thrift

2011-12-20 Thread aaron morton
It looks like you tried to pass the string "{replication_factor:2}" You need to pas a Map type , where the the key is the option and the value is the option value. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 20/12/2011, at 12:02 PM,

Re: Doubts related to composite type column names/values

2011-12-20 Thread Maxim Potekhin
With regards to static, what are major benefits as it compares with string catenation (with some convenient separator inserted)? Thanks Maxim On 12/20/2011 1:39 PM, Richard Low wrote: On Tue, Dec 20, 2011 at 5:28 PM, Ertio Lew wrote: With regard to the composite columns stuff in Cassandra,

Re: Doubts related to composite type column names/values

2011-12-20 Thread Richard Low
On Tue, Dec 20, 2011 at 5:28 PM, Ertio Lew wrote: > With regard to the composite columns stuff in Cassandra, I have the > following doubts : > > 1. What is the storage overhead of the composite type column names/values, The values are the same. For each dimension, there is 3 bytes overhead. > 2

Doubts related to composite type column names/values

2011-12-20 Thread Ertio Lew
With regard to the composite columns stuff in Cassandra, I have the following doubts : 1. What is the storage overhead of the composite type column names/values, and 2. what exactly is the difference between the DynamicComposite and Static Composite ?

about decommissioned node that returns

2011-12-20 Thread Carlos Pérez Miguel
Hello, I have several questions about decommissioned nodes that I would like to clarify: - What happens in Cassandra with a failed node that has been decommissioned if it returns to the cluster with old data and its old token? - Will it actualize the data using ReadRepair or the bootstrapping pro

Re: memory estimate for each key in the key cache

2011-12-20 Thread Kent Tong
> It is not telling you to multiply your key size by 10-12, it is telling you to > multiply the output of the nodetool cfstats reported "key cache size" by > 10-12. The  "key cache size" reported is actually the number of keys in the key cache. So, it is the same thing as suggesting each key ta

Re: Choosing a Partitioner Type for Random java.util.UUID Row Keys

2011-12-20 Thread Bryce Allen
I think it comes down to how much you benefit from row range scans, and how confident you are that going forward all data will continue to use random row keys. I'm considering using BOP as a way of working around the non indexes super column limitation. In my current schema, row keys are random UU

Re: Choosing a Partitioner Type for Random java.util.UUID Row Keys

2011-12-20 Thread Filipe Gonçalves
Generally, RandomPartitioner is the recommended one. If you already provide randomized keys it doesn't make much of a difference, the nodes should be balanced with any partitioner. However, unless you have UUID in all keys of all column families (highly unlikely) ByteOrderedPartitioner and OrderPre

RE: Drop schema Called "Index"

2011-12-20 Thread Michael Vaknine
Any way I was able to delete the keyspace using python pycassa Thanks Michael From: aaron morton [mailto:aa...@thelastpickle.com] Sent: Monday, December 19, 2011 9:39 PM To: user@cassandra.apache.org Subject: Re: Drop schema Called "Index" Which keyspace are you trying to delete ?

Re: Counters != Counts

2011-12-20 Thread Alain RODRIGUEZ
By the way, I wonder if a rolling restart is still possible when I am using counters ? Don't we lose the SPOF too, considering that counters are wrong every time we retry to write them with the client ? 2011/12/16 Alain RODRIGUEZ > Can we have a hope that counters will be replayed as safely as