Re: Using composite column names in the CLI

[David Boxenhorn]

This is what I'm talking about

https://issues.apache.org/jira/browse/CASSANDRA-2231

The on-disk format is

<(short)length><(short)length>...

I would like to be able to input these kinds of keys into the CLI, something
like

set cf[key]['constituent1':'constituent2':'constituent3'] = val


On Tue, May 17, 2011 at 2:15 AM, Sameer Farooqui wrote:

> Cassandra wouldn't know that the column name is composite of two different
> things. So you could just request the column names and values for a specific
> key like this and then just look at the column names that get returned:
>
> [default@MyKeyspace] get DemoCF[ascii('key_42')];
> => (column=CA_SanJose, value=50, timestamp=1305236885112000)
> => (column=CA_PaloAlto, value=49, timestamp=1305236885192000)
> => (column=FL_Orlando, value=45, timestamp=130523688528)
> => (column=NY_NYC, value=40, timestamp=1305236885361000)
>
>
> And I'm not sure what you mean by inputting composite column names. You
> just input them like any other column name:
>
> [default@MyKeyspace] set DemoCF['key_42']['CA_SanJose']='51';
> Value inserted.
>
>
>
>
>
> On Mon, May 16, 2011 at 2:34 PM, Aaron Morton wrote:
>
>> What do you mean by composite column names?
>>
>> Do the data type functions supported by get and set help? Or the assume
>> statement?
>>
>> Aaron
>> On 17/05/2011, at 3:21 AM, David Boxenhorn  wrote:
>>
>> > Is there a way to view composite column names in the CLI?
>> >
>> > Is there a way to input them (i.e. in the set command)?
>> >
>>
>
>

Re: Using composite column names in the CLI

[Sylvain Lebresne]

Provided you're working on a branch that has CASSANDRA-2231 applied (that's
either the cassandra-0.8.1 branch or trunk), this work 'out of the box':

The setup will look like:
[default@unknown] create keyspace test;
[default@unknown] use test;
[default@test] create column family testCF with
comparator='CompositeType(AsciiType, IntegerType(reversed=true),
IntegerType)' and default_validation_class=AsciiType;

Then:
[default@test] set testCF[a]['foo:24:24'] = 'v1';
Value inserted.
[default@test] set testCF[a]['foo:42:24'] = 'v2';
Value inserted.
[default@test] set testCF[a]['foobar:42:24'] = 'v3';
Value inserted.
[default@test] set testCF[a]['boobar:42:24'] = 'v4';
Value inserted.
[default@test] set testCF[a]['boobar:42:42'] = 'v5';
Value inserted.
[default@test] get testCF[a];
=> (column=boobar:42:24, value=v4, timestamp=1305621115813000)
=> (column=boobar:42:42, value=v5, timestamp=1305621125563000)
=> (column=foo:42:24, value=v2, timestamp=1305621096473000)
=> (column=foo:24:24, value=v1, timestamp=1305621085548000)
=> (column=foobar:42:24, value=v3, timestamp=1305621110813000)
Returned 5 results.

--
Sylvain

On Tue, May 17, 2011 at 9:20 AM, David Boxenhorn  wrote:
> This is what I'm talking about
>
> https://issues.apache.org/jira/browse/CASSANDRA-2231
>
> The on-disk format is
>
> <(short)length><(short)length> byte = 0>...
>
> I would like to be able to input these kinds of keys into the CLI, something
> like
>
> set cf[key]['constituent1':'constituent2':'constituent3'] = val
>
>
> On Tue, May 17, 2011 at 2:15 AM, Sameer Farooqui 
> wrote:
>>
>> Cassandra wouldn't know that the column name is composite of two different
>> things. So you could just request the column names and values for a specific
>> key like this and then just look at the column names that get returned:
>> [default@MyKeyspace] get DemoCF[ascii('key_42')];
>> => (column=CA_SanJose, value=50, timestamp=1305236885112000)
>> => (column=CA_PaloAlto, value=49, timestamp=1305236885192000)
>> => (column=FL_Orlando, value=45, timestamp=130523688528)
>> => (column=NY_NYC, value=40, timestamp=1305236885361000)
>>
>> And I'm not sure what you mean by inputting composite column names. You
>> just input them like any other column name:
>> [default@MyKeyspace] set DemoCF['key_42']['CA_SanJose']='51';
>> Value inserted.
>>
>>
>>
>>
>> On Mon, May 16, 2011 at 2:34 PM, Aaron Morton 
>> wrote:
>>>
>>> What do you mean by composite column names?
>>>
>>> Do the data type functions supported by get and set help? Or the assume
>>> statement?
>>>
>>> Aaron
>>> On 17/05/2011, at 3:21 AM, David Boxenhorn  wrote:
>>>
>>> > Is there a way to view composite column names in the CLI?
>>> >
>>> > Is there a way to input them (i.e. in the set command)?
>>> >
>>
>
>

Re: Using composite column names in the CLI

[David Boxenhorn]

Excellent!

(I presume there is some way of representing ":", like "\:"?)


On Tue, May 17, 2011 at 11:44 AM, Sylvain Lebresne wrote:

> Provided you're working on a branch that has CASSANDRA-2231 applied (that's
> either the cassandra-0.8.1 branch or trunk), this work 'out of the box':
>
> The setup will look like:
> [default@unknown] create keyspace test;
> [default@unknown] use test;
> [default@test] create column family testCF with
> comparator='CompositeType(AsciiType, IntegerType(reversed=true),
> IntegerType)' and default_validation_class=AsciiType;
>
> Then:
> [default@test] set testCF[a]['foo:24:24'] = 'v1';
> Value inserted.
> [default@test] set testCF[a]['foo:42:24'] = 'v2';
> Value inserted.
> [default@test] set testCF[a]['foobar:42:24'] = 'v3';
> Value inserted.
> [default@test] set testCF[a]['boobar:42:24'] = 'v4';
> Value inserted.
> [default@test] set testCF[a]['boobar:42:42'] = 'v5';
> Value inserted.
> [default@test] get testCF[a];
> => (column=boobar:42:24, value=v4, timestamp=1305621115813000)
> => (column=boobar:42:42, value=v5, timestamp=1305621125563000)
> => (column=foo:42:24, value=v2, timestamp=1305621096473000)
> => (column=foo:24:24, value=v1, timestamp=1305621085548000)
> => (column=foobar:42:24, value=v3, timestamp=1305621110813000)
> Returned 5 results.
>
> --
> Sylvain
>
> On Tue, May 17, 2011 at 9:20 AM, David Boxenhorn 
> wrote:
> > This is what I'm talking about
> >
> > https://issues.apache.org/jira/browse/CASSANDRA-2231
> >
> > The on-disk format is
> >
> > <(short)length> 0><(short)length> > byte = 0>...
> >
> > I would like to be able to input these kinds of keys into the CLI,
> something
> > like
> >
> > set cf[key]['constituent1':'constituent2':'constituent3'] = val
> >
> >
> > On Tue, May 17, 2011 at 2:15 AM, Sameer Farooqui <
> cassandral...@gmail.com>
> > wrote:
> >>
> >> Cassandra wouldn't know that the column name is composite of two
> different
> >> things. So you could just request the column names and values for a
> specific
> >> key like this and then just look at the column names that get returned:
> >> [default@MyKeyspace] get DemoCF[ascii('key_42')];
> >> => (column=CA_SanJose, value=50, timestamp=1305236885112000)
> >> => (column=CA_PaloAlto, value=49, timestamp=1305236885192000)
> >> => (column=FL_Orlando, value=45, timestamp=130523688528)
> >> => (column=NY_NYC, value=40, timestamp=1305236885361000)
> >>
> >> And I'm not sure what you mean by inputting composite column names. You
> >> just input them like any other column name:
> >> [default@MyKeyspace] set DemoCF['key_42']['CA_SanJose']='51';
> >> Value inserted.
> >>
> >>
> >>
> >>
> >> On Mon, May 16, 2011 at 2:34 PM, Aaron Morton 
> >> wrote:
> >>>
> >>> What do you mean by composite column names?
> >>>
> >>> Do the data type functions supported by get and set help? Or the assume
> >>> statement?
> >>>
> >>> Aaron
> >>> On 17/05/2011, at 3:21 AM, David Boxenhorn  wrote:
> >>>
> >>> > Is there a way to view composite column names in the CLI?
> >>> >
> >>> > Is there a way to input them (i.e. in the set command)?
> >>> >
> >>
> >
> >
>

Dynamic Snitch Problem

[Daniel Doubleday]

Hi all

after upgrading to 0.7 we have a small problem with dynamic snitch:

we have rf=3, quorum read/write and read repair prop set to 0. Thus cassandra 
always shortcuts reads to only 2 hosts.

Problem is that one of our nodes get ignored unless using a little patch and 
initialize the scores.

Anybody else sees this?

Cheers,
Daniel

btw: we had to do this on 0.6 too when using the 'shortcut' read path.

Index: src/java/org/apache/cassandra/locator/DynamicEndpointSnitch.java
===
--- src/java/org/apache/cassandra/locator/DynamicEndpointSnitch.java
(revision 1100429)
+++ src/java/org/apache/cassandra/locator/DynamicEndpointSnitch.java
(revision )
@@ -178,8 +178,16 @@
 {
 Double scored1 = scores.get(a1);
 Double scored2 = scores.get(a2);
-
+
-if (scored1 == null || scored2 == null || scored1.equals(scored2))
+if (scored1 == null) {
+scored1 = 0.0;
+}
+
+if (scored2 == null) {
+scored2 = 0.0;
+}
+
+if (scored1.equals(scored2))
 return subsnitch.compareEndpoints(target, a1, a2);
 if (scored1 < scored2)
 return -1;

Re: Using composite column names in the CLI

[Sylvain Lebresne]

> (I presume there is some way of representing ":", like "\:"?)

Well no, not yet, but we'll try to figure something I guess (we'll have the
problem with CASSANDRA-2474 I think so we'll probably use the same
solution).

But let's keep in mind this is unreleased code at this point. And let me
also add that it's just a cli limitation (actually of the
AbstractCompositeType.fromString() method).

--
Sylvain

> On Tue, May 17, 2011 at 11:44 AM, Sylvain Lebresne 
> wrote:
>>
>> Provided you're working on a branch that has CASSANDRA-2231 applied
>> (that's
>> either the cassandra-0.8.1 branch or trunk), this work 'out of the box':
>>
>> The setup will look like:
>> [default@unknown] create keyspace test;
>> [default@unknown] use test;
>> [default@test] create column family testCF with
>> comparator='CompositeType(AsciiType, IntegerType(reversed=true),
>> IntegerType)' and default_validation_class=AsciiType;
>>
>> Then:
>> [default@test] set testCF[a]['foo:24:24'] = 'v1';
>> Value inserted.
>> [default@test] set testCF[a]['foo:42:24'] = 'v2';
>> Value inserted.
>> [default@test] set testCF[a]['foobar:42:24'] = 'v3';
>> Value inserted.
>> [default@test] set testCF[a]['boobar:42:24'] = 'v4';
>> Value inserted.
>> [default@test] set testCF[a]['boobar:42:42'] = 'v5';
>> Value inserted.
>> [default@test] get testCF[a];
>> => (column=boobar:42:24, value=v4, timestamp=1305621115813000)
>> => (column=boobar:42:42, value=v5, timestamp=1305621125563000)
>> => (column=foo:42:24, value=v2, timestamp=1305621096473000)
>> => (column=foo:24:24, value=v1, timestamp=1305621085548000)
>> => (column=foobar:42:24, value=v3, timestamp=1305621110813000)
>> Returned 5 results.
>>
>> --
>> Sylvain
>>
>> On Tue, May 17, 2011 at 9:20 AM, David Boxenhorn 
>> wrote:
>> > This is what I'm talking about
>> >
>> > https://issues.apache.org/jira/browse/CASSANDRA-2231
>> >
>> > The on-disk format is
>> >
>> > <(short)length>> > 0><(short)length>> > byte = 0>...
>> >
>> > I would like to be able to input these kinds of keys into the CLI,
>> > something
>> > like
>> >
>> > set cf[key]['constituent1':'constituent2':'constituent3'] = val
>> >
>> >
>> > On Tue, May 17, 2011 at 2:15 AM, Sameer Farooqui
>> > 
>> > wrote:
>> >>
>> >> Cassandra wouldn't know that the column name is composite of two
>> >> different
>> >> things. So you could just request the column names and values for a
>> >> specific
>> >> key like this and then just look at the column names that get returned:
>> >> [default@MyKeyspace] get DemoCF[ascii('key_42')];
>> >> => (column=CA_SanJose, value=50, timestamp=1305236885112000)
>> >> => (column=CA_PaloAlto, value=49, timestamp=1305236885192000)
>> >> => (column=FL_Orlando, value=45, timestamp=130523688528)
>> >> => (column=NY_NYC, value=40, timestamp=1305236885361000)
>> >>
>> >> And I'm not sure what you mean by inputting composite column names. You
>> >> just input them like any other column name:
>> >> [default@MyKeyspace] set DemoCF['key_42']['CA_SanJose']='51';
>> >> Value inserted.
>> >>
>> >>
>> >>
>> >>
>> >> On Mon, May 16, 2011 at 2:34 PM, Aaron Morton 
>> >> wrote:
>> >>>
>> >>> What do you mean by composite column names?
>> >>>
>> >>> Do the data type functions supported by get and set help? Or the
>> >>> assume
>> >>> statement?
>> >>>
>> >>> Aaron
>> >>> On 17/05/2011, at 3:21 AM, David Boxenhorn  wrote:
>> >>>
>> >>> > Is there a way to view composite column names in the CLI?
>> >>> >
>> >>> > Is there a way to input them (i.e. in the set command)?
>> >>> >
>> >>
>> >
>> >
>
>

RE: AssertionError

[Desimpel, Ignace]

I use a custom comparator class. So I think there is a high chance that I do 
something wrong there. I was thinking that the stack trace could give a clue 
and help me on the way, maybe because some already got the same error.

 

Anyway, here is some more information you requested.

 

Yaml definition :

name: ForwardStringValues

  column_type: Super

  compare_with: 
be.landc.services.search.server.db.cassandra.node.ForwardCFStringValues

  compare_subcolumns_with: BytesType

  keys_cached: 10

  rows_cached: 0

  comment: Stores the values of functions returning string

  memtable_throughput_in_mb: 64

  memtable_operations_in_millions: 15

  min_compaction_threshold: 2

  max_compaction_threshold: 5

 

Column Family: ForwardStringValues

SSTable count: 8

Space used (live): 131311776690

Space used (total): 131311776690

Memtable Columns Count: 0

Memtable Data Size: 0

Memtable Switch Count: 0

Read Count: 1

Read Latency: 404.890 ms.

Write Count: 0

Write Latency: NaN ms.

Pending Tasks: 0

Key cache capacity: 10

Key cache size: 8

Key cache hit rate: 1.0

Row cache: disabled

Compacted row minimum size: 150

Compacted row maximum size: 7152383774

Compacted row mean size: 3064535

 

No secondary indexes.

Total database disk size 823 Gb

disk_access_mode: auto on 64 bit windows os

partitioner: org.apache.cassandra.dht.ByteOrderedPartitioner

Data was stored over a period of 5 days.

Cassandra 0.7.4 was running as an embedded server.

Batch insert, using the StorageProxy.mutate.

No errors were logged during the batch insert period.

The row key is a string representation of a positive integer value. 

The same row key is used during many different mutate calls, but all super 
column names are different for each call.

The column name of the super class stored  is composed of the 32 bytes and the 
bytes of 2 integer (positive and negative) values and the bytes (UTF8) of the 
string value :[32 bytes][4 int bytes][4 int bytes][string bytes]

The custom comparator class ...ForwardCFStringValues sorts the names by first 
sorting the string , then the 32 bytes, and then the two integer values

For each column name two subcolumns are inserted with fixed name and some small 
binary value (about 40 bytes)

 

The query :

Get_slice using thrift.

Params : 

  Row key : the string representation of the positive integer String ‘1788’ 
thus hex values 31 37 38 38

  ColumnParent : the column family ForwardStringValues

  SlicePredicate : SlicePredicate(slice_range:SliceRange(start:00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 FF FF FF FF FF FF FF FF 55 52 49 4E 41 52 59 20 54 52 41 43 54 20 49 4E 46 
45 43 54 49 4F 4E, finish:7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 
7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F FF FF FF 7F FF FF FF 7F 55 52 49 4E 
41 52 59 20 54 52 41 43 54 20 49 4E 46 45 43 54 49 4F 4E, reversed:false, 
count:1))

 

This SlicePredicate is supposed to fetch all the columns with the string ‘55 52 
49 4E 41 52 59 20 54 52 41 43 54 20 49 4E 46 45 43 54 49 4F 4E’ regardless of 
the other bytes in the column name. So the start and finish have the same 
string bytes. The rest of the bytes for the start values are set to the lowest 
possible value (32 zero bytes and the bytes  representing the integer 
value -1) , the finish is set the highest possible value (32 bytes with value 
7F, …)

 

I tested the same code but with a small data set and all seemed to be OK. Even 
on the same database I get back results without exception if I use different 
String values. I’m almost sure that there should be columns with that string. 
If the string is not present I don’t get the error.

 

 

 

 

From: Aaron Morton [mailto:aa...@thelastpickle.com] 
Sent: Monday, May 16, 2011 11:33 PM
To: user@cassandra.apache.org
Subject: Re: AssertionError

 

The code is trying to follow the column index for a row in an sstable, but it 
cannot skip as many bytes as it would like to to get to the column. Helpfully 
the help says running out of bytes is only one of the reasons why this could 
happen:)

 

Can you provide some more information about the query and the data, and also 
the upgrade history for your cluster.

 

Thanks

Aaron


On 17/05/2011, at 3:07 AM, "Desimpel, Ignace"  
wrote:

Environment : java 64 bit server, java client, thrift get_slice method, 
Cassandra 0.7.4, single node

Depending on the data I pass for a query on a CF I get the following 
listed below. Any suggestions what could be wrong based on the stack trace?

 

java.l

Re: Counter Column family Cassandra 0.8 PHP Support ?

[aaron morton]

I'm not a php type person, but I can help a little with thrift.

Install thrift 0.6 and then run this in the interface/ directory of the 
cassandra source...

thrift --gen php cassandra.thrift

You should end up with the interface/gen-php/ 

Hope that helps.


-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 17 May 2011, at 11:26, bhanu choudhary wrote:

> 
> I am using counters to read the counts on my website dynamically. I am 
> looking for phpcassandra client(?) that supports counters natively. I was 
> looking if any PHP developer could give me a lead in generating the PHP 
> client code required using thrift.
> 
> Thanks in advance!

Re: AssertionError

[Sylvain Lebresne]

First thing to do would be to update to 0.7.5.

The assertionError you're running into is a assertion where we check
if a skipBytes
did skip all the bytes we had ask him to. As it turns out, the spec
for skipBytes authorize
it to not skip all the bytes asked even with no good reason. I'm
pretty sure on a linux box
skipBytes on a file will always read the number of asked bytes unless
it reaches EOF,
but I see you're running windows, so who knows what can happen.

Anyway, long story short, it's a "bug" in 0.7.4 that has been fixed in
0.7.5. If you
still run into this in 0.7.5 at least we'll know it's something else
(and we will have
a more helpful error message).

--
Sylvain

On Tue, May 17, 2011 at 12:41 PM, Desimpel, Ignace
 wrote:
> I use a custom comparator class. So I think there is a high chance that I do
> something wrong there. I was thinking that the stack trace could give a clue
> and help me on the way, maybe because some already got the same error.
>
>
>
> Anyway, here is some more information you requested.
>
>
>
> Yaml definition :
>
> name: ForwardStringValues
>
>   column_type: Super
>
>   compare_with:
> be.landc.services.search.server.db.cassandra.node.ForwardCFStringValues
>
>   compare_subcolumns_with: BytesType
>
>   keys_cached: 10
>
>   rows_cached: 0
>
>   comment: Stores the values of functions returning string
>
>   memtable_throughput_in_mb: 64
>
>   memtable_operations_in_millions: 15
>
>   min_compaction_threshold: 2
>
>   max_compaction_threshold: 5
>
>
>
> Column Family: ForwardStringValues
>
>     SSTable count: 8
>
>     Space used (live): 131311776690
>
>     Space used (total): 131311776690
>
>     Memtable Columns Count: 0
>
>     Memtable Data Size: 0
>
>     Memtable Switch Count: 0
>
>     Read Count: 1
>
>     Read Latency: 404.890 ms.
>
>     Write Count: 0
>
>     Write Latency: NaN ms.
>
>     Pending Tasks: 0
>
>     Key cache capacity: 10
>
>     Key cache size: 8
>
>     Key cache hit rate: 1.0
>
>     Row cache: disabled
>
>     Compacted row minimum size: 150
>
>     Compacted row maximum size: 7152383774
>
>     Compacted row mean size: 3064535
>
>
>
> No secondary indexes.
>
> Total database disk size 823 Gb
>
> disk_access_mode: auto on 64 bit windows os
>
> partitioner: org.apache.cassandra.dht.ByteOrderedPartitioner
>
> Data was stored over a period of 5 days.
>
> Cassandra 0.7.4 was running as an embedded server.
>
> Batch insert, using the StorageProxy.mutate.
>
> No errors were logged during the batch insert period.
>
> The row key is a string representation of a positive integer value.
>
> The same row key is used during many different mutate calls, but all super
> column names are different for each call.
>
> The column name of the super class stored  is composed of the 32 bytes and
> the bytes of 2 integer (positive and negative) values and the bytes (UTF8)
> of the string value :[32 bytes][4 int bytes][4 int bytes][string bytes]
>
> The custom comparator class ...ForwardCFStringValues sorts the names by
> first sorting the string , then the 32 bytes, and then the two integer
> values
>
> For each column name two subcolumns are inserted with fixed name and some
> small binary value (about 40 bytes)
>
>
>
> The query :
>
> Get_slice using thrift.
>
> Params :
>
>   Row key : the string representation of the positive integer String ‘1788’
> thus hex values 31 37 38 38
>
>   ColumnParent : the column family ForwardStringValues
>
>   SlicePredicate : SlicePredicate(slice_range:SliceRange(start:00 00 00 00
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00 00 00 FF FF FF FF FF FF FF FF 55 52 49 4E 41 52 59 20 54 52 41 43 54 20
> 49 4E 46 45 43 54 49 4F 4E, finish:7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F
> 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F FF FF FF 7F FF FF FF
> 7F 55 52 49 4E 41 52 59 20 54 52 41 43 54 20 49 4E 46 45 43 54 49 4F 4E,
> reversed:false, count:1))
>
>
>
> This SlicePredicate is supposed to fetch all the columns with the string ‘55
> 52 49 4E 41 52 59 20 54 52 41 43 54 20 49 4E 46 45 43 54 49 4F 4E’
> regardless of the other bytes in the column name. So the start and finish
> have the same string bytes. The rest of the bytes for the start values are
> set to the lowest possible value (32 zero bytes and the bytes 
> representing the integer value -1) , the finish is set the highest possible
> value (32 bytes with value 7F, …)
>
>
>
> I tested the same code but with a small data set and all seemed to be OK.
> Even on the same database I get back results without exception if I use
> different String values. I’m almost sure that there should be columns with
> that string. If the string 

RE: AssertionError

[Desimpel, Ignace]

Ok, I will do that (next test will be done on some linux boxes being installed 
now, but at this time I need to gone with the current windows setup).
Question : Can I use the 0.7.4 data files as is? Do I need to backup the 
datafiles in order to be able to get back to the 0.7.4 version if needed?

Ignace

-Original Message-
From: Sylvain Lebresne [mailto:sylv...@datastax.com] 
Sent: Tuesday, May 17, 2011 1:16 PM
To: user@cassandra.apache.org
Subject: Re: AssertionError

First thing to do would be to update to 0.7.5.

The assertionError you're running into is a assertion where we check if a 
skipBytes did skip all the bytes we had ask him to. As it turns out, the spec 
for skipBytes authorize it to not skip all the bytes asked even with no good 
reason. I'm pretty sure on a linux box skipBytes on a file will always read the 
number of asked bytes unless it reaches EOF, but I see you're running windows, 
so who knows what can happen.

Anyway, long story short, it's a "bug" in 0.7.4 that has been fixed in 0.7.5. 
If you still run into this in 0.7.5 at least we'll know it's something else 
(and we will have a more helpful error message).

--
Sylvain

On Tue, May 17, 2011 at 12:41 PM, Desimpel, Ignace  
wrote:
> I use a custom comparator class. So I think there is a high chance 
> that I do something wrong there. I was thinking that the stack trace 
> could give a clue and help me on the way, maybe because some already got the 
> same error.
>
>
>
> Anyway, here is some more information you requested.
>
>
>
> Yaml definition :
>
> name: ForwardStringValues
>
>   column_type: Super
>
>   compare_with:
> be.landc.services.search.server.db.cassandra.node.ForwardCFStringValue
> s
>
>   compare_subcolumns_with: BytesType
>
>   keys_cached: 10
>
>   rows_cached: 0
>
>   comment: Stores the values of functions returning string
>
>   memtable_throughput_in_mb: 64
>
>   memtable_operations_in_millions: 15
>
>   min_compaction_threshold: 2
>
>   max_compaction_threshold: 5
>
>
>
> Column Family: ForwardStringValues
>
>     SSTable count: 8
>
>     Space used (live): 131311776690
>
>     Space used (total): 131311776690
>
>     Memtable Columns Count: 0
>
>     Memtable Data Size: 0
>
>     Memtable Switch Count: 0
>
>     Read Count: 1
>
>     Read Latency: 404.890 ms.
>
>     Write Count: 0
>
>     Write Latency: NaN ms.
>
>     Pending Tasks: 0
>
>     Key cache capacity: 10
>
>     Key cache size: 8
>
>     Key cache hit rate: 1.0
>
>     Row cache: disabled
>
>     Compacted row minimum size: 150
>
>     Compacted row maximum size: 7152383774
>
>     Compacted row mean size: 3064535
>
>
>
> No secondary indexes.
>
> Total database disk size 823 Gb
>
> disk_access_mode: auto on 64 bit windows os
>
> partitioner: org.apache.cassandra.dht.ByteOrderedPartitioner
>
> Data was stored over a period of 5 days.
>
> Cassandra 0.7.4 was running as an embedded server.
>
> Batch insert, using the StorageProxy.mutate.
>
> No errors were logged during the batch insert period.
>
> The row key is a string representation of a positive integer value.
>
> The same row key is used during many different mutate calls, but all 
> super column names are different for each call.
>
> The column name of the super class stored  is composed of the 32 bytes 
> and the bytes of 2 integer (positive and negative) values and the 
> bytes (UTF8) of the string value :[32 bytes][4 int bytes][4 int 
> bytes][string bytes]
>
> The custom comparator class ...ForwardCFStringValues sorts the names 
> by first sorting the string , then the 32 bytes, and then the two 
> integer values
>
> For each column name two subcolumns are inserted with fixed name and 
> some small binary value (about 40 bytes)
>
>
>
> The query :
>
> Get_slice using thrift.
>
> Params :
>
>   Row key : the string representation of the positive integer String '1788'
> thus hex values 31 37 38 38
>
>   ColumnParent : the column family ForwardStringValues
>
>   SlicePredicate : SlicePredicate(slice_range:SliceRange(start:00 00 
> 00 00
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
> 00 00
> 00 00 00 FF FF FF FF FF FF FF FF 55 52 49 4E 41 52 59 20 54 52 41 43 
> 54 20
> 49 4E 46 45 43 54 49 4F 4E, finish:7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 
> 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F FF FF FF 
> 7F FF FF FF 7F 55 52 49 4E 41 52 59 20 54 52 41 43 54 20 49 4E 46 45 
> 43 54 49 4F 4E, reversed:false, count:1))
>
>
>
> This SlicePredicate is supposed to fetch all the columns with the 
> string '55
> 52 49 4E 41 52 59 20 54 52 41 43 54 20 49 4E 46 45 43 54 49 4F 4E'
> regardless of the other bytes in the column name. So the start and 
> finish 

Re: AssertionError

[Sylvain Lebresne]

On Tue, May 17, 2011 at 1:46 PM, Desimpel, Ignace
 wrote:
> Ok, I will do that (next test will be done on some linux boxes being 
> installed now, but at this time I need to gone with the current windows 
> setup).
> Question : Can I use the 0.7.4 data files as is? Do I need to backup the 
> datafiles in order to be able to get back to the 0.7.4 version if needed?

Yes you can use 0.7.4 data files as is. And I can't think of a reason
why you should have problem getting back to 0.7.4 if needed, though
snapshotting before cannot hurt.

>
> Ignace
>
> -Original Message-
> From: Sylvain Lebresne [mailto:sylv...@datastax.com]
> Sent: Tuesday, May 17, 2011 1:16 PM
> To: user@cassandra.apache.org
> Subject: Re: AssertionError
>
> First thing to do would be to update to 0.7.5.
>
> The assertionError you're running into is a assertion where we check if a 
> skipBytes did skip all the bytes we had ask him to. As it turns out, the spec 
> for skipBytes authorize it to not skip all the bytes asked even with no good 
> reason. I'm pretty sure on a linux box skipBytes on a file will always read 
> the number of asked bytes unless it reaches EOF, but I see you're running 
> windows, so who knows what can happen.
>
> Anyway, long story short, it's a "bug" in 0.7.4 that has been fixed in 0.7.5. 
> If you still run into this in 0.7.5 at least we'll know it's something else 
> (and we will have a more helpful error message).
>
> --
> Sylvain
>
> On Tue, May 17, 2011 at 12:41 PM, Desimpel, Ignace 
>  wrote:
>> I use a custom comparator class. So I think there is a high chance
>> that I do something wrong there. I was thinking that the stack trace
>> could give a clue and help me on the way, maybe because some already got the 
>> same error.
>>
>>
>>
>> Anyway, here is some more information you requested.
>>
>>
>>
>> Yaml definition :
>>
>> name: ForwardStringValues
>>
>>   column_type: Super
>>
>>   compare_with:
>> be.landc.services.search.server.db.cassandra.node.ForwardCFStringValue
>> s
>>
>>   compare_subcolumns_with: BytesType
>>
>>   keys_cached: 10
>>
>>   rows_cached: 0
>>
>>   comment: Stores the values of functions returning string
>>
>>   memtable_throughput_in_mb: 64
>>
>>   memtable_operations_in_millions: 15
>>
>>   min_compaction_threshold: 2
>>
>>   max_compaction_threshold: 5
>>
>>
>>
>> Column Family: ForwardStringValues
>>
>>     SSTable count: 8
>>
>>     Space used (live): 131311776690
>>
>>     Space used (total): 131311776690
>>
>>     Memtable Columns Count: 0
>>
>>     Memtable Data Size: 0
>>
>>     Memtable Switch Count: 0
>>
>>     Read Count: 1
>>
>>     Read Latency: 404.890 ms.
>>
>>     Write Count: 0
>>
>>     Write Latency: NaN ms.
>>
>>     Pending Tasks: 0
>>
>>     Key cache capacity: 10
>>
>>     Key cache size: 8
>>
>>     Key cache hit rate: 1.0
>>
>>     Row cache: disabled
>>
>>     Compacted row minimum size: 150
>>
>>     Compacted row maximum size: 7152383774
>>
>>     Compacted row mean size: 3064535
>>
>>
>>
>> No secondary indexes.
>>
>> Total database disk size 823 Gb
>>
>> disk_access_mode: auto on 64 bit windows os
>>
>> partitioner: org.apache.cassandra.dht.ByteOrderedPartitioner
>>
>> Data was stored over a period of 5 days.
>>
>> Cassandra 0.7.4 was running as an embedded server.
>>
>> Batch insert, using the StorageProxy.mutate.
>>
>> No errors were logged during the batch insert period.
>>
>> The row key is a string representation of a positive integer value.
>>
>> The same row key is used during many different mutate calls, but all
>> super column names are different for each call.
>>
>> The column name of the super class stored  is composed of the 32 bytes
>> and the bytes of 2 integer (positive and negative) values and the
>> bytes (UTF8) of the string value :[32 bytes][4 int bytes][4 int
>> bytes][string bytes]
>>
>> The custom comparator class ...ForwardCFStringValues sorts the names
>> by first sorting the string , then the 32 bytes, and then the two
>> integer values
>>
>> For each column name two subcolumns are inserted with fixed name and
>> some small binary value (about 40 bytes)
>>
>>
>>
>> The query :
>>
>> Get_slice using thrift.
>>
>> Params :
>>
>>   Row key : the string representation of the positive integer String '1788'
>> thus hex values 31 37 38 38
>>
>>   ColumnParent : the column family ForwardStringValues
>>
>>   SlicePredicate : SlicePredicate(slice_range:SliceRange(start:00 00
>> 00 00
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 00 00
>> 00 00 00 FF FF FF FF FF FF FF FF 55 52 49 4E 41 52 59 20 54 52 41 43
>> 54 20
>> 49 4E 46 45 43 54 49 4F 4E, finish:7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F 7F
>> 7F 7F 7F 7F 7F 7F 

RE: AssertionError

[Desimpel, Ignace]

Seems like the AbstractType class has changed going from 0.7.4 to 0.7.5. 
It is now required to implement a compose and decompose method. Already did 
that, and it starts up with the 0.7.5 code using the 0.7.4 data and 
configuration (using a smaller extra test database)
Below I made a sample implementations to illustrate another question : On the 
compose method , can I simply create my own AbstractType class and use the 
given ByteBuffer. Or like in the decompose example, do I need to duplicate the 
ByteBuffer or could the paramT object  be reused or should I make a complete 
copy?

@Override
public Object compose(ByteBuffer paramByteBuffer){
ReverseCFFloatValues oNew = new ReverseCFFloatValues();
oNew.paramByteBuffer = paramByteBuffer;
return oNew;
}

@Override
public ByteBuffer decompose(Object paramT){
return 
((ReverseCFFloatValues)paramT).paramByteBuffer.duplicate();
}



-Original Message-
From: Sylvain Lebresne [mailto:sylv...@datastax.com] 
Sent: Tuesday, May 17, 2011 2:50 PM
To: user@cassandra.apache.org
Subject: Re: AssertionError

On Tue, May 17, 2011 at 1:46 PM, Desimpel, Ignace  
wrote:
> Ok, I will do that (next test will be done on some linux boxes being 
> installed now, but at this time I need to gone with the current windows 
> setup).
> Question : Can I use the 0.7.4 data files as is? Do I need to backup the 
> datafiles in order to be able to get back to the 0.7.4 version if needed?

Yes you can use 0.7.4 data files as is. And I can't think of a reason why you 
should have problem getting back to 0.7.4 if needed, though snapshotting before 
cannot hurt.

>
> Ignace
>
> -Original Message-
> From: Sylvain Lebresne [mailto:sylv...@datastax.com]
> Sent: Tuesday, May 17, 2011 1:16 PM
> To: user@cassandra.apache.org
> Subject: Re: AssertionError
>
> First thing to do would be to update to 0.7.5.
>
> The assertionError you're running into is a assertion where we check if a 
> skipBytes did skip all the bytes we had ask him to. As it turns out, the spec 
> for skipBytes authorize it to not skip all the bytes asked even with no good 
> reason. I'm pretty sure on a linux box skipBytes on a file will always read 
> the number of asked bytes unless it reaches EOF, but I see you're running 
> windows, so who knows what can happen.
>
> Anyway, long story short, it's a "bug" in 0.7.4 that has been fixed in 0.7.5. 
> If you still run into this in 0.7.5 at least we'll know it's something else 
> (and we will have a more helpful error message).
>
> --
> Sylvain
>
> On Tue, May 17, 2011 at 12:41 PM, Desimpel, Ignace 
>  wrote:
>> I use a custom comparator class. So I think there is a high chance 
>> that I do something wrong there. I was thinking that the stack trace 
>> could give a clue and help me on the way, maybe because some already got the 
>> same error.
>>
>>
>>
>> Anyway, here is some more information you requested.
>>
>>
>>
>> Yaml definition :
>>
>> name: ForwardStringValues
>>
>>   column_type: Super
>>
>>   compare_with:
>> be.landc.services.search.server.db.cassandra.node.ForwardCFStringValu
>> e
>> s
>>
>>   compare_subcolumns_with: BytesType
>>
>>   keys_cached: 10
>>
>>   rows_cached: 0
>>
>>   comment: Stores the values of functions returning string
>>
>>   memtable_throughput_in_mb: 64
>>
>>   memtable_operations_in_millions: 15
>>
>>   min_compaction_threshold: 2
>>
>>   max_compaction_threshold: 5
>>
>>
>>
>> Column Family: ForwardStringValues
>>
>>     SSTable count: 8
>>
>>     Space used (live): 131311776690
>>
>>     Space used (total): 131311776690
>>
>>     Memtable Columns Count: 0
>>
>>     Memtable Data Size: 0
>>
>>     Memtable Switch Count: 0
>>
>>     Read Count: 1
>>
>>     Read Latency: 404.890 ms.
>>
>>     Write Count: 0
>>
>>     Write Latency: NaN ms.
>>
>>     Pending Tasks: 0
>>
>>     Key cache capacity: 10
>>
>>     Key cache size: 8
>>
>>     Key cache hit rate: 1.0
>>
>>     Row cache: disabled
>>
>>     Compacted row minimum size: 150
>>
>>     Compacted row maximum size: 7152383774
>>
>>     Compacted row mean size: 3064535
>>
>>
>>
>> No secondary indexes.
>>
>> Total database disk size 823 Gb
>>
>> disk_access_mode: auto on 64 bit windows os
>>
>> partitioner: org.apache.cassandra.dht.ByteOrderedPartitioner
>>
>> Data was stored over a period of 5 days.
>>
>> Cassandra 0.7.4 was running as an embedded server.
>>
>> Batch insert, using the StorageProxy.mutate.
>>
>> No errors were logged during the batch insert period.
>>
>> The row key is a string representation of a positive integer value.
>>
>> The same row key is used during m

Re: Dynamic Snitch Problem

[Jonathan Ellis]

I guess otherwise it NPEs?

Looks like a bug, can you open a ticket?

On Tue, May 17, 2011 at 4:19 AM, Daniel Doubleday
 wrote:
> Hi all
>
> after upgrading to 0.7 we have a small problem with dynamic snitch:
>
> we have rf=3, quorum read/write and read repair prop set to 0. Thus cassandra 
> always shortcuts reads to only 2 hosts.
>
> Problem is that one of our nodes get ignored unless using a little patch and 
> initialize the scores.
>
> Anybody else sees this?
>
> Cheers,
> Daniel
>
> btw: we had to do this on 0.6 too when using the 'shortcut' read path.
>
> Index: src/java/org/apache/cassandra/locator/DynamicEndpointSnitch.java
> ===
> --- src/java/org/apache/cassandra/locator/DynamicEndpointSnitch.java    
> (revision 1100429)
> +++ src/java/org/apache/cassandra/locator/DynamicEndpointSnitch.java    
> (revision )
> @@ -178,8 +178,16 @@
>     {
>         Double scored1 = scores.get(a1);
>         Double scored2 = scores.get(a2);
> -
> +
> -        if (scored1 == null || scored2 == null || scored1.equals(scored2))
> +        if (scored1 == null) {
> +            scored1 = 0.0;
> +        }
> +
> +        if (scored2 == null) {
> +            scored2 = 0.0;
> +        }
> +
> +        if (scored1.equals(scored2))
>             return subsnitch.compareEndpoints(target, a1, a2);
>         if (scored1 < scored2)
>             return -1;
>
>
>
>
>
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: Cassandra Hackathon?

[Nathan Milford]

I'll tag along too.

Outbrain would be up for sponsoring beer or snacks if needed.

Nathan Milford
http://blog.milford.io



On Tue, May 17, 2011 at 2:29 AM, Joseph Stein  wrote:

> awesome!
>
> this week I am (finally) getting cassandra (0.8) going for existing
> projects we have in production.
>
> Looking at https://issues.apache.org/jira/browse/CASSANDRA-2495 is a place
> I was thinking maybe I could start to help out but I am not sure that is the
> best starting point though it is a sore spot for some of what we want to be
> doing in future projects.  Any thoughts from folks on this?  should I come
> up with an approach and comment in JIRA is best?  start smaller? bigger?
> some other tickets good for the gander?  more help for 0.80 in some specific
> places while I am working on it?
>
> I want the project I am working on to go well with Cassandra so the more I
> jump into it the better.
>
> if it just the two of us we will have more than enough pizza and beer
> (Medialet's treat) but hopefully we can get some others too.
>
> On Tue, May 17, 2011 at 12:04 AM, Edward Capriolo 
> wrote:
>
>> I had it on our list of ideas for the Cassandra NYC meetup. I am down for
>> action.
>>
>>
>>
>>
>> On Mon, May 16, 2011 at 9:40 PM, Joseph Stein  wrote:
>>
>>> Any interest for a Cassandra Hackathon evening in NYC?  Any committer(s)
>>> going to be in the NYC area together that can lead/guide this?
>>>
>>> http://www.meetup.com/NYC-Cassandra-User-Group/events/18635801/
>>>
>>> I have a thumbs up to use our office www.medialets.com in the Milk
>>> Studios building. It is a big open space with 12 tables (2-3 people per
>>> table) all in one big room + a conference room we can gather around a big
>>> screen if/when need be too..
>>>
>>> I would love to start contributing code myself and think this is a great
>>> way to get it going for others too to get over the hump (and simply make
>>> time) to knock out tickets together with good guidance growing the
>>> community.
>>>
>>>
>>> /*
>>> Joe Stein
>>> http://www.linkedin.com/in/charmalloc
>>> Twitter: @allthingshadoop
>>> */
>>>
>>
>>
>
>
> --
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop
> */
>

Re: Inconsistent results using secondary indexes between two DC

[Jonathan Ellis]

Nothing comes to mind.

I'd start by using sstable2json to see if the missing rows are in the
main data CF -- i.e., are they just unindexed, or are they missing
completely?

On Sun, May 15, 2011 at 4:33 PM, Wojciech Pietrzok  wrote:
> Hello,
>
> I've noticed strange behaviour of Cassandra when using secondary indexes.
> There are 2 Data Centers, each with 2 nodes, RF=4, on all nodes
> Cassandra 0.7.5 is installed.
> When I connect to one of the nodes in DC1 and perform query using
> secondary indexes ("get ColumnFamily where column = 'foo'" in
> cassandra-cli) I always get correct number of rows returned, no matter
> which ConsistencyLevel is set.
> When I connect to one of the nodes in DC2 and perform same query using
> ConsistencyLevel LOCAL_QUORUM the results are correct. But using
> ConsistencyLevel ONE Cassandra doesn't return correct number of rows
> (it seems that most of the times there some of the rows are missing).
> Tried running nodetool repair, and nodetool scrub but this doesn't seem to 
> help.
>
> What might the cause of such behaviour?
>
> --
> KosciaK
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: Commitlog Disk Full

[mcasandra]

Do you see anything in log files?

--
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Commitlog-Disk-Full-tp6356797p6374234.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.

Question regarding Compaction

[Ali Ahsan]

Hi All,

I was  diagnosing  high  CPU usage on my Cassandra 0.6.3.Then i came 
across  Cassandra wiki link which suggested following.



"

To lower compaction priority (thus reducing its impact on the rest of 
the system, and making it take longer), add these options to 
cassandra.in.sh in 0.6.3 or later:


-XX:+UseThreadPriorities \
-XX:ThreadPriorityPolicy=42 \
-Dcassandra.compaction.priority=1 \

"

http://wiki.apache.org/cassandra/PerformanceTuning


After adding above in my cassandra.in.sh my high CPU usage seems to 
dropped very very low in fact it seem nothing is running on machine's.


So here is my question do i need to run nodetool compact or not and what 
about noretool rapier.


--
S.Ali Ahsan

Re: Commitlog Disk Full

[Sanjeev Kulkarni]

Here is the snapshot of the logs from one of the machine between the time
when I mailed to when it finally started to look at the commitlogs. At the
end you can see that it is discarding obsolete commitlogs. Not sure what
that means.


INFO [HintedHandoff:1] 2011-05-17 01:17:10,510 ColumnFamilyStore.java (line
1070) Enqueuing flush of Memtable-HintsColumnFamily@866759847(153737 bytes,
3271 operations)
 INFO [FlushWriter:309] 2011-05-17 01:17:10,511 Memtable.java (line 158)
Writing Memtable-HintsColumnFamily@866759847(153737 bytes, 3271 operations)
 INFO [CompactionExecutor:1] 2011-05-17 01:17:10,511 CompactionManager.java
(line 395) Compacting
[SSTableReader(path='/mnt/cassandra/data/system/HintsColumnFamily-f-92-Data.db'),SSTableReader(path='/mnt/cassandra/data/system/HintsColumnFamily-f-91-Data.db')]
 INFO [FlushWriter:309] 2011-05-17 01:17:10,652 Memtable.java (line 165)
Completed flushing /mnt/cassandra/data/system/HintsColumnFamily-f-93-Data.db
(375072 bytes)
 INFO [CompactionExecutor:1] 2011-05-17 01:17:10,654 CompactionManager.java
(line 482) Compacted to
/mnt/cassandra/data/system/HintsColumnFamily-tmp-f-94-Data.db.  514,078 to
511,862 (~99% of original) bytes for 2 keys.  Time: 142ms.
 INFO [HintedHandoff:1] 2011-05-17 01:17:10,654 HintedHandOffManager.java
(line 360) Finished hinted handoff of 3271 rows to endpoint /10.32.6.238
 INFO [COMMIT-LOG-WRITER] 2011-05-17 01:17:10,753 CommitLog.java (line 440)
Discarding obsolete commit
log:CommitLogSegment(/mnt/cassandra/commitlog/CommitLog-1305593058271.log)
 INFO [COMMIT-LOG-WRITER] 2011-05-17 01:17:10,754 CommitLog.java (line 440)
Discarding obsolete commit
log:CommitLogSegment(/mnt/cassandra/commitlog/CommitLog-1305593776786.log)
 INFO [COMMIT-LOG-WRITER] 2011-05-17 01:17:10,754 CommitLog.java (line 440)
Discarding obsolete commit
log:CommitLogSegment(/mnt/cassandra/commitlog/CommitLog-1305593808049.log)
 INFO [COMMIT-LOG-WRITER] 2011-05-17 01:17:10,754 CommitLog.java (line 440)
Discarding obsolete commit
log:CommitLogSegment(/mnt/cassandra/commitlog/CommitLog-1305593840228.log)
 INFO [HintedHandoff:1] 2011-05-17 01:17:58,010 HintedHandOffManager.java
(line 304) Started hinted handoff for endpoint /10.32.6.238
 INFO [HintedHandoff:1] 2011-05-17 01:17:58,111 ColumnFamilyStore.java (line
1070) Enqueuing flush of Memtable-HintsColumnFamily@1025764186(47 bytes, 1
operations)
 INFO [FlushWriter:309] 2011-05-17 01:17:58,111 Memtable.java (line 158)
Writing Memtable-HintsColumnFamily@1025764186(47 bytes, 1 operations)
 INFO [CompactionExecutor:1] 2011-05-17 01:17:58,112 CompactionManager.java
(line 395) Compacting
[SSTableReader(path='/mnt/cassandra/data/system/HintsColumnFamily-f-94-Data.db'),SSTableReader(path='/mnt/cassandra/data/system/HintsColumnFamily-f-93-Data.db')]
 INFO [FlushWriter:309] 2011-05-17 01:17:58,119 Memtable.java (line 165)
Completed flushing /mnt/cassandra/data/system/HintsColumnFamily-f-95-Data.db
(127 bytes)
 INFO [CompactionExecutor:1] 2011-05-17 01:17:58,192 CompactionManager.java
(line 482) Compacted to
/mnt/cassandra/data/system/HintsColumnFamily-tmp-f-96-Data.db.  886,934 to
123,830 (~13% of original) bytes for 2 keys.  Time: 80ms.
 INFO [HintedHandoff:1] 2011-05-17 01:17:58,192 HintedHandOffManager.java
(line 360) Finished hinted handoff of 1 rows to endpoint /10.32.6.238
 INFO [NonPeriodicTasks:1] 2011-05-17 02:58:51,856 SSTable.java (line 147)
Deleted /mnt/cassandra/data/system/HintsColumnFamily-f-93
 INFO [NonPeriodicTasks:1] 2011-05-17 02:58:51,857 SSTable.java (line 147)
Deleted /mnt/cassandra/data/system/HintsColumnFamily-f-94
 INFO [CompactionExecutor:1] 2011-05-17 04:39:16,679 CacheWriter.java (line
96) Saved ObjectKeySpace-Profile-KeyCache (20 items) in 352 ms
 INFO [CompactionExecutor:1] 2011-05-17 04:42:24,773 CacheWriter.java (line
96) Saved ObjectKeySpace-Location-KeyCache (20 items) in 392 ms
 INFO [CompactionExecutor:1] 2011-05-17 04:43:05,117 CacheWriter.java (line
96) Saved ObjectKeySpace-SpaceLocation-KeyCache (186874 items) in 358 ms
 INFO [CompactionExecutor:1] 2011-05-17 04:43:35,001 CacheWriter.java (line
96) Saved ObjectKeySpace-PartitionInfo-KeyCache (29073 items) in 38 ms
 INFO [MutationStage:886] 2011-05-17 06:32:02,024 ColumnFamilyStore.java
(line 1070) Enqueuing flush of Memtable-PartitionInfo@1500472048(136213003
bytes, 2081174 operations)
 INFO [FlushWriter:310] 2011-05-17 06:32:02,036 Memtable.java (line 158)
Writing Memtable-PartitionInfo@1500472048(136213003 bytes, 2081174
operations)
 INFO [FlushWriter:310] 2011-05-17 06:32:11,001 Memtable.java (line 165)
Completed flushing
/mnt/cassandra/data/ObjectKeySpace/PartitionInfo-f-350-Data.db (137487577
bytes)
 INFO [NonPeriodicTasks:1] 2011-05-17 06:35:47,390 SSTable.java (line 147)
Deleted /mnt/cassandra/data/system/HintsColumnFamily-f-92
 INFO [NonPeriodicTasks:1] 2011-05-17 06:35:47,391 SSTable.java (line 147)
Deleted /mnt/cassandra/data/system/HintsColumnFamily-f-91
 INFO [MutationStage:899] 2011-05-17 06:41:37,170 ColumnFamilyStore.java

Re: Question regarding Compaction

[Jonathan Ellis]

You've had people on this list tell you you should upgrade off such an
old release at least twice already this month.

The reason we do this is, the odds are good that the answer is
"someone else ran into that problem a year ago and it got fixed."
Personally, I don't think it's worth the time to troubleshoot before
upgrading.

Re compaction and repair, you can read up on both at
http://www.datastax.com/docs/0.8/operations/scheduled_tasks.

On Tue, May 17, 2011 at 1:30 PM, Ali Ahsan  wrote:
> Hi All,
>
> I was  diagnosing  high  CPU usage on my Cassandra 0.6.3.Then i came across
>  Cassandra wiki link which suggested following.
>
>
> "
>
> To lower compaction priority (thus reducing its impact on the rest of the
> system, and making it take longer), add these options to cassandra.in.sh in
> 0.6.3 or later:
>
> -XX:+UseThreadPriorities \
> -XX:ThreadPriorityPolicy=42 \
> -Dcassandra.compaction.priority=1 \
>
> "
>
> http://wiki.apache.org/cassandra/PerformanceTuning
>
>
> After adding above in my cassandra.in.sh my high CPU usage seems to dropped
> very very low in fact it seem nothing is running on machine's.
>
> So here is my question do i need to run nodetool compact or not and what
> about noretool rapier.
>
> --
> S.Ali Ahsan
>
>
>
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: Question regarding Compaction

[Ali Ahsan]

On 05/17/2011 11:57 PM, Jonathan Ellis wrote:

You've had people on this list tell you you should upgrade off such an
old release at least twice already this month.

The reason we do this is, the odds are good that the answer is
"someone else ran into that problem a year ago and it got fixed."
Personally, I don't think it's worth the time to troubleshoot before
upgrading


Yes true we need to upgrade ,And we are planning for this.

--
S.Ali Ahsan

Re: problems using property snitch file

[aaron morton]

Looking at the code the exception below can happen if the ip address cannot be 
found in the topology and there is no default set. 

To test this theory set the logging the DEBUG and look for a log message from 
the PropertyFileSnitch that says
"Could not find end point information for {}, will use default" {} will be the 
endpoint. 

Also during startup the snitch will log what it was able to read so you can 
check it is read what you think. 

I there is a problem with the topology that would probably explain the other 
error.

If that does not fix it can you provide the keyspace definition, and settings 
such as listen_address and rpc_address . 

Cheers

-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 17 May 2011, at 03:44, Anurag Gujral wrote:

> All lines have different IP address,yes I am trying to move single node in DC1
> Thanks
> Anurag
> 
> On Mon, May 16, 2011 at 3:19 AM, aaron morton  wrote:
> Do all the lines have the same IP address or is that just how you redacted 
> them ?
> 
> is the single node in DC1 the one you are moving?
> 
> Cheers
> 
> -
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 16 May 2011, at 12:28, Anurag Gujral wrote:
> 
>> IPAddress1=DC1:RAC1
>> IPAddress1=DC2:RAC1
>> IPAddress1=DC2:RAC1
>> IPAddress1=DC2:RAC1
>> 
>> Thanks
>> Anurag
>> 
>> On Sun, May 15, 2011 at 4:52 PM, aaron morton  
>> wrote:
>> Whats does your cassandra-topology.properties file look like ?
>> 
>> Cheers
>> 
>> -
>> Aaron Morton
>> Freelance Cassandra Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>> 
>> On 14 May 2011, at 11:48, Anurag Gujral wrote:
>> 
>> > Hi All,
>> >   I have built a cassandra cluster which is using property snitch 
>> > file to specify my network topology to cassandra.The keyspace I have use 
>> > network topology strategy.
>> >
>> > When I tried to add new node to this cluster,I get the following error:
>> > RROR [main] 2011-05-13 23:45:44,152 AbstractCassandraDaemon.java (line 
>> > 333) Exception encountered during startup.
>> > java.lang.NullPointerException
>> > at 
>> > org.apache.cassandra.locator.PropertyFileSnitch.getDatacenter(PropertyFileSnitch.java:96)
>> > at 
>> > org.apache.cassandra.locator.NetworkTopologyStrategy.calculateNaturalEndpoints(NetworkTopologyStrategy.java:87)
>> > at 
>> > org.apache.cassandra.locator.AbstractReplicationStrategy.getAddressRanges(AbstractReplicationStrategy.java:196)
>> > at 
>> > org.apache.cassandra.locator.AbstractReplicationStrategy.getPendingAddressRanges(AbstractReplicationStrategy.java:230)
>> > at 
>> > org.apache.cassandra.dht.BootStrapper.getRangesWithSources(BootStrapper.java:195)
>> > at 
>> > org.apache.cassandra.dht.BootStrapper.bootstrap(BootStrapper.java:83)
>> > at 
>> > org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:525)
>> > at 
>> > org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:453)
>> > at 
>> > org.apache.cassandra.service.StorageService.initServer(StorageService.java:403)
>> > at 
>> > org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:194)
>> > at 
>> > org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
>> > at 
>> > org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
>> >
>> >
>> > Please Advise.
>> >
>> >
>> > Thanks
>> > Anurag
>> >
>> 
>> 
> 
> 

Re: Questions about using MD5 encryption with SimpleAuthenticator

[aaron morton]

Use the plain text password via the cli, the server will make a hash and 
compare it to the one in the file. 

wrt SHA-2 I'm not a security guy but MD5 is probably "good enough" for the 
problem of storing passwords in plain text in a file. 

Hope that helps. 

-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 17 May 2011, at 10:59, Sameer Farooqui wrote:

> By the way, just noticed a typo in my email below. I'm using the correct 
> keyspace name in all locations on the cluster... however in my examples 
> below, I used MyKeyspace in some spots and MDR in other spots, but in the 
> cluster I'm specifying the same keyspace name everywhere, so that's not the 
> issue.
> 
> - Sameer
> 
> 
> On Mon, May 16, 2011 at 3:55 PM, Sameer Farooqui  
> wrote:
> Hi all,
> 
> We are trying to use MD5 encrypted passwords. Quick question first - Is SHA-2 
> supported yet? US-CERT of the U. S. Department of Homeland Security has said 
> that MD5 "should be considered cryptographically broken and unsuitable for 
> further use”, and SHA-2 family of hash functions is recommended.
> 
> The issue I'm seeing is that when I turn on MD5 encryption, I can't log into 
> the cluster from Cassandra-CLI (I get a login failure).
> 
> The cassandra.in.sh file has been changed as so:
> 
> JVM_OPTS="
> 
> -Dpasswd.properties=/home/ubuntu/apache-cassandra-0.8.0-beta1/conf/passwd.properties
>  \
> 
> -Daccess.properties=/home/ubuntu/apache-cassandra-0.8.0-beta1/conf/access.properties
>  \
> -Dpasswd.mode=MD5"
> 
> 
> And I ran this python script to generate a MD5 hash:
> ubuntu@darknet:~$ python
> Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
> [GCC 4.4.5] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from hashlib import md5
> >>> p = "nosql"
> >>> h = md5(p).hexdigest()
> >>> print h
> 9fa1b39e7eb877367213e6f7e37d0b01
> 
> 
> Then I updated the passwd.properties file with the new hashed password:
> jdoe=9fa1b39e7eb877367213e6f7e37d0b01
> 
> 
> Also, the access.properties file is properly set so that jdoe has rw access 
> to the keyspace and CF:
> MyKeyspace.=jdoe,jsmith
> MyKeyspace.MyCF.=jsmith,jdoe
> 
> 
> But when I try to connect to the cluster now, I'm getting a login failure. I 
> have tried a few different ways of connecting:
> 
> Ran this from the Cassandra CLI: 
> [default@unknown] connect ec2-50-19-26-189.compute-1.amazonaws.com/9160 jdoe 
> '9fa1b39e7eb877367213e6f7e37d0b01';
> Login failure. Did you specify 'keyspace', 'username' and 'password'?
> 
> 
> Ran these from the Ubuntu CLI:
> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$ 
> bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u jdoe 
> -pw 9fa1b39e7eb877367213e6f7e37d0b01 -k MDR
> Login failure. Did you specify 'keyspace', 'username' and 'password'?
> 
> 
> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$ 
> bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u jdoe 
> -pw '9fa1b39e7eb877367213e6f7e37d0b01' -k MDR
> Login failure. Did you specify 'keyspace', 'username' and 'password'?
> 
> 
> Hmm, what am I doing wrong?
> 
> - Sameer
> 
> 
> 

Re: Questions about using MD5 encryption with SimpleAuthenticator

[Jonathan Ellis]

http://codahale.com/how-to-safely-store-a-password/

On Tue, May 17, 2011 at 3:03 PM, aaron morton  wrote:
> Use the plain text password via the cli, the server will make a hash and
> compare it to the one in the file.
> wrt SHA-2 I'm not a security guy but MD5 is probably "good enough" for the
> problem of storing passwords in plain text in a file.
> Hope that helps.
> -
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
> On 17 May 2011, at 10:59, Sameer Farooqui wrote:
>
> By the way, just noticed a typo in my email below. I'm using the correct
> keyspace name in all locations on the cluster... however in my examples
> below, I used MyKeyspace in some spots and MDR in other spots, but in the
> cluster I'm specifying the same keyspace name everywhere, so that's not the
> issue.
> - Sameer
>
>
> On Mon, May 16, 2011 at 3:55 PM, Sameer Farooqui 
> wrote:
>>
>> Hi all,
>> We are trying to use MD5 encrypted passwords. Quick question first - Is
>> SHA-2 supported yet? US-CERT of the U. S. Department of Homeland Security
>> has said that MD5 "should be considered cryptographically broken and
>> unsuitable for further use”, and SHA-2 family of hash functions is
>> recommended.
>> The issue I'm seeing is that when I turn on MD5 encryption, I can't log
>> into the cluster from Cassandra-CLI (I get a login failure).
>> The cassandra.in.sh file has been changed as so:
>> JVM_OPTS="
>>
>> -Dpasswd.properties=/home/ubuntu/apache-cassandra-0.8.0-beta1/conf/passwd.properties
>> \
>>
>> -Daccess.properties=/home/ubuntu/apache-cassandra-0.8.0-beta1/conf/access.properties
>> \
>>         -Dpasswd.mode=MD5"
>>
>> And I ran this python script to generate a MD5 hash:
>> ubuntu@darknet:~$ python
>> Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
>> [GCC 4.4.5] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> from hashlib import md5
>> >>> p = "nosql"
>> >>> h = md5(p).hexdigest()
>> >>> print h
>> 9fa1b39e7eb877367213e6f7e37d0b01
>>
>> Then I updated the passwd.properties file with the new hashed password:
>> jdoe=9fa1b39e7eb877367213e6f7e37d0b01
>>
>> Also, the access.properties file is properly set so that jdoe has rw
>> access to the keyspace and CF:
>> MyKeyspace.=jdoe,jsmith
>> MyKeyspace.MyCF.=jsmith,jdoe
>>
>> But when I try to connect to the cluster now, I'm getting a login failure.
>> I have tried a few different ways of connecting:
>> Ran this from the Cassandra CLI:
>> [default@unknown] connect ec2-50-19-26-189.compute-1.amazonaws.com/9160
>> jdoe '9fa1b39e7eb877367213e6f7e37d0b01';
>> Login failure. Did you specify 'keyspace', 'username' and 'password'?
>>
>> Ran these from the Ubuntu CLI:
>> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$
>> bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u
>> jdoe -pw 9fa1b39e7eb877367213e6f7e37d0b01 -k MDR
>> Login failure. Did you specify 'keyspace', 'username' and 'password'?
>>
>> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$
>> bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u
>> jdoe -pw '9fa1b39e7eb877367213e6f7e37d0b01' -k MDR
>> Login failure. Did you specify 'keyspace', 'username' and 'password'?
>>
>> Hmm, what am I doing wrong?
>> - Sameer
>>
>
>
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: Commitlog Disk Full

[mcasandra]

Those messages are "ok" to ignore. It's basically deleting the files that are
already flused as SSTables.

Which version are you running?

Have you tried restarting the node?

Pick one node and send "ls -ltr" output also the complete log files since
your last restart from the same node. I looked at the code and it looks like
you should see something in the logs for those files.

--
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Commitlog-Disk-Full-tp6356797p6375353.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.

Re: Questions about using MD5 encryption with SimpleAuthenticator

[Sameer Farooqui]

Hey Aaron,

Unfortunately it fails with plaintext password also:

ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$
bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u
jdoe -pw 'nosql' -k MDR  Login failure. Did you specify 'keyspace',
'username' and 'password'?
Welcome to the Cassandra CLI.


ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$
bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u
jdoe -pw nosql -k MDR
Login failure. Did you specify 'keyspace', 'username' and 'password'?
Welcome to the Cassandra CLI.


Regarding the security of MD5, I'm not a security guy either, but it seems
quiet easy to crack, especially for short passwords.

This website was quickly able to decrypt my MD5 digest (which is honestly
not very complex) and give me the original plaintext:
http://md5.noisette.ch/index.php

Longer list of MD5 rainbow table sites:
http://www.stottmeister.com/blog/2009/04/14/how-to-crack-md5-passwords/

Anyway, any help with the original question of how to input the password the
the Cassandra-CLI would be much appreciated!


- Sameer



On Tue, May 17, 2011 at 1:03 PM, aaron morton wrote:

> Use the plain text password via the cli, the server will make a hash and
> compare it to the one in the file.
>
> wrt SHA-2 I'm not a security guy but MD5 is probably "good enough" for the
> problem of storing passwords in plain text in a file.
>
> Hope that helps.
>
> -
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 17 May 2011, at 10:59, Sameer Farooqui wrote:
>
> By the way, just noticed a typo in my email below. I'm using the correct
> keyspace name in all locations on the cluster... however in my examples
> below, I used MyKeyspace in some spots and MDR in other spots, but in the
> cluster I'm specifying the same keyspace name everywhere, so that's not the
> issue.
>
> - Sameer
>
>
> On Mon, May 16, 2011 at 3:55 PM, Sameer Farooqui 
> wrote:
>
>> Hi all,
>>
>> We are trying to use MD5 encrypted passwords. Quick question first - Is
>> SHA-2 supported yet? US-CERT of the U. S. Department of Homeland Security
>> has said that MD5 "should be considered cryptographically broken and
>> unsuitable for further use”, and SHA-2 family of hash functions is
>> recommended.
>>
>> The issue I'm seeing is that when I turn on MD5 encryption, I can't log
>> into the cluster from Cassandra-CLI (I get a login failure).
>>
>> The cassandra.in.sh file has been changed as so:
>>
>> JVM_OPTS="
>>
>> -Dpasswd.properties=/home/ubuntu/apache-cassandra-0.8.0-beta1/conf/passwd.properties
>> \
>>
>> -Daccess.properties=/home/ubuntu/apache-cassandra-0.8.0-beta1/conf/access.properties
>> \
>> -Dpasswd.mode=MD5"
>>
>>
>> And I ran this python script to generate a MD5 hash:
>> ubuntu@darknet:~$ python
>> Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
>> [GCC 4.4.5] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> from hashlib import md5
>> >>> p = "nosql"
>> >>> h = md5(p).hexdigest()
>> >>> print h
>> 9fa1b39e7eb877367213e6f7e37d0b01
>>
>>
>> Then I updated the passwd.properties file with the new hashed password:
>> jdoe=9fa1b39e7eb877367213e6f7e37d0b01
>>
>>
>> Also, the access.properties file is properly set so that jdoe has rw
>> access to the keyspace and CF:
>> MyKeyspace.=jdoe,jsmith
>> MyKeyspace.MyCF.=jsmith,jdoe
>>
>>
>> But when I try to connect to the cluster now, I'm getting a login failure.
>> I have tried a few different ways of connecting:
>>
>> Ran this from the Cassandra CLI:
>> [default@unknown] connect ec2-50-19-26-189.compute-1.amazonaws.com/9160jdoe 
>> '9fa1b39e7eb877367213e6f7e37d0b01';
>> Login failure. Did you specify 'keyspace', 'username' and 'password'?
>>
>>
>> Ran these from the Ubuntu CLI:
>> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$
>> bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u
>> jdoe -pw 9fa1b39e7eb877367213e6f7e37d0b01 -k MDR
>> Login failure. Did you specify 'keyspace', 'username' and 'password'?
>>
>>
>> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$
>> bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u
>> jdoe -pw '9fa1b39e7eb877367213e6f7e37d0b01' -k MDR
>> Login failure. Did you specify 'keyspace', 'username' and 'password'?
>>
>>
>> Hmm, what am I doing wrong?
>>
>> - Sameer
>>
>>
>>
>
>

[RELEASE] Apache Cassandra 0.8.0 rc1

[Eric Evans]

I am pleased to announce the release of Apache Cassandra 0.8.0 rc1.

The final release is upon us so if you're planning to give us a hand
with testing, now is the time!

As always, be sure to have a look at the changelog[1] and release
notes[2]. Report any problems you find[3], and if you have any
questions, please ask.

Thanks!

[1]: http://goo.gl/Rh3A3 (CHANGES.txt)
[2]: http://goo.gl/wbXGM (NEWS.txt)
[3]: https://issues.apache.org/jira/browse/CASSANDRA
[3]: http://cassandra.apache.org/download
[4]: http://wiki.apache.org/cassandra/DebianPackaging

-- 
Eric Evans
eev...@rackspace.com

Re: [RELEASE] Apache Cassandra 0.8.0 rc1

[Stephen Connolly]

Eric,

You forgot to release the staging repository on repository.apache.org so
that the artifacts get pushed to Maven Central.

-Stephen

On 17 May 2011 23:15, Eric Evans  wrote:

>
> I am pleased to announce the release of Apache Cassandra 0.8.0 rc1.
>
> The final release is upon us so if you're planning to give us a hand
> with testing, now is the time!
>
> As always, be sure to have a look at the changelog[1] and release
> notes[2]. Report any problems you find[3], and if you have any
> questions, please ask.
>
> Thanks!
>
> [1]: http://goo.gl/Rh3A3 (CHANGES.txt)
> [2]: http://goo.gl/wbXGM (NEWS.txt)
> [3]: https://issues.apache.org/jira/browse/CASSANDRA
> [3]: http://cassandra.apache.org/download
> [4]: http://wiki.apache.org/cassandra/DebianPackaging
>
> --
> Eric Evans
> eev...@rackspace.com
>
>

Cassandra Vs. Oracle Coherence

[Karamel, Raghu]

Hi,

I am new to Cassandra and very excited with the technology. I am evaluating it 
and trying to understand the difference between Cassandra and Oracle Coherence. 
Precisely , looking for reasons why would some select Cassandra over Oracle 
Coherence. Does anyone did the exercise of comparing them? Appreciate if you 
can share some information on that.

Regrads
-RK

Re: Questions about using MD5 encryption with SimpleAuthenticator

[Sameer Farooqui]

Opps, my bad... please ignore the email below. It actually works with the
plain text password (I had forgotten to update the passwd.properties file on
one node which was causing the login to fail).

Example of successful login:
ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$
bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u
jdoe -pw 'nosql' -k MDR  Connected to: "Demo_Cluster_beta1" on
ec2-50-19-26-189.compute-1.amazonaws.com/9160
Welcome to the Cassandra CLI.


Would still be nice though to use the bcrypt hash over MD5 for stronger
security.


- Sameer


On Tue, May 17, 2011 at 3:05 PM, Sameer Farooqui wrote:

> Hey Aaron,
>
> Unfortunately it fails with plaintext password also:
>
> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$
> bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u
> jdoe -pw 'nosql' -k MDR  Login failure. Did you specify 'keyspace',
> 'username' and 'password'?
> Welcome to the Cassandra CLI.
> 
>
> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$
> bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u
> jdoe -pw nosql -k MDR
> Login failure. Did you specify 'keyspace', 'username' and 'password'?
> Welcome to the Cassandra CLI.
> 
>
> Regarding the security of MD5, I'm not a security guy either, but it seems
> quiet easy to crack, especially for short passwords.
>
> This website was quickly able to decrypt my MD5 digest (which is honestly
> not very complex) and give me the original plaintext:
> http://md5.noisette.ch/index.php
>
> Longer list of MD5 rainbow table sites:
> http://www.stottmeister.com/blog/2009/04/14/how-to-crack-md5-passwords/
>
> Anyway, any help with the original question of how to input the password
> the the Cassandra-CLI would be much appreciated!
>
>
> - Sameer
>
>
>
> On Tue, May 17, 2011 at 1:03 PM, aaron morton wrote:
>
>> Use the plain text password via the cli, the server will make a hash and
>> compare it to the one in the file.
>>
>> wrt SHA-2 I'm not a security guy but MD5 is probably "good enough" for the
>> problem of storing passwords in plain text in a file.
>>
>> Hope that helps.
>>
>>  -
>> Aaron Morton
>> Freelance Cassandra Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 17 May 2011, at 10:59, Sameer Farooqui wrote:
>>
>> By the way, just noticed a typo in my email below. I'm using the correct
>> keyspace name in all locations on the cluster... however in my examples
>> below, I used MyKeyspace in some spots and MDR in other spots, but in the
>> cluster I'm specifying the same keyspace name everywhere, so that's not the
>> issue.
>>
>> - Sameer
>>
>>
>> On Mon, May 16, 2011 at 3:55 PM, Sameer Farooqui > > wrote:
>>
>>> Hi all,
>>>
>>> We are trying to use MD5 encrypted passwords. Quick question first - Is
>>> SHA-2 supported yet? US-CERT of the U. S. Department of Homeland
>>> Security has said that MD5 "should be considered cryptographically broken
>>> and unsuitable for further use”, and SHA-2 family of hash functions is
>>> recommended.
>>>
>>> The issue I'm seeing is that when I turn on MD5 encryption, I can't log
>>> into the cluster from Cassandra-CLI (I get a login failure).
>>>
>>> The cassandra.in.sh file has been changed as so:
>>>
>>> JVM_OPTS="
>>>
>>> -Dpasswd.properties=/home/ubuntu/apache-cassandra-0.8.0-beta1/conf/passwd.properties
>>> \
>>>
>>> -Daccess.properties=/home/ubuntu/apache-cassandra-0.8.0-beta1/conf/access.properties
>>> \
>>> -Dpasswd.mode=MD5"
>>>
>>>
>>> And I ran this python script to generate a MD5 hash:
>>> ubuntu@darknet:~$ python
>>> Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
>>> [GCC 4.4.5] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>> >>> from hashlib import md5
>>> >>> p = "nosql"
>>> >>> h = md5(p).hexdigest()
>>> >>> print h
>>> 9fa1b39e7eb877367213e6f7e37d0b01
>>>
>>>
>>> Then I updated the passwd.properties file with the new hashed password:
>>> jdoe=9fa1b39e7eb877367213e6f7e37d0b01
>>>
>>>
>>> Also, the access.properties file is properly set so that jdoe has rw
>>> access to the keyspace and CF:
>>> MyKeyspace.=jdoe,jsmith
>>> MyKeyspace.MyCF.=jsmith,jdoe
>>>
>>>
>>> But when I try to connect to the cluster now, I'm getting a login
>>> failure. I have tried a few different ways of connecting:
>>>
>>> Ran this from the Cassandra CLI:
>>> [default@unknown] connect ec2-50-19-26-189.compute-1.amazonaws.com/9160jdoe 
>>> '9fa1b39e7eb877367213e6f7e37d0b01';
>>> Login failure. Did you specify 'keyspace', 'username' and 'password'?
>>>
>>>
>>> Ran these from the Ubuntu CLI:
>>> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$
>>> bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u
>>> jdoe -pw 9fa1b39e7eb877367213e6f7e37d0b01 -k MDR
>>> Login failure. Did you specify 'keyspace', 'username' and 'password'?
>>>
>>>
>>> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$
>>> bin/c

Re: How to configure internode encryption in 0.8.0?

[Sameer Farooqui]

Thanks for the link, Jeremy.

I generated the keystore and truststore for inter-node communication using
the link in the YAML file:
http://download.oracle.com/javase/6/docs/technotes/guides/security/jsse/JSSERefGuide.html#CreateKeystore

Unfortunately,
the default instructions in the above link
used TLS_RSA_WITH_AES_256_CBC_SHA. So, when I start Cassandra now, I get
this error:

ERROR 00:10:38,734 Exception encountered during startup.
java.lang.IllegalArgumentException: Cannot support
TLS_RSA_WITH_AES_256_CBC_SHA   with currently installed providers
at
com.sun.net.ssl.internal.ssl.CipherSuiteList.(CipherSuiteList.j
 ava:79)
at
com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.setEnabledCipherSuit
 es(SSLServerSocketImpl.java:166)
at
org.apache.cassandra.security.SSLFactory.getServerSocket(SSLFactory.j
 ava:55)


The YAML file states that the cipher suite for authentication should
be: TLS_RSA_WITH_AES_128_CBC_SHA.

This is my first time using keytool and I've searched the web to see how I
can change the cipher from AES_256 to AES_128, but haven't found the answer.

Anyone know how to change the cipher to AES_128?

Here are the commands I used to generate the non-working keystore and
truststore:

1) keytool -genkeypair -alias jdoe -keyalg RSA -validity 7 -keystore
.keystore
2) keytool -list -v -keystore .keystore
3) keytool -export -alias jdoe -keystore .keystore -rfc -file jdoe.cer
4) cat jdoe.cer
5) keytool -import -alias jdoecert -file jdoe.cer -keystore .truststore
6) keytool -list -v -keystore .truststore


- Sameer

On Mon, May 16, 2011 at 5:35 PM, Jeremy Hanna wrote:

> Take a look at cassandra.yaml in your 0.8 download at the very bottom.
>  There are docs and examples there.
> e.g.
> http://svn.apache.org/repos/asf/cassandra/tags/cassandra-0.8.0-beta2/conf/cassandra.yaml
>
> On May 16, 2011, at 6:36 PM, Sameer Farooqui wrote:
>
> > I understand that 0.8.0 has configurable internode encryption
> (CASSANDRA-1567, 2152).
> >
> > I haven't been able to find any info on how to configure it though on
> this mailing list or the Datastax website.
> >
> > Can somebody point me towards how to set this up?
> >
> > - Sameer
>
>

Re: Questions about using MD5 encryption with SimpleAuthenticator

[aaron morton]

If you need it create a ticket on 
https://issues.apache.org/jira/browse/CASSANDRA 


Aaron

-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 18 May 2011, at 10:52, Sameer Farooqui wrote:

> Opps, my bad... please ignore the email below. It actually works with the 
> plain text password (I had forgotten to update the passwd.properties file on 
> one node which was causing the login to fail).
> 
> Example of successful login:
> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$ 
> bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u jdoe 
> -pw 'nosql' -k MDR  Connected to: "Demo_Cluster_beta1" on 
> ec2-50-19-26-189.compute-1.amazonaws.com/9160
> Welcome to the Cassandra CLI.
> 
> 
> Would still be nice though to use the bcrypt hash over MD5 for stronger 
> security.
> 
> 
> - Sameer
> 
> 
> On Tue, May 17, 2011 at 3:05 PM, Sameer Farooqui  
> wrote:
> Hey Aaron,
> 
> Unfortunately it fails with plaintext password also:
> 
> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$ 
> bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u jdoe 
> -pw 'nosql' -k MDR  Login failure. Did you specify 'keyspace', 'username' and 
> 'password'?
> Welcome to the Cassandra CLI.
> 
> 
> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassandra-0.8.0-beta1$ 
> bin/cassandra-cli -h ec2-50-19-26-189.compute-1.amazonaws.com -p 9160 -u jdoe 
> -pw nosql -k MDR
> Login failure. Did you specify 'keyspace', 'username' and 'password'?
> Welcome to the Cassandra CLI.
> 
> 
> Regarding the security of MD5, I'm not a security guy either, but it seems 
> quiet easy to crack, especially for short passwords.
> 
> This website was quickly able to decrypt my MD5 digest (which is honestly not 
> very complex) and give me the original plaintext: 
> http://md5.noisette.ch/index.php
> 
> Longer list of MD5 rainbow table sites: 
> http://www.stottmeister.com/blog/2009/04/14/how-to-crack-md5-passwords/
> 
> Anyway, any help with the original question of how to input the password the 
> the Cassandra-CLI would be much appreciated!
> 
> 
> - Sameer
> 
> 
> 
> On Tue, May 17, 2011 at 1:03 PM, aaron morton  wrote:
> Use the plain text password via the cli, the server will make a hash and 
> compare it to the one in the file. 
> 
> wrt SHA-2 I'm not a security guy but MD5 is probably "good enough" for the 
> problem of storing passwords in plain text in a file. 
> 
> Hope that helps. 
> 
> -
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 17 May 2011, at 10:59, Sameer Farooqui wrote:
> 
>> By the way, just noticed a typo in my email below. I'm using the correct 
>> keyspace name in all locations on the cluster... however in my examples 
>> below, I used MyKeyspace in some spots and MDR in other spots, but in the 
>> cluster I'm specifying the same keyspace name everywhere, so that's not the 
>> issue.
>> 
>> - Sameer
>> 
>> 
>> On Mon, May 16, 2011 at 3:55 PM, Sameer Farooqui  
>> wrote:
>> Hi all,
>> 
>> We are trying to use MD5 encrypted passwords. Quick question first - Is 
>> SHA-2 supported yet? US-CERT of the U. S. Department of Homeland Security 
>> has said that MD5 "should be considered cryptographically broken and 
>> unsuitable for further use”, and SHA-2 family of hash functions is 
>> recommended.
>> 
>> The issue I'm seeing is that when I turn on MD5 encryption, I can't log into 
>> the cluster from Cassandra-CLI (I get a login failure).
>> 
>> The cassandra.in.sh file has been changed as so:
>> 
>> JVM_OPTS="
>> 
>> -Dpasswd.properties=/home/ubuntu/apache-cassandra-0.8.0-beta1/conf/passwd.properties
>>  \
>> 
>> -Daccess.properties=/home/ubuntu/apache-cassandra-0.8.0-beta1/conf/access.properties
>>  \
>> -Dpasswd.mode=MD5"
>> 
>> 
>> And I ran this python script to generate a MD5 hash:
>> ubuntu@darknet:~$ python
>> Python 2.6.6 (r266:84292, Sep 15 2010, 15:52:39)
>> [GCC 4.4.5] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> from hashlib import md5
>> >>> p = "nosql"
>> >>> h = md5(p).hexdigest()
>> >>> print h
>> 9fa1b39e7eb877367213e6f7e37d0b01
>> 
>> 
>> Then I updated the passwd.properties file with the new hashed password:
>> jdoe=9fa1b39e7eb877367213e6f7e37d0b01
>> 
>> 
>> Also, the access.properties file is properly set so that jdoe has rw access 
>> to the keyspace and CF:
>> MyKeyspace.=jdoe,jsmith
>> MyKeyspace.MyCF.=jsmith,jdoe
>> 
>> 
>> But when I try to connect to the cluster now, I'm getting a login failure. I 
>> have tried a few different ways of connecting:
>> 
>> Ran this from the Cassandra CLI: 
>> [default@unknown] connect ec2-50-19-26-189.compute-1.amazonaws.com/9160 jdoe 
>> '9fa1b39e7eb877367213e6f7e37d0b01';
>> Login failure. Did you specify 'keyspace', 'username' and 'password'?
>> 
>> 
>> Ran these from the Ubuntu CLI:
>> ubuntu@domU-12-31-39-0C-D9-13:~/apache-cassand

Gossiper question

[Cassa L]

Hi,
  I have 9 node cluster with RF-3 and using Cassandra0.70/Hector26. Recently
we are seeing lot of "UnavailableException" at the client side. Whenever
this happens, I found following pattern in Cassandra node's log file at that
given time,

* INFO [ScheduledTasks:1] 2011-05-13 02:59:55,365 Gossiper.java (line 195)
InetAddress /**.**.***.54 is now dead.*
* INFO [ScheduledTasks:1] 2011-05-13 02:59:57,369 Gossiper.java (line 195)
InetAddress /.**.***.**59 is now dead.*
 INFO [HintedHandoff:1] 2011-05-13 03:00:04,706 HintedHandOffManager.java
(line 192) Started hinted handoff for endpoint /***.**.***.*54
* INFO [GossipStage:1] 2011-05-13 03:00:04,706 Gossiper.java (line 569)
InetAddress /.**.*.54 is now UP*
 INFO [HintedHandoff:1] 2011-05-13 03:00:04,706 HintedHandOffManager.java
(line 248) Finished hinted handoff of 0 rows to endpoint /***.**..54
 INFO [HintedHandoff:1] 2011-05-13 03:00:20,601 HintedHandOffManager.java
(line 192) Started hinted handoff for endpoint /***.**..59
* INFO [GossipStage:1] 2011-05-13 03:00:20,601 Gossiper.java (line 569)
InetAddress /.**.*.59 is now UP*

The exception occurred at "2011-05-13 03:00:00,664". I am wondering what why
this dead/up pattern is occurring at Gossip.

Thanks in advance,
Cassa L.

Re: Gossiper question

[Ali Ahsan]

On 05/18/2011 09:28 AM, Cassa L wrote:

Hi,
  I have 9 node cluster with RF-3 and using Cassandra0.70/Hector26. 
Recently we are seeing lot of "UnavailableException" at the client 
side. Whenever this happens, I found following pattern in Cassandra 
node's log file at that given time,


* INFO [ScheduledTasks:1] 2011-05-13 02:59:55,365 Gossiper.java (line 
195) InetAddress /**.**.***.54 is now dead.*
* INFO [ScheduledTasks:1] 2011-05-13 02:59:57,369 Gossiper.java (line 
195) InetAddress /.**.***.**59 is now dead.*
 INFO [HintedHandoff:1] 2011-05-13 03:00:04,706 
HintedHandOffManager.java (line 192) Started hinted handoff for 
endpoint /***.**.***.*54
* INFO [GossipStage:1] 2011-05-13 03:00:04,706 Gossiper.java (line 
569) InetAddress /.**.*.54 is now UP*
 INFO [HintedHandoff:1] 2011-05-13 03:00:04,706 
HintedHandOffManager.java (line 248) Finished hinted handoff of 0 rows 
to endpoint /***.**..54
 INFO [HintedHandoff:1] 2011-05-13 03:00:20,601 
HintedHandOffManager.java (line 192) Started hinted handoff for 
endpoint /***.**..59
* INFO [GossipStage:1] 2011-05-13 03:00:20,601 Gossiper.java (line 
569) InetAddress /.**.*.59 is now UP*


The exception occurred at "2011-05-13 03:00:00,664". I am wondering 
what why this dead/up pattern is occurring at Gossip.


Thanks in advance,
Cassa L.


I am not sure do check system Load when you see this happening.If load 
is to High then it means CPU was not able to give time to Network I/O.


--
S.Ali Ahsan