Re: SIGSEGV during compaction?

[Sylvain Lebresne]

Are you using current trunk ? Or 0.8 ?

Because if on trunk, a SIGSEGV could also be due to CASSANDRA-2521,
if we happen to force the unmapping of a file but tries to access it afterwards
(which shouldn't happen but ...).

--
Sylvain

On Thu, Sep 8, 2011 at 7:36 AM, Yang  wrote:
> h, all other things remaining the same, I put jna.jar into classpath,
> now it successfully completed a compaction without problems
>
> On Wed, Sep 7, 2011 at 10:06 PM, Yang  wrote:
>> thanks Jonathan.
>>
>> I tried openJdk too, same , filed bug to both Oracle and openJdk
>>
>>
>> tried -XX:-UseCompressedOops , same SEGV
>>
>> Oracle bug site asks "does it appear with -server and -Xint", I tried
>> these options, so far no SEGV yet, maybe slower, but haven't measured
>> exactly
>>
>>
>>
>> On Wed, Sep 7, 2011 at 8:56 PM, Jonathan Ellis  wrote:
>>> You should report a bug to Oracle.
>>>
>>> In the meantime you could try turning off compressed oops -- that's
>>> been a source of a lot of GC bugs in the past.
>>>
>>> On Wed, Sep 7, 2011 at 8:22 PM, Yang  wrote:
 some info in the debug file that JVM exported:

 #
 # A fatal error has been detected by the Java Runtime Environment:
 #
 #  SIGSEGV (0xb) at pc=0x2b37cbfa, pid=7236, tid=1179806016
 #
 # JRE version: 6.0_27-b07
 # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.2-b06 mixed mode
 linux-amd64 compressed oops)
 # Problematic frame:
 # J  
 com.cgm.whisky.filter.WithinLimitIpFrequencyCap.isValid(Lorg/apache/avro/specific/SpecificRecord;)Lcom/cgm/whisky/EventsFilter$ValidityCode;
 #
 # If you would like to submit a bug report, please visit:
 #   http://java.sun.com/webapps/bugreport/crash.jsp
 #

 ---  T H R E A D  ---

 Current thread (0x2aaab80e2800):  JavaThread "pool-3-thread-8"
 [_thread_in_Java, id=7669,
 stack(0x46426000,0x46527000)]

 siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR),
 si_addr=0x2aaabc00

 Registers:
 RAX=0x0007914355e8, RBX=0x058a,
 RCX=0x000791461b38, RDX=0x
 RSP=0x465259f0, RBP=0xf222b894,
 RSI=0x000791433f20, RDI=0x2b37ca60
 R8 =0xd0931f61, R9 =0xf2286ab2,
 R10=0x, R11=0x2aaabc00
 R12=0x, R13=0x465259f0,
 R14=0x0002, R15=0x2aaab80e2800
 RIP=0x2b37cbfa, EFLAGS=0x00010202,
 CSGSFS=0x0133, ERR=0x0004
  TRAPNO=0x000e

 Top of Stack: (sp=0x465259f0)
 0x465259f0:   00068a828dc8 000791433f20
 0x46525a00:   00079145ee60 0589058a


 On Wed, Sep 7, 2011 at 6:21 PM, Yang  wrote:
> I started compaction using nodetool,
> then always reproducibly, I get a SEGV in a code that I added to the
> Cassandra code, which simply calls get_slice().
>
> have you seen SEGV associated with compaction? anyone could suggest a
> route on how to debug this?
>
> I filed a bug on sun website, right now the only possible approach I
> can try is to use another JDK
>
>
> Thanks
> Yang
>

>>>
>>>
>>>
>>> --
>>> Jonathan Ellis
>>> Project Chair, Apache Cassandra
>>> co-founder of DataStax, the source for professional Cassandra support
>>> http://www.datastax.com
>>>
>>
>

row key as long type

[Thamizh]

Hi All,

Is there a way to store number(longtype) as row key in Cassadra?

I wanted to execute range query based on row key value. e.g $list 
info[12345:]; . It should list all the rowkeys which are >= 12345. 


Is there a way accompolish this in cassandra? Secondary index does 
not helped me. So I am trying to store column value 'ip' as rowkey here.

data model:

create keyspace ipinfo with placement_strategy =
'org.apache.cassandra.locator.SimpleStrategy' and strategy_options =  
[{replication_factor:1}];

use rng;

create column family info with comparator = AsciiType
and key_validation_class = UTF8Type
and column_metadata =
[{
column_name : domain,
validation_class : UTF8Type,
index_type : 0,
index_name : domain_idx},
{
column_name : ip,
validation_class : LongType,
index_type : 0,
index_name : ip_idx
}];


Regards,

  Thamizhannal 

[RELEASE] Apache Cassandra 0.8.5 released

[Sylvain Lebresne]

The Cassandra team is pleased to announce the release of Apache Cassandra
version 0.8.5.

Cassandra is a highly scalable second-generation distributed database,
bringing together Dynamo's fully distributed design and Bigtable's
ColumnFamily-based data model. You can read more here:

 http://cassandra.apache.org/

Downloads of source and binary distributions are listed in our download
section:

 http://cassandra.apache.org/download/

This version is a maintenance/bug fix release[1]. Please pay attention to the
release notes[2] before upgrading and let us know[3] if you were to encounter
any problem.

Have fun!


[1]: http://goo.gl/A5YmF (CHANGES.txt)
[2]: http://goo.gl/J5Iix (NEWS.txt)
[3]: https://issues.apache.org/jira/browse/CASSANDRA

Re: [RELEASE] Apache Cassandra 0.8.5 released

[Roshan Dawrani]

Hi,

Looking forward to having 0.8.5 artifacts available from maven repo too.

Cheers.

On Thu, Sep 8, 2011 at 5:45 PM, Sylvain Lebresne wrote:

> The Cassandra team is pleased to announce the release of Apache Cassandra
> version 0.8.5.
>
> Cassandra is a highly scalable second-generation distributed database,
> bringing together Dynamo's fully distributed design and Bigtable's
> ColumnFamily-based data model. You can read more here:
>
>  http://cassandra.apache.org/
>
> Downloads of source and binary distributions are listed in our download
> section:
>
>  http://cassandra.apache.org/download/
>
> This version is a maintenance/bug fix release[1]. Please pay attention to
> the
> release notes[2] before upgrading and let us know[3] if you were to
> encounter
> any problem.
>
> Have fun!
>
>
> [1]: http://goo.gl/A5YmF (CHANGES.txt)
> [2]: http://goo.gl/J5Iix (NEWS.txt)
> [3]: https://issues.apache.org/jira/browse/CASSANDRA
>



-- 
Roshan
Blog: http://roshandawrani.wordpress.com/
Twitter: @roshandawrani 
Skype: roshandawrani

Re: [RELEASE] Apache Cassandra 0.8.5 released

[Dikang Gu]

Congratulations!

On Thu, Sep 8, 2011 at 8:15 PM, Sylvain Lebresne wrote:

> The Cassandra team is pleased to announce the release of Apache Cassandra
> version 0.8.5.
>
> Cassandra is a highly scalable second-generation distributed database,
> bringing together Dynamo's fully distributed design and Bigtable's
> ColumnFamily-based data model. You can read more here:
>
>  http://cassandra.apache.org/
>
> Downloads of source and binary distributions are listed in our download
> section:
>
>  http://cassandra.apache.org/download/
>
> This version is a maintenance/bug fix release[1]. Please pay attention to
> the
> release notes[2] before upgrading and let us know[3] if you were to
> encounter
> any problem.
>
> Have fun!
>
>
> [1]: http://goo.gl/A5YmF (CHANGES.txt)
> [2]: http://goo.gl/J5Iix (NEWS.txt)
> [3]: https://issues.apache.org/jira/browse/CASSANDRA
>



-- 
Dikang Gu

0086 - 18611140205

Exception encountered during startup.

[Dikang Gu]

I have a 4 cassandra 0.8.1 nodes in the cluster, one node crashes and I'm
trying to restart it.

But I encounter the following errors during the startup, is this a known
bug?


DEBUG [main] 2011-09-08 20:26:17,959 Table.java (line 305) Initializing
system.NodeIdInfo
DEBUG [main] 2011-09-08 20:26:17,963 ColumnFamilyStore.java (line 264)
Starting CFS NodeIdInfo
DEBUG [main] 2011-09-08 20:26:17,967 AutoSavingCache.java (line 175)
KeyCache capacity for NodeIdInfo is 1
ERROR [main] 2011-09-08 20:26:17,969 AbstractCassandraDaemon.java (line 332)
Exception encountered during startup.
java.lang.RuntimeException: javax.management.InstanceAlreadyExistsException:
org.apache.cassandra.db:type=ColumnFamilies,keyspace=system,columnfamily=NodeIdInfo
at
org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:315)
at
org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:455)
at
org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:436)
at org.apache.cassandra.db.Table.initCf(Table.java:369)
at org.apache.cassandra.db.Table.(Table.java:306)
at org.apache.cassandra.db.Table.open(Table.java:111)
at org.apache.cassandra.db.SystemTable.checkHealth(SystemTable.java:212)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:128)
at
org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:315)
at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:80)
Caused by: javax.management.InstanceAlreadyExistsException:
org.apache.cassandra.db:type=ColumnFamilies,keyspace=system,columnfamily=NodeIdInfo
at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:453)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1484)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:963)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
at
org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:311)
... 9 more

Thanks.

-- 
Dikang Gu

0086 - 18611140205

Re: [RELEASE] Apache Cassandra 0.8.5 released

[Helder Oliveira]

Congrats guys!

On Sep 8, 2011, at 1:37 PM, Dikang Gu wrote:

> Congratulations! 
> 
> On Thu, Sep 8, 2011 at 8:15 PM, Sylvain Lebresne  wrote:
> The Cassandra team is pleased to announce the release of Apache Cassandra
> version 0.8.5.
> 
> Cassandra is a highly scalable second-generation distributed database,
> bringing together Dynamo's fully distributed design and Bigtable's
> ColumnFamily-based data model. You can read more here:
> 
>  http://cassandra.apache.org/
> 
> Downloads of source and binary distributions are listed in our download
> section:
> 
>  http://cassandra.apache.org/download/
> 
> This version is a maintenance/bug fix release[1]. Please pay attention to the
> release notes[2] before upgrading and let us know[3] if you were to encounter
> any problem.
> 
> Have fun!
> 
> 
> [1]: http://goo.gl/A5YmF (CHANGES.txt)
> [2]: http://goo.gl/J5Iix (NEWS.txt)
> [3]: https://issues.apache.org/jira/browse/CASSANDRA
> 
> 
> 
> -- 
> Dikang Gu
> 
> 0086 - 18611140205
> 

Re: Exception encountered during startup.

[Jonathan Ellis]

See this thread:
http://mail-archives.apache.org/mod_mbox/cassandra-user/201109.mbox/%3ccaldd-zhuflt3urdsk0ahsmqdj-n1keyxonn4rgzjjz13cag...@mail.gmail.com%3E

On Thu, Sep 8, 2011 at 7:36 AM, Dikang Gu  wrote:
> I have a 4 cassandra 0.8.1 nodes in the cluster, one node crashes and I'm
> trying to restart it.
> But I encounter the following errors during the startup, is this a known
> bug?
>
> DEBUG [main] 2011-09-08 20:26:17,959 Table.java (line 305) Initializing
> system.NodeIdInfo
> DEBUG [main] 2011-09-08 20:26:17,963 ColumnFamilyStore.java (line 264)
> Starting CFS NodeIdInfo
> DEBUG [main] 2011-09-08 20:26:17,967 AutoSavingCache.java (line 175)
> KeyCache capacity for NodeIdInfo is 1
> ERROR [main] 2011-09-08 20:26:17,969 AbstractCassandraDaemon.java (line 332)
> Exception encountered during startup.
> java.lang.RuntimeException: javax.management.InstanceAlreadyExistsException:
> org.apache.cassandra.db:type=ColumnFamilies,keyspace=system,columnfamily=NodeIdInfo
> at
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:315)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:455)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:436)
> at org.apache.cassandra.db.Table.initCf(Table.java:369)
> at org.apache.cassandra.db.Table.(Table.java:306)
> at org.apache.cassandra.db.Table.open(Table.java:111)
> at org.apache.cassandra.db.SystemTable.checkHealth(SystemTable.java:212)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:128)
> at
> org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:315)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:80)
> Caused by: javax.management.InstanceAlreadyExistsException:
> org.apache.cassandra.db:type=ColumnFamilies,keyspace=system,columnfamily=NodeIdInfo
> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:453)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1484)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:963)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
> at
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:311)
> ... 9 more
> Thanks.
> --
> Dikang Gu
> 0086 - 18611140205
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: Secondary index update issue

[Jonathan Ellis]

If the queries are working successfully, I'm afraid I don't understand
the problem.

On Wed, Sep 7, 2011 at 12:48 PM, Thamizh  wrote:
>
> Hi,
>
> Here is my sample row. I wanted to execute below 2 queries. CF:ip
> 1. get ip where ip=19268678;
> 2. get ip where domain='google.com';
>
> Here both ip & domain has secondary indexes.
>
> RowKey: 19268678
> => (column=country, value=in, timestamp=1315398995980)
> => (column=domain, value=google.com, timestamp=1315398995980)
> => (column=ip, value=19268678, timestamp=1315398995980)
>
> What encoding format should I use?. Here I defined ip as LongType & domain as 
> utf8.
>
> When I upload data using above mentioned index type on CF definition then, 
> above queries are working sucessfully. Why?
>
>
> Regards,
> Thamizhannal P
>
> --- On Wed, 7/9/11, Jonathan Ellis  wrote:
>
> From: Jonathan Ellis 
> Subject: Re: Secondary index update issue
> To: user@cassandra.apache.org
> Date: Wednesday, 7 September, 2011, 7:29 PM
>
> My guess would be you're querying using a different encoding and there
> really is no data for your query as given.  Hard to say without more
> details.
>
> On Wed, Sep 7, 2011 at 8:13 AM, Thamizh  wrote:
> >
> > Hi All,
> >
> > I have created KS & CF using cassandra-0.7.8 and inserted some rows and 
> > column values(around 1000 rows). Later, I wanted to index 2 column values. 
> > So, I issued 'update column family..' command. After, when I query based on 
> > indexed value it says "Row does not found". After indexing 1. Issued 
> > nodetool flush 2.restarted Cassandra once. Though it is same. But, I could 
> > see some XXX-Index.db file on cassandra data directory. What am I missing?
> >
> > Here are CF details,
> >
> > create column family ipinfo with column_type=Standard and
> > default_validation_class =   UTF8Type and comparator=UTF8Type and
> > keys_cached=25000 and rows_cached=5000 and column_metadata=[
> > { column_name : country, validation_class : UTF8Type},
> > { column_name : ip, validation_class : LongType},
> > { column_name : domain, validation_class : UTF8Type },
> > ];
> >
> > update column family ip with column_type=Standard and
> > default_validation_class = UTF8Type and comparator=UTF8Type and
> > keys_cached=25000 and rows_cached=5000 and column_metadata=[
> > {column_name : country, validation_class : UTF8Type },
> > {column_name : domain, validation_class : UTF8Type, index_type: KEYS},
> > {column_name : ip, validation_class : LongType, index_type: KEYS}
> > ];
> >
> > Any suggestions would be appreciated
> >
> > Regards,
> > Thamizhannal P
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com



--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: row key as long type

[amulya rattan]

Row key can certainly be of type long..you'd just have to set
key_validataion_class to be LongType.

However, doing list on column family would throw an error..please look at
http://wiki.apache.org/cassandra/FAQ#a_long_is_exactly_8_bytes

On Thu, Sep 8, 2011 at 8:14 AM, Thamizh  wrote:

> Hi All,
>
> Is there a way to store number(longtype) as row key in Cassadra?
>
> I wanted to execute range query based on row key value. e.g $list
> info[12345:]; . It should list all the rowkeys which are >= 12345.
>
>
> Is there a way accompolish this in cassandra? Secondary index does not
> helped me. So I am trying to store column value 'ip' as rowkey here.
>
> data model:
>
> create keyspace ipinfo with placement_strategy =
> 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options =
> [{replication_factor:1}];
>
> use rng;
>
> create column family info with comparator = AsciiType
> and key_validation_class = UTF8Type
> and column_metadata =
> [{
> column_name : domain,
> validation_class : UTF8Type,
> index_type : 0,
> index_name : domain_idx},
> {
> column_name : ip,
> validation_class : LongType,
> index_type : 0,
> index_name : ip_idx
> }];
>
> Regards,
> Thamizhannal
>

Re: [RELEASE] Apache Cassandra 0.8.5 released

[Roshan Dawrani]

Hi,

The artefacts at
http://mvnrepository.com/artifact/org.apache.cassandra/cassandra-all still
do not reflect 0.8.5.

Does the availability at maven lag behind by much?

Cheers.

On Thu, Sep 8, 2011 at 5:45 PM, Sylvain Lebresne wrote:

> The Cassandra team is pleased to announce the release of Apache Cassandra
> version 0.8.5.
>
> Cassandra is a highly scalable second-generation distributed database,
> bringing together Dynamo's fully distributed design and Bigtable's
> ColumnFamily-based data model. You can read more here:
>
>  http://cassandra.apache.org/
>
> Downloads of source and binary distributions are listed in our download
> section:
>
>  http://cassandra.apache.org/download/
>
> This version is a maintenance/bug fix release[1]. Please pay attention to
> the
> release notes[2] before upgrading and let us know[3] if you were to
> encounter
> any problem.
>
> Have fun!
>
>
> [1]: http://goo.gl/A5YmF (CHANGES.txt)
> [2]: http://goo.gl/J5Iix (NEWS.txt)
> [3]: https://issues.apache.org/jira/browse/CASSANDRA
>



-- 
Roshan
Blog: http://roshandawrani.wordpress.com/
Twitter: @roshandawrani 
Skype: roshandawrani

Re: row key as long type

[Jonathan Ellis]

List should work fine on any schema, including long keys.

On Thu, Sep 8, 2011 at 8:23 AM, amulya rattan  wrote:

> Row key can certainly be of type long..you'd just have to set
> key_validataion_class to be LongType.
>
> However, doing list on column family would throw an error..please look at
> http://wiki.apache.org/cassandra/FAQ#a_long_is_exactly_8_bytes
>
>
> On Thu, Sep 8, 2011 at 8:14 AM, Thamizh  wrote:
>
>> Hi All,
>>
>> Is there a way to store number(longtype) as row key in Cassadra?
>>
>> I wanted to execute range query based on row key value. e.g $list
>> info[12345:]; . It should list all the rowkeys which are >= 12345.
>>
>>
>> Is there a way accompolish this in cassandra? Secondary index does not
>> helped me. So I am trying to store column value 'ip' as rowkey here.
>>
>> data model:
>>
>> create keyspace ipinfo with placement_strategy =
>> 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options =
>> [{replication_factor:1}];
>>
>> use rng;
>>
>> create column family info with comparator = AsciiType
>>
>>
>> and key_validation_class = UTF8Type
>> and column_metadata =
>> [{
>> column_name : domain,
>> validation_class : UTF8Type,
>> index_type : 0,
>> index_name : domain_idx},
>> {
>> column_name : ip,
>>
>>
>> validation_class : LongType,
>> index_type : 0,
>> index_name : ip_idx
>> }];
>>
>> Regards,
>> Thamizhannal
>>
>
>


-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: [RELEASE] Apache Cassandra 0.8.5 released

[Stephen Connolly]

can take up to 12 hours for the sync to central

- Stephen

---
Sent from my Android phone, so random spelling mistakes, random nonsense
words and other nonsense are a direct result of using swype to type on the
screen
On 8 Sep 2011 06:40, "Roshan Dawrani"  wrote:
> Hi,
>
> The artefacts at
> http://mvnrepository.com/artifact/org.apache.cassandra/cassandra-all still
> do not reflect 0.8.5.
>
> Does the availability at maven lag behind by much?
>
> Cheers.
>
> On Thu, Sep 8, 2011 at 5:45 PM, Sylvain Lebresne wrote:
>
>> The Cassandra team is pleased to announce the release of Apache Cassandra
>> version 0.8.5.
>>
>> Cassandra is a highly scalable second-generation distributed database,
>> bringing together Dynamo's fully distributed design and Bigtable's
>> ColumnFamily-based data model. You can read more here:
>>
>> http://cassandra.apache.org/
>>
>> Downloads of source and binary distributions are listed in our download
>> section:
>>
>> http://cassandra.apache.org/download/
>>
>> This version is a maintenance/bug fix release[1]. Please pay attention to
>> the
>> release notes[2] before upgrading and let us know[3] if you were to
>> encounter
>> any problem.
>>
>> Have fun!
>>
>>
>> [1]: http://goo.gl/A5YmF (CHANGES.txt)
>> [2]: http://goo.gl/J5Iix (NEWS.txt)
>> [3]: https://issues.apache.org/jira/browse/CASSANDRA
>>
>
>
>
> --
> Roshan
> Blog: http://roshandawrani.wordpress.com/
> Twitter: @roshandawrani 
> Skype: roshandawrani

Read times on Cassandra

[Ben Ashton]

Hi Guys,

I have our multi DC setup running on EC2.

Latency on the connections between the servers is <2ms, monitored
writes are ~3-4ms, reads are 50 to 2200 ms, normally ~80ms

Is this normal? I know there are some optimisation java guys can do
with hector, but is there anything I can do on a systems level to
speed up reads.

Thanks,

Ben

Re: Read times on Cassandra

[Jonathan Ellis]

What is your bottleneck?

spyced.blogspot.com/2010/01/linux-performance-basics.html

On Thu, Sep 8, 2011 at 10:14 AM, Ben Ashton  wrote:
> Hi Guys,
>
> I have our multi DC setup running on EC2.
>
> Latency on the connections between the servers is <2ms, monitored
> writes are ~3-4ms, reads are 50 to 2200 ms, normally ~80ms
>
> Is this normal? I know there are some optimisation java guys can do
> with hector, but is there anything I can do on a systems level to
> speed up reads.
>
> Thanks,
>
> Ben
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: SIGSEGV during compaction?

[Yang]

this is trunk.


sorry I did more tests, the -XX:-UseCompressedOops suggested by
Jonathan actually DOES solve the problem.  my previous tries possibly
used the wrong scripts.

Thanks guys

Yang

On Thu, Sep 8, 2011 at 12:07 AM, Sylvain Lebresne  wrote:
> Are you using current trunk ? Or 0.8 ?
>
> Because if on trunk, a SIGSEGV could also be due to CASSANDRA-2521,
> if we happen to force the unmapping of a file but tries to access it 
> afterwards
> (which shouldn't happen but ...).
>
> --
> Sylvain
>
> On Thu, Sep 8, 2011 at 7:36 AM, Yang  wrote:
>> h, all other things remaining the same, I put jna.jar into classpath,
>> now it successfully completed a compaction without problems
>>
>> On Wed, Sep 7, 2011 at 10:06 PM, Yang  wrote:
>>> thanks Jonathan.
>>>
>>> I tried openJdk too, same , filed bug to both Oracle and openJdk
>>>
>>>
>>> tried -XX:-UseCompressedOops , same SEGV
>>>
>>> Oracle bug site asks "does it appear with -server and -Xint", I tried
>>> these options, so far no SEGV yet, maybe slower, but haven't measured
>>> exactly
>>>
>>>
>>>
>>> On Wed, Sep 7, 2011 at 8:56 PM, Jonathan Ellis  wrote:
 You should report a bug to Oracle.

 In the meantime you could try turning off compressed oops -- that's
 been a source of a lot of GC bugs in the past.

 On Wed, Sep 7, 2011 at 8:22 PM, Yang  wrote:
> some info in the debug file that JVM exported:
>
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x2b37cbfa, pid=7236, tid=1179806016
> #
> # JRE version: 6.0_27-b07
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.2-b06 mixed mode
> linux-amd64 compressed oops)
> # Problematic frame:
> # J  
> com.cgm.whisky.filter.WithinLimitIpFrequencyCap.isValid(Lorg/apache/avro/specific/SpecificRecord;)Lcom/cgm/whisky/EventsFilter$ValidityCode;
> #
> # If you would like to submit a bug report, please visit:
> #   http://java.sun.com/webapps/bugreport/crash.jsp
> #
>
> ---  T H R E A D  ---
>
> Current thread (0x2aaab80e2800):  JavaThread "pool-3-thread-8"
> [_thread_in_Java, id=7669,
> stack(0x46426000,0x46527000)]
>
> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR),
> si_addr=0x2aaabc00
>
> Registers:
> RAX=0x0007914355e8, RBX=0x058a,
> RCX=0x000791461b38, RDX=0x
> RSP=0x465259f0, RBP=0xf222b894,
> RSI=0x000791433f20, RDI=0x2b37ca60
> R8 =0xd0931f61, R9 =0xf2286ab2,
> R10=0x, R11=0x2aaabc00
> R12=0x, R13=0x465259f0,
> R14=0x0002, R15=0x2aaab80e2800
> RIP=0x2b37cbfa, EFLAGS=0x00010202,
> CSGSFS=0x0133, ERR=0x0004
>  TRAPNO=0x000e
>
> Top of Stack: (sp=0x465259f0)
> 0x465259f0:   00068a828dc8 000791433f20
> 0x46525a00:   00079145ee60 0589058a
>
>
> On Wed, Sep 7, 2011 at 6:21 PM, Yang  wrote:
>> I started compaction using nodetool,
>> then always reproducibly, I get a SEGV in a code that I added to the
>> Cassandra code, which simply calls get_slice().
>>
>> have you seen SEGV associated with compaction? anyone could suggest a
>> route on how to debug this?
>>
>> I filed a bug on sun website, right now the only possible approach I
>> can try is to use another JDK
>>
>>
>> Thanks
>> Yang
>>
>



 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder of DataStax, the source for professional Cassandra support
 http://www.datastax.com

>>>
>>
>

Re: weird freeze

[Yang]

ok, found this is likely due to GC... I'm seeing full GC that runs
20seconds without actually removing anything:


35240.526: [Full GC [PSYoungGen: 1760704K->1668729K(1848128K)]
[PSOldGen: 4095999K->4095999K(4096000K)] 5856703K->5764729K(5944128K)
[PSPermGen: 24885K->24871K(25152K)], 20.6780790 secs] [Times:
user=20.60 sys=0.00, real=20.68 secs]


I'm trying to do a MAT anaysis now

On Wed, Sep 7, 2011 at 11:01 AM, yang yang  wrote:
> my cassandra server (from github source head, plus I added some code that just
> calls thrift.CassandraServer.batch_mutate() in the same JVM )   runs fine 
> under
> heavy load for about 5 hours, then it froze,
>
> when I checked the jstack, all the mutation stages are blocked on
> Table.switchlock.ReadLock().lock(),
> and the corresponding writeLock is being held by Table.maybeSwitchMemlock(),
> which is blocked on CommitLog.instance.getContext(), which is blocked on
> get()'ting from a future.
>
>
> "COMMIT-LOG-WRITER" prio=10 tid=0x2aaab4a6c000 nid=0x66df waiting on
> condition [0x428d6000]
>   java.lang.Thread.State: WAITING (parking)
>        at sun.misc.Unsafe.$$YJP$$park(Native Method)
>        - parking to wait for  <0x0007f72bb5f0> (a
> java.util.concurrent.FutureTask$Sync)
>        at sun.misc.Unsafe.park(Unsafe.java)
>        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
>        at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
>
>        at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
>
>        at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
>
>        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
>        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>        at
> org.apache.cassandra.db.commitlog.CommitLog.getContext(CommitLog.java:388)
>        at
> org.apache.cassandra.db.ColumnFamilyStore.maybeSwitchMemtable(ColumnFamilyStore.java:661)
>
>        at
> org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:733)
>        at
> org.apache.cassandra.db.commitlog.CommitLog.createNewSegment(CommitLog.java:575)
>        at
> org.apache.cassandra.db.commitlog.CommitLog.access$300(CommitLog.java:82)
>        at
> org.apache.cassandra.db.commitlog.CommitLog$LogRecordAdder.run(CommitLog.java:598)
>
>        at
> org.apache.cassandra.db.commitlog.PeriodicCommitLogExecutorService$1.runMayThrow(PeriodicCommitLogExecutorService.java:49)
>
>        at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>        at java.lang.Thread.run(Thread.java:662)
>
>
>
> I tried to search for the thread that does the getting from future in 
> commitLog,
> but it's not present. anyone has an idea why the 
> CommitLog.instance.getContext()
> is not returning? the getContext() Callable is not yet scheduled by the
> CommitLog executor?
>
>
> Thanks a lot
> Yang

Re: [RELEASE] Apache Cassandra 0.8.5 released

[Anthony Ikeda]

Cool, all those fixes will be going into our prod env! Congrats!

Anthony

On Thu, Sep 8, 2011 at 7:51 AM, Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:

> can take up to 12 hours for the sync to central
>
> - Stephen
>
> ---
> Sent from my Android phone, so random spelling mistakes, random nonsense
> words and other nonsense are a direct result of using swype to type on the
> screen
> On 8 Sep 2011 06:40, "Roshan Dawrani"  wrote:
> > Hi,
> >
> > The artefacts at
> > http://mvnrepository.com/artifact/org.apache.cassandra/cassandra-allstill
> > do not reflect 0.8.5.
> >
> > Does the availability at maven lag behind by much?
> >
> > Cheers.
> >
> > On Thu, Sep 8, 2011 at 5:45 PM, Sylvain Lebresne  >wrote:
> >
> >> The Cassandra team is pleased to announce the release of Apache
> Cassandra
> >> version 0.8.5.
> >>
> >> Cassandra is a highly scalable second-generation distributed database,
> >> bringing together Dynamo's fully distributed design and Bigtable's
> >> ColumnFamily-based data model. You can read more here:
> >>
> >> http://cassandra.apache.org/
> >>
> >> Downloads of source and binary distributions are listed in our download
> >> section:
> >>
> >> http://cassandra.apache.org/download/
> >>
> >> This version is a maintenance/bug fix release[1]. Please pay attention
> to
> >> the
> >> release notes[2] before upgrading and let us know[3] if you were to
> >> encounter
> >> any problem.
> >>
> >> Have fun!
> >>
> >>
> >> [1]: http://goo.gl/A5YmF (CHANGES.txt)
> >> [2]: http://goo.gl/J5Iix (NEWS.txt)
> >> [3]: https://issues.apache.org/jira/browse/CASSANDRA
> >>
> >
> >
> >
> > --
> > Roshan
> > Blog: http://roshandawrani.wordpress.com/
> > Twitter: @roshandawrani 
> > Skype: roshandawrani
>

requests should be disabled when doing compaction?

[Yang]

I did a nodetool compact while the Cassandra server was serving
requests, it almost never finishes, and the service latency becomes
very long
then I terminated the traffic,  compaction finishes shortly after.

should I disable client requests to a node being compacted while I
carry out the compaction?

Thanks
Yang

Re: requests should be disabled when doing compaction?

[Jonathan Ellis]

(1) You should plan enough capacity that you can handle normal
compaction load while serving requests
(1a) You can control how aggressive compaction is using the
compaction_throughput_mb_per_sec setting

(2) It is not recommended to force compaction of everything using
nodetool compact

On Thu, Sep 8, 2011 at 2:34 PM, Yang  wrote:
> I did a nodetool compact while the Cassandra server was serving
> requests, it almost never finishes, and the service latency becomes
> very long
> then I terminated the traffic,  compaction finishes shortly after.
>
> should I disable client requests to a node being compacted while I
> carry out the compaction?
>
> Thanks
> Yang
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: Replicate On Write behavior

[David Hawthorne]

It was exactly due to 2890, and the fact that the first replica is always the 
one with the lowest value IP address.  I patched cassandra to pick a random 
node out of the replica set in StorageProxy.java findSuitableEndpoint:

Random rng = new Random();

return endpoints.get(rng.nextInt(endpoints.size()));  // instead of return 
endpoints.get(0);

Now work load is evenly balanced among all 5 nodes and I'm getting 2.5x the 
inserts/sec throughput.

Here's the behavior I saw, and "disk work" refers to the ReplicateOnWrite load 
of a counter insert:

One node will get RF/n of the disk work.  Two nodes will always get 0 disk work.

in a 3 node cluster, 1 node gets disk hit really hard.  You get the performance 
of a one-node cluster.
in a 6 node cluster, 1 node gets hit with 50% of the disk work, giving you the 
performance of ~2 node cluster.
in a 10 node cluster, 1 node gets 30% of the disk work, giving you the 
performance of a ~3 node cluster.

I confirmed this behavior with a 3, 4, and 5 node cluster size.


> 
>> On another note, on a 5-node cluster, I'm only seeing 3 nodes with 
>> ReplicateOnWrite Completed tasks in nodetool tpstats output.  Is that 
>> normal?  I'm using RandomPartitioner...
>> 
>> Address DC  RackStatus State   LoadOwns  
>>   Token
>>
>> 136112946768375385385349842972707284580
>> 10.0.0.57datacenter1 rack1   Up Normal  2.26 GB 20.00%  0
>> 10.0.0.56datacenter1 rack1   Up Normal  2.47 GB 20.00%  
>> 34028236692093846346337460743176821145
>> 10.0.0.55datacenter1 rack1   Up Normal  2.52 GB 20.00%  
>> 68056473384187692692674921486353642290
>> 10.0.0.54datacenter1 rack1   Up Normal  950.97 MB   20.00%  
>> 102084710076281539039012382229530463435
>> 10.0.0.72datacenter1 rack1   Up Normal  383.25 MB   20.00%  
>> 136112946768375385385349842972707284580
>> 
>> The nodes with ReplicateOnWrites are the 3 in the middle.  The first node 
>> and last node both have a count of 0.  This is a clean cluster, and I've 
>> been doing 3k ... 2.5k (decaying performance) inserts/sec for the last 12 
>> hours.  The last time this test ran, it went all the way down to 500 
>> inserts/sec before I killed it.
> 
> Could be due to https://issues.apache.org/jira//browse/CASSANDRA-2890.
> 
> --
> Sylvain

Re: row key as long type

[amulya rattan]

But I explicitly remember List throwing "long is exactly 8 bytes" when i
invoked on a column family with long as key.
Why would that happen?

On Thu, Sep 8, 2011 at 10:07 AM, Jonathan Ellis  wrote:

> List should work fine on any schema, including long keys.
>
>
> On Thu, Sep 8, 2011 at 8:23 AM, amulya rattan wrote:
>
>> Row key can certainly be of type long..you'd just have to set
>> key_validataion_class to be LongType.
>>
>> However, doing list on column family would throw an error..please look at
>> http://wiki.apache.org/cassandra/FAQ#a_long_is_exactly_8_bytes
>>
>>
>> On Thu, Sep 8, 2011 at 8:14 AM, Thamizh  wrote:
>>
>>> Hi All,
>>>
>>> Is there a way to store number(longtype) as row key in Cassadra?
>>>
>>> I wanted to execute range query based on row key value. e.g $list
>>> info[12345:]; . It should list all the rowkeys which are >= 12345.
>>>
>>>
>>> Is there a way accompolish this in cassandra? Secondary index does not
>>> helped me. So I am trying to store column value 'ip' as rowkey here.
>>>
>>> data model:
>>>
>>> create keyspace ipinfo with placement_strategy =
>>> 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options =
>>> [{replication_factor:1}];
>>>
>>> use rng;
>>>
>>> create column family info with comparator = AsciiType
>>>
>>>
>>>
>>> and key_validation_class = UTF8Type
>>> and column_metadata =
>>> [{
>>> column_name : domain,
>>> validation_class : UTF8Type,
>>> index_type : 0,
>>> index_name : domain_idx},
>>> {
>>> column_name : ip,
>>>
>>>
>>>
>>> validation_class : LongType,
>>> index_type : 0,
>>> index_name : ip_idx
>>> }];
>>>
>>> Regards,
>>> Thamizhannal
>>>
>>
>>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>

Question on using consistency level with NetworkTopologyStrategy

[Anand Somani]

Hi,

Have a requirement, where data is spread across multiple DC for disaster
recovery. So I would use the NTS, that is clear, but I have some questions
with this scenario

   - I have 2 Data Centers
   - RF - 2 (active DC) , 2 (passive DC)
   - with NTS - Consistency level options are - LOCAL_QUORUM and EACH
   _QUORUM
   - I want LOCAL_QUORUM and 1 remote copy (not 2) for write to succeed, if
   I used EACH_QUORUM - it would mean that I need both the remote nodes up (as
   I understand from http://www.datastax.com/docs/0.8/operations/datacenter
   ).

So if that is my requirement what consistency Level should I be using for my
writes? Is that even possible with NTS or another strategy? I could use the
SimpleStrategy with Quorum, but that would mean sending 2 copies (instead of
1 per DC that NTS uses to optimize on WAN traffic) to remote DC (since it
does not understand DC's)?

Thanks
Anand

Re: row key as long type

[Jonathan Ellis]

Probably because you had some non-long data in it, then added the long type
later.

On Thu, Sep 8, 2011 at 2:51 PM, amulya rattan  wrote:

> But I explicitly remember List throwing "long is exactly 8 bytes" when i
> invoked on a column family with long as key.
> Why would that happen?
>
>
> On Thu, Sep 8, 2011 at 10:07 AM, Jonathan Ellis  wrote:
>
>> List should work fine on any schema, including long keys.
>>
>>
>> On Thu, Sep 8, 2011 at 8:23 AM, amulya rattan wrote:
>>
>>> Row key can certainly be of type long..you'd just have to set
>>> key_validataion_class to be LongType.
>>>
>>> However, doing list on column family would throw an error..please look
>>> at http://wiki.apache.org/cassandra/FAQ#a_long_is_exactly_8_bytes
>>>
>>>
>>> On Thu, Sep 8, 2011 at 8:14 AM, Thamizh  wrote:
>>>
 Hi All,

 Is there a way to store number(longtype) as row key in Cassadra?

 I wanted to execute range query based on row key value. e.g $list
 info[12345:]; . It should list all the rowkeys which are >= 12345.


 Is there a way accompolish this in cassandra? Secondary index does not
 helped me. So I am trying to store column value 'ip' as rowkey here.

 data model:

 create keyspace ipinfo with placement_strategy =
 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options =
 [{replication_factor:1}];

 use rng;

 create column family info with comparator = AsciiType





 and key_validation_class = UTF8Type
 and column_metadata =
 [{
 column_name : domain,
 validation_class : UTF8Type,
 index_type : 0,
 index_name : domain_idx},
 {
 column_name : ip,





 validation_class : LongType,
 index_type : 0,
 index_name : ip_idx
 }];

 Regards,
 Thamizhannal

>>>
>>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>>
>
>


-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: no stack trace :(

[Dean Hiller]

to close on this issue.  I just had whatever the default cassandra 0.8.3
untar had and fixing DNS resolved the issue for me.  My computer was not in
DNS so I guess it has to be when trying a multinode install.
thanks,
Dean

On Sat, Aug 6, 2011 at 10:10 PM, aaron morton wrote:

> Do you have MX4J in the class path ?
>
> It feels like an error from there: MalformedURLException is a checked
> exception a it's only used in the cassandra code when reading a file; and
> "agent" sounds like JMX Agent.
>
> I had a quick search through mx4j source and while I could not find an
> exact match, there is a lot of places where MalformedURLException is thrown.
>
> Hope that helps.
>
>
>  -
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 6 Aug 2011, at 07:02, Dean Hiller wrote:
>
> nope, just one and then it exits and quits(won't start up) and I can't
> compile the code since the repository is down :(
>
> I am working on web-tier setup and will come back hoping the cloudera
> repository goes back online in the next day.(crossing my fingers).
>
> Dean
>
> On Fri, Aug 5, 2011 at 12:07 PM, mcasandra  wrote:
>
>> Are you seeing lot of these errors? Can you try
>> XX:-OmitStackTraceInFastThrow
>>
>> --
>> View this message in context:
>> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/no-stack-trace-tp6654590p6657485.html
>> Sent from the cassandra-u...@incubator.apache.org mailing list archive at
>> Nabble.com.
>>
>
>
>

Re: question on setup for writes into 2 datacenters

[Dean Hiller]

thanks this is perfect...yeah, we want 4 nodes(2 in each data center) so
just a bit more than 3 ;) so we can have both datacenters be active at the
same time and not have to write to the other datacenter except in the
background.  This is perfect..hbase couldn't do this which I wanted
badly along with the in-memory write and write to disk in the background.
thanks
Dean

On Sun, Jul 24, 2011 at 4:40 PM, aaron morton wrote:

> Quick reminder, with RF == 2 the QUORUM is 2 as well. So when using
> LOCAL_QUORUM with RF 2+2 you will effectively be using LOCAL_ALL which may
> not be what you want. As De La Soul sang, 3 is the magic number for minimum
> fault tolerance (QUORUM is then 2).
>
> Cheers
>
>  -
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 23 Jul 2011, at 10:04, Sameer Farooqui wrote:
>
> It sounds like what you're looking for is write consistency of
> local_quorum:
> http://www.datastax.com/docs/0.8/consistency/index#write-consistency
>
> local_quorum would mean the write has to be successful on a majority of
> nodes in DC1 (so 2) before it is considered successful.
>
> If you use just quorum write, it'll have to be committed to 3 replicas out
> of the 4 before it's considered successful.
>
>
>
>
> On Fri, Jul 22, 2011 at 1:57 PM, Dean Hiller  wrote:
>
>> Ideally, we would want to have a replication factor of 4, and a minimum
>> write consistency of 2 (which looking at the default in cassandra.yaml is to
>> memory first with asynch to disk...perfect so far!!!)
>>
>> Now, obviously, I can get the partitioner setup to make sure I get 2
>> replicas in each data center.  The next thing I would want to guarantee
>> however is that if a write came into datacenter 1, it would write to the two
>> nodes in datacenter 1 and asynchronously replicate to datacenter 2.  Is this
>> possible?  Does cassandra already handle that or is there something I could
>> do to get cassandra to do that?
>>
>> In this mode, I believe I can have both datacenters be live as well as be
>> backup for the other not wasting resources.
>>
>> thanks,
>> Dean
>>
>
>
>

possible feature request RP vs. OPP

[Dean Hiller]

I was wondering something.  Since I can take OPP and I can create a layer
that for certain column families, I hash the key so that some column
families are just like RP but on top of OPP and some of my other column
families are then on OPP directly so I could use lucandra, why not make RP
deprecated and instead allow users to create OPP by column family or RP
where RP == doing the hash of the key on my behalf and prefixing my key with
that hashcode and stripping it back off when I read it in again.

ie. why have RP when you could do RP per column family with the above
reasoning on top of OPP and have the best of both worlds?

ie. I think of having some column families random and then some column
famiiles ordered so I could range query or use lucandra on top of those
ones.

thoughts?  I was just curious.
thanks,
Dean

Anybody out there using 0.8 in production

[Anand Somani]

Hi

Currently we are using 0.7.4 and was wondering if I should upgrade to
0.7.8/9 or move to 0.8? Is anybody using 0.8 in production and what is their
experience?

Thanks

Re: possible feature request RP vs. OPP

[Edward Capriolo]

On Thu, Sep 8, 2011 at 5:12 PM, Dean Hiller  wrote:

> I was wondering something.  Since I can take OPP and I can create a layer
> that for certain column families, I hash the key so that some column
> families are just like RP but on top of OPP and some of my other column
> families are then on OPP directly so I could use lucandra, why not make RP
> deprecated and instead allow users to create OPP by column family or RP
> where RP == doing the hash of the key on my behalf and prefixing my key with
> that hashcode and stripping it back off when I read it in again.
>
> ie. why have RP when you could do RP per column family with the above
> reasoning on top of OPP and have the best of both worlds?
>
> ie. I think of having some column families random and then some column
> famiiles ordered so I could range query or use lucandra on top of those
> ones.
>
> thoughts?  I was just curious.
> thanks,
> Dean
>
>
You can use ByteOrderPartitioner and hash data yourself. However that makes
every row key will be 128bits larger as the key has to be:

md5+originalkey

http://www.datastax.com/wp-content/uploads/2011/07/Scaling_Solr_with_Cassandra-CassandraSF2011.pdf

Solandra now uses a 'modified' RandomPartitioner.

Re: Question on using consistency level with NetworkTopologyStrategy

[Edward Capriolo]

On Thu, Sep 8, 2011 at 4:14 PM, Anand Somani  wrote:

> Hi,
>
> Have a requirement, where data is spread across multiple DC for disaster
> recovery. So I would use the NTS, that is clear, but I have some questions
> with this scenario
>
>- I have 2 Data Centers
>- RF - 2 (active DC) , 2 (passive DC)
>- with NTS - Consistency level options are - LOCAL_QUORUM and EACH
>_QUORUM
>- I want LOCAL_QUORUM and 1 remote copy (not 2) for write to succeed,
>if I used EACH_QUORUM - it would mean that I need both the remote nodes up
>(as I understand from
>http://www.datastax.com/docs/0.8/operations/datacenter).
>
> So if that is my requirement what consistency Level should I be using for
> my writes? Is that even possible with NTS or another strategy? I could use
> the SimpleStrategy with Quorum, but that would mean sending 2 copies
> (instead of 1 per DC that NTS uses to optimize on WAN traffic) to remote DC
> (since it does not understand DC's)?
>
> Thanks
> Anand
>

Cassandra may not be doing what you think.

QUORUM levels work oddly with less then 3 replicas.

You will probably will get closest to what you want with CL.TWO or
CL.THREE.

Re: Question on using consistency level with NetworkTopologyStrategy

[Jonathan Ellis]

CL.QUORUM is supported with any replication strategy, not just simple.

Also, Cassandra's optimizing of cross-DC writes only requires that it
know (via a correctly configured Snitch) where each node is located.
It is not affected by replication strategy choice.

On Thu, Sep 8, 2011 at 3:14 PM, Anand Somani  wrote:
> Hi,
>
> Have a requirement, where data is spread across multiple DC for disaster
> recovery. So I would use the NTS, that is clear, but I have some questions
> with this scenario
>
> I have 2 Data Centers
> RF - 2 (active DC) , 2 (passive DC)
> with NTS - Consistency level options are - LOCAL_QUORUM and EACH _QUORUM
> I want LOCAL_QUORUM and 1 remote copy (not 2) for write to succeed, if I
> used EACH_QUORUM - it would mean that I need both the remote nodes up (as I
> understand from http://www.datastax.com/docs/0.8/operations/datacenter).
>
> So if that is my requirement what consistency Level should I be using for my
> writes? Is that even possible with NTS or another strategy? I could use the
> SimpleStrategy with Quorum, but that would mean sending 2 copies (instead of
> 1 per DC that NTS uses to optimize on WAN traffic) to remote DC (since it
> does not understand DC's)?
>
> Thanks
> Anand
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: Anybody out there using 0.8 in production

[Anthony Ikeda]

We plan to and have been using it in Dev and QA. There are some bugs that
have been fixed that we are looking forward to in 0.8.5 and probably that
would be the better build for production (there is a quorum bug that we will
need).

Otherwise no other 0.8 issues that we are aware of. We did go through a long
analysis of properly configuring connection options and getting the server
setup right but DataStax were abe to support us and get us sorted.

Anthony


On Thu, Sep 8, 2011 at 2:44 PM, Anand Somani  wrote:

> Hi
>
> Currently we are using 0.7.4 and was wondering if I should upgrade to
> 0.7.8/9 or move to 0.8? Is anybody using 0.8 in production and what is their
> experience?
>
> Thanks
>

what's the difference between repair CF separately and repair the entire node?

[Yan Chunlu]

I have 3 nodes and RF=3.  I  tried to repair every node in the cluster by
using "nodetool repair mykeyspace mycf" on every column family.  it finished
within 3 hours, the data size is no more than 50GB.
after the repair, I have tried using nodetool repair immediately to repair
the entire node, but 48 hours has past it still going on. "compactionstats"
shows it is doing "SSTable rebuild".

so I am frustrating about why does "nodetool repair" so slow?   how does it
different with repair every CF?

I didn't tried to repair the system keyspace, does it also need to repair?
 thanks!

mysterious data disappearance - what happened?

[Yiming Sun]

Hello,

If two different instances of cassandra are running on separate machines,
but both are unfortunately configured to use the default cluster name, "Test
Cluster", do they gang up as one cluster (even though they were intended to
be two separate stand-alone instances), so that dropping keyspace on one
machine would result in the disappearance of data from the other?

We ran into some mysterious data disappearance situations -- and we are not
sure if this is something with our configuration.

We installed 2 instances of cassandra 0.8.2 on 2 separate machines, one is
silk.cs.our.domain and the other smoke.cs.our.domain.  Both the code and
data directories reside on the local partition of each machine, and we left
the the yaml file pretty much the default, except we changed the data
directory paths and also the listen addresses to point to each host.  What I
want to point out is, both files had the default cluster name "Test
Cluster".

Both cassandra instances were loaded with identical keyspaces, data sets,
and the intention was one would be used as production and the other testing
cassandra server.

Then yesterday we were trying to drop a keyspace on silk, which was the
testing server, and sometimes later, a user complained that he could not get
any data from the production machine.  We were intrigued but thought maybe
someone made a mistake.

Then the same thing happened again today.  So we began to suspect that two
servers somehow discovered each other.  In fact, after we shut down the
cassandra on silk, and then load a keyspace definition into smoke using
cassandra-cli, it even said unreachable destination and the IP was that of
silk.  I just want to confirm that the culprit was because we used the same
cluster name for both machines.  So in other words, if we want to launch two
separate instances of cassandra and keep them separate, we must make sure
each uses a different cluster name or else they will gang up into the same
cluster?  But how do they even discover each other?  Can someone enlighten
me please?  Thanks.


-- Y.

Re: Anybody out there using 0.8 in production

[Dominic Williams]

Hi I've just migrated to 0.8.5 and from first looks it is a giant leap
forward
- better use of CPU and memory
- able to scrub files previously unfixable on 0.7.6-2
etc

On 9 September 2011 01:45, Anthony Ikeda wrote:

> We plan to and have been using it in Dev and QA. There are some bugs that
> have been fixed that we are looking forward to in 0.8.5 and probably that
> would be the better build for production (there is a quorum bug that we will
> need).
>
> Otherwise no other 0.8 issues that we are aware of. We did go through a
> long analysis of properly configuring connection options and getting the
> server setup right but DataStax were abe to support us and get us sorted.
>
> Anthony
>
>
> On Thu, Sep 8, 2011 at 2:44 PM, Anand Somani  wrote:
>
>> Hi
>>
>> Currently we are using 0.7.4 and was wondering if I should upgrade to
>> 0.7.8/9 or move to 0.8? Is anybody using 0.8 in production and what is their
>> experience?
>>
>> Thanks
>>
>
>

Re: Anybody out there using 0.8 in production

[Eric Czech]

We just migrated from .7.5 to .8.4 in our production environment and it was
definitely the least painful transition yet (coming all the way from the .4
release series).  It's been about a week for us but so far so good.

On Thu, Sep 8, 2011 at 9:25 PM, Dominic Williams <
dwilli...@fightmymonster.com> wrote:

> Hi I've just migrated to 0.8.5 and from first looks it is a giant leap
> forward
> - better use of CPU and memory
> - able to scrub files previously unfixable on 0.7.6-2
> etc
>
>
> On 9 September 2011 01:45, Anthony Ikeda wrote:
>
>> We plan to and have been using it in Dev and QA. There are some bugs that
>> have been fixed that we are looking forward to in 0.8.5 and probably that
>> would be the better build for production (there is a quorum bug that we will
>> need).
>>
>> Otherwise no other 0.8 issues that we are aware of. We did go through a
>> long analysis of properly configuring connection options and getting the
>> server setup right but DataStax were abe to support us and get us sorted.
>>
>> Anthony
>>
>>
>> On Thu, Sep 8, 2011 at 2:44 PM, Anand Somani wrote:
>>
>>> Hi
>>>
>>> Currently we are using 0.7.4 and was wondering if I should upgrade to
>>> 0.7.8/9 or move to 0.8? Is anybody using 0.8 in production and what is their
>>> experience?
>>>
>>> Thanks
>>>
>>
>>
>

Re: Anybody out there using 0.8 in production

[Chris Goffinet]

Twitter runs 0.8 in production/closer to trunk. No big issues from us.


On Thu, Sep 8, 2011 at 8:53 PM, Eric Czech  wrote:

> We just migrated from .7.5 to .8.4 in our production environment and it was
> definitely the least painful transition yet (coming all the way from the .4
> release series).  It's been about a week for us but so far so good.
>
>
> On Thu, Sep 8, 2011 at 9:25 PM, Dominic Williams <
> dwilli...@fightmymonster.com> wrote:
>
>> Hi I've just migrated to 0.8.5 and from first looks it is a giant leap
>> forward
>> - better use of CPU and memory
>> - able to scrub files previously unfixable on 0.7.6-2
>> etc
>>
>>
>> On 9 September 2011 01:45, Anthony Ikeda wrote:
>>
>>> We plan to and have been using it in Dev and QA. There are some bugs that
>>> have been fixed that we are looking forward to in 0.8.5 and probably that
>>> would be the better build for production (there is a quorum bug that we will
>>> need).
>>>
>>> Otherwise no other 0.8 issues that we are aware of. We did go through a
>>> long analysis of properly configuring connection options and getting the
>>> server setup right but DataStax were abe to support us and get us sorted.
>>>
>>> Anthony
>>>
>>>
>>> On Thu, Sep 8, 2011 at 2:44 PM, Anand Somani wrote:
>>>
 Hi

 Currently we are using 0.7.4 and was wondering if I should upgrade to
 0.7.8/9 or move to 0.8? Is anybody using 0.8 in production and what is 
 their
 experience?

 Thanks

>>>
>>>
>>
>

Re: Anybody out there using 0.8 in production

[Jeremy Hanna]

We run 0.8 in production and it's been working well for us.  There are some new 
settings that we had to tune for - for example, the default concurrent 
compaction is the number of cores.  We had to tune that down because we also 
run hadoop jobs on our nodes.

On Sep 8, 2011, at 4:44 PM, Anand Somani wrote:

> Hi 
> 
> Currently we are using 0.7.4 and was wondering if I should upgrade to 0.7.8/9 
> or move to 0.8? Is anybody using 0.8 in production and what is their 
> experience?
> 
> Thanks

disk full and COMMIT-LOG-WRITER ?

[Yang]

I found the reason of my server freeze:

COMMIT-LOG-WRITER thread is gone, dead, so the blocking queue in
PeriodicCommitLogExecutorService is full, then all mutationStage jobs
are stuck on the mutations flushing.

the COMMIT-LOG-WRITER thread died because at one time the disk was full,
I cleaned up the disk space (not deleting cassandra files, but other files),
but then since the thread is gone, system is still stuck.
so I had to restart the server.


is it better to let the WRITER thread handle file system exceptions or
let it die? granted letting disk go full is not
a good practice, but letting the system proceed after disk is freed
seems a more natural expectation.

Thanks
Yang

Re: disk full and COMMIT-LOG-WRITER ?

[Yang]

ok, found past discussions:

http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/cassandra-server-disk-full-td6560725.html

On Thu, Sep 8, 2011 at 11:00 PM, Yang  wrote:
> I found the reason of my server freeze:
>
> COMMIT-LOG-WRITER thread is gone, dead, so the blocking queue in
> PeriodicCommitLogExecutorService is full, then all mutationStage jobs
> are stuck on the mutations flushing.
>
> the COMMIT-LOG-WRITER thread died because at one time the disk was full,
> I cleaned up the disk space (not deleting cassandra files, but other files),
> but then since the thread is gone, system is still stuck.
> so I had to restart the server.
>
>
> is it better to let the WRITER thread handle file system exceptions or
> let it die? granted letting disk go full is not
> a good practice, but letting the system proceed after disk is freed
> seems a more natural expectation.
>
> Thanks
> Yang
>