RE: Problem while streaming SSTables with BulkOutputFormat

[Ralph Romanos]

Hello again,
I noticed that this issue happens whenever a reduce task is done (so the 
SSTable is generated) while an SSTable already generated is being streamed to 
the cluster. I think that the error is therefore caused because cassandra 
cannot queue SSTables that are streamed to the cluster.Does that make sense?
Cheers,Ralph

From: matgan...@hotmail.com
To: user@cassandra.apache.org
Subject: RE: Problem while streaming SSTables with BulkOutputFormat
Date: Tue, 9 Oct 2012 22:29:41 +





Aaron,Thank you for your answer, I tried to move to Cassandra 1.1.5, but the 
error still occurs.When I set a single task or less per hadoop node, the error 
does not happen.However, when I have more than one task on any of the nodes 
(Hadoop only node or Hadoop+Cassandra node),the error happens. When it happens, 
the task fails and is sent after a while to another node and completed. 
Ultimately, I getall my tasks done but it takes much more time.
Is it possible that streaming multiple SSTables generated from two different 
tasks done by the same node to the Cassandra cluster is thecause of this issue?
CheersRalph
> Subject: Re: Problem while streaming SSTables with BulkOutputFormat
> From: aa...@thelastpickle.com
> Date: Wed, 10 Oct 2012 10:05:13 +1300
> To: user@cassandra.apache.org
> 
> Something, somewhere, at some point is breaking the connection. Sorry I 
> cannot be of more help :)
> 
> Something caused the streaming to fail, which started a retry, which failed 
> because the pipe was broken. 
> 
> Are there any earlier errors in the logs ? 
> Did this happen on one of the nodes that has both a task tacker and cassandra 
> ?
> 
> Cheers
> 
> 
> On 9/10/2012, at 4:06 AM, Ralph Romanos  wrote:
> 
> > Hello,
> > 
> > I am using BulkOutputFormat to load data from a .csv file into Cassandra. I 
> > am using Cassandra 1.1.3 and Hadoop 0.20.2.
> > I have 7 hadoop nodes: 1 namenode/jobtracker and 6 datanodes/tasktrackers. 
> > Cassandra is installed on 4 of these 6 datanodes/tasktrackers.
> > The issue happens when I have more than 1 reducer, SSTables are generated 
> > in each node, however, I get the following error in the tasktracker's logs 
> > when they 
> > are streamed into the Cassandra cluster:
> > 
> > Exception in thread "Streaming to /172.16.110.79:1" 
> > java.lang.RuntimeException: java.io.EOFException
> > at 
> > org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:628)
> > at 
> > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
> > Source)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> > at java.lang.Thread.run(Unknown Source)
> > Caused by: java.io.EOFException
> > at java.io.DataInputStream.readInt(Unknown Source)
> > at 
> > org.apache.cassandra.streaming.FileStreamTask.receiveReply(FileStreamTask.java:194)
> > at 
> > org.apache.cassandra.streaming.FileStreamTask.stream(FileStreamTask.java:181)
> > at 
> > org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:94)
> > at 
> > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> > ... 3 more
> > Exception in thread "Streaming to /172.16.110.92:1" 
> > java.lang.RuntimeException: java.io.EOFException
> > at 
> > org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:628)
> > at 
> > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
> > Source)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> > at java.lang.Thread.run(Unknown Source)
> > Caused by: java.io.EOFException
> > at java.io.DataInputStream.readInt(Unknown Source)
> > at 
> > org.apache.cassandra.streaming.FileStreamTask.receiveReply(FileStreamTask.java:194)
> > at 
> > org.apache.cassandra.streaming.FileStreamTask.stream(FileStreamTask.java:181)
> > at 
> > org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:94)
> > at 
> > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> > ... 3 more
> > 
> >  ...
> > 
> > This is what I get in the logs of one of my Cassandra nodes:
> > ERROR 16:47:34,904 Sending retry message failed, closing session.
> > java.io.IOException: Broken pipe
> > at sun.nio.ch.FileDispatcher.write0(Native Method)
> > at sun.nio.ch.SocketDispatcher.write(Unknown Source)
> > at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> > at sun.nio.ch.IOUtil.write(Unknown Source)
> > at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
> > at java.nio.channels.Channels.writeFullyImpl(Unknown Source)
> > at java.nio.channels.Channels.writeFully(Unknown Source)
> > at java.nio.channels.Channels.access$000(Unknown Source)
> > at java.nio.channels.Channels$1.write(Unkn

Re: 1.1.1 is "repair" still needed ?

[Watanabe Maki]

Even if HH works fine, HH will not be created until the failure detector marks  
the node is dead. 
HH will not be created for partially timeouted mutation request ( but meets CL 
) also... In my understanding...


On 2012/10/11, at 5:55, Rob Coli  wrote:

> On Tue, Oct 9, 2012 at 12:56 PM, Oleg Dulin  wrote:
>> My understanding is that the repair has to happen within gc_grace period.
>> [ snip ]
>> So the question is, is this still needed ? Do we even need to run nodetool
>> repair ?
> 
> If Hinted Handoff works in your version of Cassandra, and that version
> is > 1.0, you "should" not need to repair if no node has crashed or
> been down for longer than max_hint_window_in_ms. This is because after
> 1.0, any failed write to a remote replica results in a hint, so any
> DELETE should eventually be fully replicated.
> 
> However hinted handoff is meaningfully broken between 1.1.0 and 1.1.6
> (unreleased) so you cannot rely on the above heuristic for
> consistency. In these versions, you have to repair (or read repair
> 100% of keys) once every GCGraceSeconds to prevent the possibility of
> zombie data. If it were possible to repair on a per-columnfamily
> basis, you could get a significant win by only repairing
> columnfamilies which take DELETE traffic.
> 
> https://issues.apache.org/jira/browse/CASSANDRA-4772
> 
> =Rob
> 
> -- 
> =Robert Coli
> AIM>ALK - rc...@palominodb.com
> YAHOO - rcoli.palominob
> SKYPE - rcoli_palominodb

Create column family with Composite key column via thrift API

[Vivek Mishra]

Hi,

I know one way is to execute cql query via thrift client  to create a
column family having compound primary/composite columns. But is it the only
way?

Looks like i would end up creating  own "CQLTranslator/Wrapper" to deal
with compound primary/composite columns!(Or may be something else in near
future). Thrift way of dealing with this is really different, as column
family metadata for such column families created on "cqlsh" is quite
different for Thrift!

I know, have started little late and these are basic things how
CQL/composite column works. But is there anything i am
missing/misunderstood on this part?

-Vivek

CQL Sets and Maps

[Hiller, Dean]

I was reading Brian's post

http://mail-archives.apache.org/mod_mbox/cassandra-dev/201210.mbox/%3ccajhhpg20rrcajqjdnf8sf7wnhblo6j+aofksgbxyxwcoocg...@mail.gmail.com%3E

In which he asks

> Any insight into why CQL puts that in column name?
> Where does it store the metadata related to compound key
> interpretation? Wouldn't that be a better place for that since it
> shouldn't change within a table?

I have those same questions and would like to understand how it stores stuff 
better.  For example, if PlayOrm has the following

User {
   @Embedded
Private List emails;
@Embedded
Private List otherStuff
@OneToMany
Private List owners;
}

It ends up storing

rowkey: userid
=> column=emails:email1Id:title, value="some email title"
=> column=emails:email1Id:contents, value="some contents in email really really 
long"
=> column=emails:email2Id:title, value="some other email"
=> column=owners:ownerId29,  value=null
=> column=owners:ownerId57,  value=null

Basically using "emails" as the prefix since User can have other embedded 
objects, and using emailId as the next prefix so you can have many unique 
emails and then having each email property.  How is it actually stored when 
doing Sets and Maps in CQL??  Ideally, I would like PlayOrm to overlay on top 
of that.

Thanks,
Dean

unsubscribe

[Chris Favero]

unsubscribe

cassandra + pig

[William Oberman]

I'm wondering how many people are using cassandra + pig out there?  I
recently went through the effort of validating things at a much higher
level than I previously did(*), and found a few issues:
https://issues.apache.org/jira/browse/CASSANDRA-4748
https://issues.apache.org/jira/browse/CASSANDRA-4749
https://issues.apache.org/jira/browse/CASSANDRA-4789

In general, it seems like the widerow implementation still has rough edges.
 I'm concerned I'm not understanding why other people aren't using the
feature, and thus finding these problems.  Is everyone else just setting a
high static limit?  E.g.  LOAD 'cassandra://KEYSPACE/CF?limit=X" where X >=
the max size of any key?  Is everyone else using data models that result in
keys with # columns always less than 1024?  Do newer version of hadoop
consume the cassandra API in a way that work around these issues?  I'm
using CDH3 == hadoop 0.20.2, pig 0.8.1.

(*) I took a random subsample of 50,000 keys of my production data (approx
1M total key/value pairs, some keys having only a single value and some
having 1000's).  I then wrote both a pig script and simple procedural
version of the pig script.  Then I compared the results.  Obviously I
started with differences, though after locally patching my code to fix the
above 3 bugs (though, really only two issues), I now (finally) get the same
results.

Re: 1.1.1 is "repair" still needed ?

[B. Todd Burruss]

as of 1.0 (CASSANDRA-2034) hints are generated for nodes that timeout.

On Thu, Oct 11, 2012 at 3:55 AM, Watanabe Maki  wrote:
> Even if HH works fine, HH will not be created until the failure detector 
> marks  the node is dead.
> HH will not be created for partially timeouted mutation request ( but meets 
> CL ) also... In my understanding...
>
>
> On 2012/10/11, at 5:55, Rob Coli  wrote:
>
>> On Tue, Oct 9, 2012 at 12:56 PM, Oleg Dulin  wrote:
>>> My understanding is that the repair has to happen within gc_grace period.
>>> [ snip ]
>>> So the question is, is this still needed ? Do we even need to run nodetool
>>> repair ?
>>
>> If Hinted Handoff works in your version of Cassandra, and that version
>> is > 1.0, you "should" not need to repair if no node has crashed or
>> been down for longer than max_hint_window_in_ms. This is because after
>> 1.0, any failed write to a remote replica results in a hint, so any
>> DELETE should eventually be fully replicated.
>>
>> However hinted handoff is meaningfully broken between 1.1.0 and 1.1.6
>> (unreleased) so you cannot rely on the above heuristic for
>> consistency. In these versions, you have to repair (or read repair
>> 100% of keys) once every GCGraceSeconds to prevent the possibility of
>> zombie data. If it were possible to repair on a per-columnfamily
>> basis, you could get a significant win by only repairing
>> columnfamilies which take DELETE traffic.
>>
>> https://issues.apache.org/jira/browse/CASSANDRA-4772
>>
>> =Rob
>>
>> --
>> =Robert Coli
>> AIM>ALK - rc...@palominodb.com
>> YAHOO - rcoli.palominob
>> SKYPE - rcoli_palominodb

Re: cassandra + pig

[Jeremy Hanna]

The Dachis Group (where I just came from, now at DataStax) uses pig with 
cassandra for a lot of things.  However, we weren't using the widerow 
implementation yet since wide row support is new to 1.1.x and we were on 0.7, 
then 0.8, then 1.0.x.

I think since it's new to 1.1's hadoop support, it sounds like there are some 
rough edges like you say.  But issues that are reproducible on tickets for any 
problems are much appreciated and they will get addressed.

On Oct 11, 2012, at 10:43 AM, William Oberman  wrote:

> I'm wondering how many people are using cassandra + pig out there?  I 
> recently went through the effort of validating things at a much higher level 
> than I previously did(*), and found a few issues:
> https://issues.apache.org/jira/browse/CASSANDRA-4748
> https://issues.apache.org/jira/browse/CASSANDRA-4749
> https://issues.apache.org/jira/browse/CASSANDRA-4789
> 
> In general, it seems like the widerow implementation still has rough edges.  
> I'm concerned I'm not understanding why other people aren't using the 
> feature, and thus finding these problems.  Is everyone else just setting a 
> high static limit?  E.g.  LOAD 'cassandra://KEYSPACE/CF?limit=X" where X >= 
> the max size of any key?  Is everyone else using data models that result in 
> keys with # columns always less than 1024?  Do newer version of hadoop 
> consume the cassandra API in a way that work around these issues?  I'm using 
> CDH3 == hadoop 0.20.2, pig 0.8.1.
> 
> (*) I took a random subsample of 50,000 keys of my production data (approx 1M 
> total key/value pairs, some keys having only a single value and some having 
> 1000's).  I then wrote both a pig script and simple procedural version of the 
> pig script.  Then I compared the results.  Obviously I started with 
> differences, though after locally patching my code to fix the above 3 bugs 
> (though, really only two issues), I now (finally) get the same results.

Re: cassandra + pig

[William Oberman]

If you don't mind me asking, how are you handling the fact that pre-widerow
you are only getting a static number of columns per key (default 1024)?  Or
am I not understanding the "limit" concept?

On Thu, Oct 11, 2012 at 11:25 AM, Jeremy Hanna
wrote:

> The Dachis Group (where I just came from, now at DataStax) uses pig with
> cassandra for a lot of things.  However, we weren't using the widerow
> implementation yet since wide row support is new to 1.1.x and we were on
> 0.7, then 0.8, then 1.0.x.
>
> I think since it's new to 1.1's hadoop support, it sounds like there are
> some rough edges like you say.  But issues that are reproducible on tickets
> for any problems are much appreciated and they will get addressed.
>
> On Oct 11, 2012, at 10:43 AM, William Oberman 
> wrote:
>
> > I'm wondering how many people are using cassandra + pig out there?  I
> recently went through the effort of validating things at a much higher
> level than I previously did(*), and found a few issues:
> > https://issues.apache.org/jira/browse/CASSANDRA-4748
> > https://issues.apache.org/jira/browse/CASSANDRA-4749
> > https://issues.apache.org/jira/browse/CASSANDRA-4789
> >
> > In general, it seems like the widerow implementation still has rough
> edges.  I'm concerned I'm not understanding why other people aren't using
> the feature, and thus finding these problems.  Is everyone else just
> setting a high static limit?  E.g.  LOAD 'cassandra://KEYSPACE/CF?limit=X"
> where X >= the max size of any key?  Is everyone else using data models
> that result in keys with # columns always less than 1024?  Do newer version
> of hadoop consume the cassandra API in a way that work around these issues?
>  I'm using CDH3 == hadoop 0.20.2, pig 0.8.1.
> >
> > (*) I took a random subsample of 50,000 keys of my production data
> (approx 1M total key/value pairs, some keys having only a single value and
> some having 1000's).  I then wrote both a pig script and simple procedural
> version of the pig script.  Then I compared the results.  Obviously I
> started with differences, though after locally patching my code to fix the
> above 3 bugs (though, really only two issues), I now (finally) get the same
> results.
>
>

unnecessary tombstone's transmission during repair process

[Alexey Zotov]

Hi Guys,

I have a question about merkle tree construction and repair process. When
mercle tree is constructing it calculates hashes. For DeletedColumn it
calculates hash using value. Value of DeletedColumn is a serialized local
deletion time. We know that local deletion time can be different on
different nodes for the same tombstone. So hashes of the same tombstone on
different nodes will be different. Is it true? I think that local deletion
time shouldn't be considered in hash's calculation.

We've provided several tests:
// we have 3 node, RF=2, CL=QUORUM. So we have strong consistency.
1. Populate data to all nodes. Run repair process. No any streams were
transmitted. It's predictable behaviour.
2. Then we removed some columns for some rows. No any nodes we down. All
writes were done successfully. We run repair. There were some streams. It's
strange for me, because all data should be consistent.

We've created some patch and applied it.
1. Result of the first test is the same.
2. Result of the second test: there were no any unnecessary streams as I
expected.


My question is:
Is transmission of the equals tombstones during repair process a feature?
:) or is it a bug?
If it's a bug, I'll create ticket and attach patch to it.

Re: can't get cqlsh running

[Nick Bailey]

It looks like easy_install is only recognizing python2.4 on your
system. It is installing the cql module for that version. The cqlsh
script explicitly looks for and runs with python2.6 since 2.4 isn't
supported.

I believe you can run 'python2.6 easy_install cql' to force it to use
that python install.

-Nick

On Thu, Oct 11, 2012 at 10:45 AM, Tim Dunphy  wrote:
> Hey guys,
>
>  I'm on cassandra 1.1.5 on a centos 5.8 machine. I have the cassandra bin
> directory on my path so that i can simply type 'cassandra-cli' from anywhere
> on my path to get into the cassandra command line environment. It's great!
>
> But I'd like to start using the cql shell (cqlsh) but apparently I don't
> know enough about python to get this working. This is what happens when I
> try to run cqlsh:
>
> [root@beta:~] #cqlsh
>
> Python CQL driver not installed, or not on PYTHONPATH.
> You might try "easy_install cql".
>
> Python: /usr/bin/python2.6
> Module load path: ['/usr/local/apache-cassandra-1.1.5/bin/../pylib',
> '/usr/local/apache-cassandra-1.1.5/bin',
> '/usr/local/apache-cassandra-1.1.5/bin', '/usr/lib64/python26.zip',
> '/usr/lib64/python2.6', '/usr/lib64/python2.6/plat-linux2',
> '/usr/lib64/python2.6/lib-tk', '/usr/lib64/python2.6/lib-old',
> '/usr/lib64/python2.6/lib-dynload', '/usr/lib64/python2.6/site-packages',
> '/usr/lib/python2.6/site-packages',
> '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info']
>
> Error: No module named cql
>
> But easy_install claims that it's already installed:
>
> [root@beta:~] #easy_install cql
> Searching for cql
> Best match: cql 1.0.10
> Processing cql-1.0.10-py2.4.egg
> cql 1.0.10 is already the active version in easy-install.pth
>
> Using /usr/lib/python2.4/site-packages/cql-1.0.10-py2.4.egg
> Processing dependencies for cql
>
> I'm thinking that I just don't know how to set the PYTHONPATH variable or
> where to point it to. Can someone give me a pointer here?
>
> Thanks
> Tim
>
> --
> GPG me!!
>
> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
>
>

Re: cassandra + pig

[Jeremy Hanna]

For our use case, we had a lot of narrow column families and the couple of 
column families that had wide rows, we did our own paging through them.  I 
don't recall if we did paging in pig or mapreduce but you should be able to do 
that in both since pig allows you to specify the slice start.

On Oct 11, 2012, at 11:28 AM, William Oberman  wrote:

> If you don't mind me asking, how are you handling the fact that pre-widerow 
> you are only getting a static number of columns per key (default 1024)?  Or 
> am I not understanding the "limit" concept?  
> 
> On Thu, Oct 11, 2012 at 11:25 AM, Jeremy Hanna  
> wrote:
> The Dachis Group (where I just came from, now at DataStax) uses pig with 
> cassandra for a lot of things.  However, we weren't using the widerow 
> implementation yet since wide row support is new to 1.1.x and we were on 0.7, 
> then 0.8, then 1.0.x.
> 
> I think since it's new to 1.1's hadoop support, it sounds like there are some 
> rough edges like you say.  But issues that are reproducible on tickets for 
> any problems are much appreciated and they will get addressed.
> 
> On Oct 11, 2012, at 10:43 AM, William Oberman  
> wrote:
> 
> > I'm wondering how many people are using cassandra + pig out there?  I 
> > recently went through the effort of validating things at a much higher 
> > level than I previously did(*), and found a few issues:
> > https://issues.apache.org/jira/browse/CASSANDRA-4748
> > https://issues.apache.org/jira/browse/CASSANDRA-4749
> > https://issues.apache.org/jira/browse/CASSANDRA-4789
> >
> > In general, it seems like the widerow implementation still has rough edges. 
> >  I'm concerned I'm not understanding why other people aren't using the 
> > feature, and thus finding these problems.  Is everyone else just setting a 
> > high static limit?  E.g.  LOAD 'cassandra://KEYSPACE/CF?limit=X" where X >= 
> > the max size of any key?  Is everyone else using data models that result in 
> > keys with # columns always less than 1024?  Do newer version of hadoop 
> > consume the cassandra API in a way that work around these issues?  I'm 
> > using CDH3 == hadoop 0.20.2, pig 0.8.1.
> >
> > (*) I took a random subsample of 50,000 keys of my production data (approx 
> > 1M total key/value pairs, some keys having only a single value and some 
> > having 1000's).  I then wrote both a pig script and simple procedural 
> > version of the pig script.  Then I compared the results.  Obviously I 
> > started with differences, though after locally patching my code to fix the 
> > above 3 bugs (though, really only two issues), I now (finally) get the same 
> > results.
> 
> 
> 

Re: cassandra + pig

[William Oberman]

Thanks Jeremy!  Maybe figuring out how to do paging in pig would have been
easier, but I found the widerow setting first which led me where I am
today.  I don't mind helping to blaze trails, or contribute back when doing
so, but I usually try to follow rather than lead when it comes to
tools/software I choose use.  I didn't realize how close to the edge I was
getting in this case :-)

On Thu, Oct 11, 2012 at 1:03 PM, Jeremy Hanna wrote:

> For our use case, we had a lot of narrow column families and the couple of
> column families that had wide rows, we did our own paging through them.  I
> don't recall if we did paging in pig or mapreduce but you should be able to
> do that in both since pig allows you to specify the slice start.
>
> On Oct 11, 2012, at 11:28 AM, William Oberman 
> wrote:
>
> > If you don't mind me asking, how are you handling the fact that
> pre-widerow you are only getting a static number of columns per key
> (default 1024)?  Or am I not understanding the "limit" concept?
> >
> > On Thu, Oct 11, 2012 at 11:25 AM, Jeremy Hanna <
> jeremy.hanna1...@gmail.com> wrote:
> > The Dachis Group (where I just came from, now at DataStax) uses pig with
> cassandra for a lot of things.  However, we weren't using the widerow
> implementation yet since wide row support is new to 1.1.x and we were on
> 0.7, then 0.8, then 1.0.x.
> >
> > I think since it's new to 1.1's hadoop support, it sounds like there are
> some rough edges like you say.  But issues that are reproducible on tickets
> for any problems are much appreciated and they will get addressed.
> >
> > On Oct 11, 2012, at 10:43 AM, William Oberman 
> wrote:
> >
> > > I'm wondering how many people are using cassandra + pig out there?  I
> recently went through the effort of validating things at a much higher
> level than I previously did(*), and found a few issues:
> > > https://issues.apache.org/jira/browse/CASSANDRA-4748
> > > https://issues.apache.org/jira/browse/CASSANDRA-4749
> > > https://issues.apache.org/jira/browse/CASSANDRA-4789
> > >
> > > In general, it seems like the widerow implementation still has rough
> edges.  I'm concerned I'm not understanding why other people aren't using
> the feature, and thus finding these problems.  Is everyone else just
> setting a high static limit?  E.g.  LOAD 'cassandra://KEYSPACE/CF?limit=X"
> where X >= the max size of any key?  Is everyone else using data models
> that result in keys with # columns always less than 1024?  Do newer version
> of hadoop consume the cassandra API in a way that work around these issues?
>  I'm using CDH3 == hadoop 0.20.2, pig 0.8.1.
> > >
> > > (*) I took a random subsample of 50,000 keys of my production data
> (approx 1M total key/value pairs, some keys having only a single value and
> some having 1000's).  I then wrote both a pig script and simple procedural
> version of the pig script.  Then I compared the results.  Obviously I
> started with differences, though after locally patching my code to fix the
> above 3 bugs (though, really only two issues), I now (finally) get the same
> results.
> >
> >
> >
>
>

unsubscribe

[Siddiqui, Akmal]

unsubscribe 

Re: unnecessary tombstone's transmission during repair process

[Rob Coli]

On Thu, Oct 11, 2012 at 8:41 AM, Alexey Zotov  wrote:
> Value of DeletedColumn is a serialized local
> deletion time. We know that local deletion time can be different on
> different nodes for the same tombstone. So hashes of the same tombstone on
> different nodes will be different. Is it true?

Yes, this seems correct based on my understanding of the process of
writing tombstones.

> I think that local deletion time shouldn't be considered in hash's 
> calculation.

I think you are correct; the only thing that matters is whether the
tombstone exists or not. There may be something I am missing about why
the very-unlikely-to-be-identical value should be considered a merkle
tree failure.

https://issues.apache.org/jira/browse/CASSANDRA-2279

Seems related to this issue, fwiw.

> Is transmission of the equals tombstones during repair process a feature? :)
> or is it a bug?

I think it's a bug.

> If it's a bug, I'll create ticket and attach patch to it.

Yay!

=Rob

-- 
=Robert Coli
AIM>ALK - rc...@palominodb.com
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb

Re: unnecessary tombstone's transmission during repair process

[Sylvain Lebresne]

> I have a question about merkle tree construction and repair process. When
> mercle tree is constructing it calculates hashes. For DeletedColumn it
> calculates hash using value. Value of DeletedColumn is a serialized local
> deletion time.

The deletion time time is not local to each replica, it's computed
only once by the coordinator node that received the deletion
initially.

> We know that local deletion time can be different on
> different nodes for the same tombstone.

Given the above, no it cannot.

--
Sylvain

Re: cassandra 1.0.8 memory usage

[Jason Wee]

what jvm version?

On Thu, Oct 11, 2012 at 2:04 PM, Daniel Woo  wrote:

> Hi guys,
>
> I am running a mini cluster with 6 nodes, recently we see very frequent
> ParNewGC on two nodes. It takes 200 - 800 ms on average, sometimes it takes
> 5 seconds. You know, hte ParNewGC is stop-of-wolrd GC and our client throws
> SocketTimeoutException every 3 minutes.
>
> I checked the load, it seems well balanced, and the two nodes are running
> on the same hardware: 2 * 4 cores xeon with 16G RAM, we give cassandrda 4G
> heap, including 800MB young generation. We did not see any swap usage
> during the GC, any idea about this?
>
> Then I took a heap dump, it shows that 5 instances of JmxMBeanServer holds
> 500MB memory and most of the referenced objects are JMX mbean related, it's
> kind of wired to me and looks like a memory leak.
>
> --
> Thanks & Regards,
> Daniel
>

Re: cassandra 1.0.8 memory usage

[Rob Coli]

On Wed, Oct 10, 2012 at 11:04 PM, Daniel Woo  wrote:
> I am running a mini cluster with 6 nodes, recently we see very frequent
> ParNewGC on two nodes. It takes 200 - 800 ms on average, sometimes it takes
> 5 seconds. You know, hte ParNewGC is stop-of-wolrd GC and our client throws
> SocketTimeoutException every 3 minutes.

What version of Cassandra? What JVM? Are JNA and Jamm working?

> I checked the load, it seems well balanced, and the two nodes are running on
> the same hardware: 2 * 4 cores xeon with 16G RAM, we give cassandrda 4G
> heap, including 800MB young generation. We did not see any swap usage during
> the GC, any idea about this?

It sounds like the two nodes that are pathological right now have
exhausted the perm gen with actual non-garbage, probably mostly the
Bloom filters and the JMX MBeans.

> Then I took a heap dump, it shows that 5 instances of JmxMBeanServer holds
> 500MB memory and most of the referenced objects are JMX mbean related, it's
> kind of wired to me and looks like a memory leak.

Do you have a "large" number of ColumnFamilies? How large is the data
stored per node?

=Rob

-- 
=Robert Coli
AIM>ALK - rc...@palominodb.com
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb

Re: cassandra 1.0.8 memory usage

[Rob Coli]

On Thu, Oct 11, 2012 at 11:02 AM, Rob Coli  wrote:
> On Wed, Oct 10, 2012 at 11:04 PM, Daniel Woo  wrote:
>  We did not see any swap usage during the GC, any idea about this?

As an aside.. you shouldn't have swap enabled on a Cassandra node,
generally. As a simple example, if you have swap enabled and use the
off-heap row cache, the kernel might swap your row cache.

=Rob

-- 
=Robert Coli
AIM>ALK - rc...@palominodb.com
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb

Perlcassa - Perl Cassandra 'Client'

[Michael Kjellman]

Hi- A few months back I wrote a Perl client for Cassandra and I realized I 
never sent it out to this list. While I realize that while Perl is not the 
language du jour hopefully this will help someone else out. :) Code is 
periodically thrown up on CPAN but look at 
http://github.com/mkjellman/perlcassa for the most current version. It supports 
serialization/deserialization of validation classes, composite columns, as well 
as connection pooling (without using ResourcePool which fails miserably when 
run under mod_perl).

Best,
michael

'Like' us on Facebook for exclusive content and other resources on all 
Barracuda Networks solutions.
Visit http://barracudanetworks.com/facebook

Re: [problem with OOM in nodes]

[Hiller, Dean]

"Splitting one report to multiple rows is uncomfortably"

WHY?  Reading from N disks is way faster than reading from 1 disk.

I think in terms of PlayOrm and then explain the model you can use so I
think in objects first

Report {
  String uniqueId
  String reportName; //may be indexable and query able
  String description;
  CursorToMany rows;
}

ReportRow {
  String uniqueId;
  String somedata;
  String someMoreData;
}

Each row in Report in PlayOrm is backed by two rows in the database in
this special case of using CursorToMany

ReportRow -> reportName=somename, description="some desc"
CursorToManyRow in index table -> reportRowKey56, reportRowKey78,
reportRowKey89 (there are NO values in this row and this row can have less
than 10 million valuesŠif your report is beyond 10 million, let me know
and I have a different design).

Then each report row is basically the same structure as above.  You can
then 

1. Read in the report
2. As you read from CursorToMany, it does a BATCH slice into the
CursorToManyRow AND then does a MULTIGET in parallel to fetch report
rows(ie. It is all in parallel so get lots of rows from many disks really
fast)
3. Print the rows out

If you have more than 10 million rows in a report, let me know.  You can
do what PlayOrm does yourself of course ;).

Later,
Dean



On 9/23/12 11:14 PM, "Denis Gabaydulin"  wrote:

>On Sun, Sep 23, 2012 at 10:41 PM, aaron morton 
>wrote:
>> /var/log/cassandra$ cat system.log | grep "Compacting large" | grep -E
>> "[0-9]+ bytes" -o | cut -d " " -f 1 |  awk '{ foo = $1 / 1024 / 1024 ;
>> print foo "MB" }'  | sort -nr | head -n 50
>>
>>
>> Is it bad signal?
>>
>> Sorry, I do not know what this is outputting.
>>
>
>This is outputting size of big rows which cassandra had compacted before.
>
>> As I can see in cfstats, compacted row maximum size: 386857368 !
>>
>> Yes.
>> Having rows in the 100's of MB is will cause problems. Doubly so if
>>they are
>> large super columns.
>>
>
>What exactly is the problem with big rows?
>And, how can we should place our data in this case (see the schema in
>the previous replies)? Splitting one report to multiple rows is
>uncomfortably :-(
>
>
>> Cheers
>>
>>
>>
>> -
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 22/09/2012, at 5:07 AM, Denis Gabaydulin  wrote:
>>
>> And some stuff from log:
>>
>>
>> /var/log/cassandra$ cat system.log | grep "Compacting large" | grep -E
>> "[0-9]+ bytes" -o | cut -d " " -f 1 |  awk '{ foo = $1 / 1024 / 1024 ;
>> print foo "MB" }'  | sort -nr | head -n 50
>> 3821.55MB
>> 3337.85MB
>> 1221.64MB
>> 1128.67MB
>> 930.666MB
>> 916.4MB
>> 861.114MB
>> 843.325MB
>> 711.813MB
>> 706.992MB
>> 674.282MB
>> 673.861MB
>> 658.305MB
>> 557.756MB
>> 531.577MB
>> 493.112MB
>> 492.513MB
>> 492.291MB
>> 484.484MB
>> 479.908MB
>> 465.742MB
>> 464.015MB
>> 459.95MB
>> 454.472MB
>> 441.248MB
>> 428.763MB
>> 424.028MB
>> 416.663MB
>> 416.191MB
>> 409.341MB
>> 406.895MB
>> 397.314MB
>> 388.27MB
>> 376.714MB
>> 371.298MB
>> 368.819MB
>> 366.92MB
>> 361.371MB
>> 360.509MB
>> 356.168MB
>> 355.012MB
>> 354.897MB
>> 354.759MB
>> 347.986MB
>> 344.109MB
>> 335.546MB
>> 329.529MB
>> 326.857MB
>> 326.252MB
>> 326.237MB
>>
>> Is it bad signal?
>>
>> On Fri, Sep 21, 2012 at 8:22 PM, Denis Gabaydulin 
>>wrote:
>>
>> Found one more intersting fact.
>> As I can see in cfstats, compacted row maximum size: 386857368 !
>>
>> On Fri, Sep 21, 2012 at 12:50 PM, Denis Gabaydulin 
>> wrote:
>>
>> Reports - is a SuperColumnFamily
>>
>> Each report has unique identifier (report_id). This is a key of
>> SuperColumnFamily.
>> And a report saved in separate row.
>>
>> A report is consisted of report rows (may vary between 1 and 50,
>> but most are small).
>>
>> Each report row is saved in separate super column. Hector based code:
>>
>> superCfMutator.addInsertion(
>>  report_id,
>>  "Reports",
>>  HFactory.createSuperColumn(
>>report_row_id,
>>mapper.convertObject(object),
>>columnDefinition.getTopSerializer(),
>>columnDefinition.getSubSerializer(),
>>inferringSerializer
>>  )
>> );
>>
>> We have two frequent operation:
>>
>> 1. count report rows by report_id (calculate number of super columns
>> in the row).
>> 2. get report rows by report_id and range predicate (get super columns
>> from the row with range predicate).
>>
>> I can't see here a big super columns :-(
>>
>> On Fri, Sep 21, 2012 at 3:10 AM, Tyler Hobbs  wrote:
>>
>> I'm not 100% that I understand your data model and read patterns
>>correctly,
>> but it sounds like you have large supercolumns and are requesting some
>>of
>> the subcolumns from individual super columns.  If that's the case, the
>>issue
>> is that Cassandra must deserialize the entire supercolumn in memory
>>whenever
>> you read *any* of the subcolumns.  This is one of the reasons why
>>composite
>> columns are recommended over supercolumns.
>>
>>
>> On Thu, Sep 20, 2012 at 6:45 AM, Denis Gabaydulin 
>>wrote:
>>
>>
>> p.s.

CRM:0267190

[Joseph Heinzen]

unsubscribe

Joseph Heinzen
Senior VP, UCC Sales
Tel. 571-297-4162 | Mobile. 703-463-7145
Fax. 703-891-1073 | jhein...@microtech.net | 
www.MicroTech.net
[Description: C:\Users\joseph.heinzen\AppData\Roaming\Microsoft\Signatures\New 
Email Signature HQ v3 (Joseph Heinzen)-Image01.jpg]
The Fastest Growing Hispanic-Owned Business in the Nation (2009, 2010 & 2011)
8330 Boone Blvd, Suite 600, Vienna, VA 22182
A Service-Disabled Veteran-Owned Business
DISCLAIMER: The information in this email, and any attached document(s), is 
MicroTech proprietary data and intended only for recipient(s) addressed above. 
If you are not an intended recipient, you are requested to notify the sender 
above and delete any copies of this transmission. Thank you in advance for your 
cooperation. If you have any questions, please contact the sender at (703) 
891-1073.
<>

Re: can't get cqlsh running

[Tim Dunphy]

>
>
> I believe you can run 'python2.6 easy_install cql' to force it to use
> that python install.
>

Well initially I tried going:

[root@beta:~] #python2.6 easy_install
python2.6: can't open file 'easy_install': [Errno 2] No such file or
directory

But when I used the full paths of each:

/usr/bin/python2.6 /usr/bin/easy_install cql

It worked like a charm!

[root@beta:~] #cqlsh
Connected to Test Cluster at beta.jokefire.com:9160.
[cqlsh 2.0.0 | Cassandra unknown | CQL spec unknown | Thrift protocol
19.20.0]
Use HELP for help.
cqlsh>

So, thanks for your advice! That really did the trick!

Tim

On Thu, Oct 11, 2012 at 11:56 AM, Nick Bailey  wrote:

> It looks like easy_install is only recognizing python2.4 on your
> system. It is installing the cql module for that version. The cqlsh
> script explicitly looks for and runs with python2.6 since 2.4 isn't
> supported.
>
> I believe you can run 'python2.6 easy_install cql' to force it to use
> that python install.
>
>
> -Nick
>
> On Thu, Oct 11, 2012 at 10:45 AM, Tim Dunphy  wrote:
> > Hey guys,
> >
> >  I'm on cassandra 1.1.5 on a centos 5.8 machine. I have the cassandra bin
> > directory on my path so that i can simply type 'cassandra-cli' from
> anywhere
> > on my path to get into the cassandra command line environment. It's
> great!
> >
> > But I'd like to start using the cql shell (cqlsh) but apparently I don't
> > know enough about python to get this working. This is what happens when I
> > try to run cqlsh:
> >
> > [root@beta:~] #cqlsh
> >
> > Python CQL driver not installed, or not on PYTHONPATH.
> > You might try "easy_install cql".
> >
> > Python: /usr/bin/python2.6
> > Module load path: ['/usr/local/apache-cassandra-1.1.5/bin/../pylib',
> > '/usr/local/apache-cassandra-1.1.5/bin',
> > '/usr/local/apache-cassandra-1.1.5/bin', '/usr/lib64/python26.zip',
> > '/usr/lib64/python2.6', '/usr/lib64/python2.6/plat-linux2',
> > '/usr/lib64/python2.6/lib-tk', '/usr/lib64/python2.6/lib-old',
> > '/usr/lib64/python2.6/lib-dynload', '/usr/lib64/python2.6/site-packages',
> > '/usr/lib/python2.6/site-packages',
> > '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info']
> >
> > Error: No module named cql
> >
> > But easy_install claims that it's already installed:
> >
> > [root@beta:~] #easy_install cql
> > Searching for cql
> > Best match: cql 1.0.10
> > Processing cql-1.0.10-py2.4.egg
> > cql 1.0.10 is already the active version in easy-install.pth
> >
> > Using /usr/lib/python2.4/site-packages/cql-1.0.10-py2.4.egg
> > Processing dependencies for cql
> >
> > I'm thinking that I just don't know how to set the PYTHONPATH variable or
> > where to point it to. Can someone give me a pointer here?
> >
> > Thanks
> > Tim
> >
> > --
> > GPG me!!
> >
> > gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
> >
> >
>



-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B

Re: CRM:0267190

[Michael Shuler]

On 10/11/2012 02:37 PM, Joseph Heinzen wrote:
> unsubscribe

http://wiki.apache.org/cassandra/FAQ#unsubscribe

> *Joseph Heinzen*
> Senior VP, UCC Sales
> Tel. 571-297-4162 | Mobile. 703-463-7145 
> Fax. 703-891-1073 | _jhein...@microtech.net
> _ | _www.MicroTech.net
> _
> *Description:
> C:\Users\joseph.heinzen\AppData\Roaming\Microsoft\Signatures\New Email
> Signature HQ v3 (Joseph Heinzen)-Image01.jpg**
> *The Fastest Growing Hispanic-Owned Business in the Nation (2009, 2010 &
> 2011)
> 8330 Boone Blvd, Suite 600, Vienna, VA 22182
> A Service-Disabled Veteran-Owned Business

http://www.ietf.org/rfc/rfc1855.txt (re: subject; signature)

> /DISCLAIMER: The information in this email, and any attached
> document(s), is MicroTech proprietary data and intended only for
> recipient(s) addressed above. If you are not an intended recipient, you
> are requested to notify the sender above and delete any copies of this
> transmission. Thank you in advance for your cooperation. If you have any
> questions, please contact the sender at (703) 891-1073./

and finally, http://www.pbandjelly.org/2011/03/to-whom-it-may-concern/

-- 
Kind regards,
Michael

Re: unsubscribe

[Tyler Hobbs]

http://wiki.apache.org/cassandra/FAQ#unsubscribe

On Thu, Oct 11, 2012 at 12:41 PM, Siddiqui, Akmal <
akmal.siddi...@broadvision.com> wrote:

> unsubscribe
>



-- 
Tyler Hobbs
DataStax 

Re: READ messages dropped

[Tyler Hobbs]

On Wed, Oct 10, 2012 at 3:10 PM, Tamar Fraenkel  wrote:

>
> What I did noticed while looking at the logs (which are also running
> OpsCenter), is that there is some correlation between the dropped reads and
> flushes of OpsCenter column families to disk and or compactions. What are
> the rollups CFs? why is there so much traffic in them?


The rollups CFs hold the performance metric data that OpsCenter stores
about your cluster.  Typically these aren't actually very high traffic
column families, but that depends on how many column families you have
(more CFs require more metrics to be stored).  If you have a lot of column
families, you have a couple of options for reducing the amount of metric
data that's stored:
http://www.datastax.com/docs/opscenter/trouble_shooting_opsc#limiting-the-metrics-collected-by-opscenter

Assuming you don't have a large number of CFs, your nodes may legitimately
be nearing capacity.

-- 
Tyler Hobbs
DataStax 

Re: Option for ordering columns by timestamp in CF

[Tyler Hobbs]

Without thinking too deeply about it, this is basically equivalent to
disabling timestamps for a column family and using timestamps for column
names, though in a very indirect (and potentially confusing) manner.  So,
if you want to open a ticket, I would suggest framing it as "make column
timestamps optional".

On Wed, Oct 10, 2012 at 4:44 AM, Ertio Lew  wrote:

> I think Cassandra should provide an configurable option on per column
> family basis to do columns sorting by time-stamp rather than column names.
> This would be really helpful to maintain time-sorted columns without using
> up the column name as time-stamps which might otherwise be used to store
> most relevant column names useful for retrievals. Very frequently we need
> to store data sorted in time order. Therefore I think this may be a very
> general requirement & not specific to just my use-case alone.
>
> Does it makes sense to create an issue for this ?
>
>
>
>
> On Fri, Mar 25, 2011 at 2:38 AM, aaron morton wrote:
>
>> If you mean order by the column timestamp (as passed by the client) that
>> it not possible.
>>
>> Can you use your own timestamps as the column name and store them as long
>> values ?
>>
>> Aaron
>>
>> On 25 Mar 2011, at 09:30, Narendra Sharma wrote:
>>
>> > Cassandra 0.7.4
>> > Column names in my CF are of type byte[] but I want to order columns by
>> timestamp. What is the best way to achieve this? Does it make sense for
>> Cassandra to support ordering of columns by timestamp as option for a
>> column family irrespective of the column name type?
>> >
>> > Thanks,
>> > Naren
>>
>>
>


-- 
Tyler Hobbs
DataStax 

RE: unsubscribe

[Siddiqui, Akmal]

http://wiki.apache.org/cassandra/FAQ#unsubscribe 

-Original Message-
From: Chris Favero [mailto:chris.fav...@tricast.com] 
Sent: Thursday, October 11, 2012 7:37 AM
To: user@cassandra.apache.org
Subject: unsubscribe

unsubscribe

RE: unsubscribe

[Siddiqui, Akmal]

thanks



From: Tyler Hobbs [mailto:ty...@datastax.com] 
Sent: Thursday, October 11, 2012 1:42 PM
To: user@cassandra.apache.org
Subject: Re: unsubscribe


http://wiki.apache.org/cassandra/FAQ#unsubscribe


On Thu, Oct 11, 2012 at 12:41 PM, Siddiqui, Akmal
 wrote:


unsubscribe





-- 
Tyler Hobbs
DataStax  

Repair Failing due to bad network

[David Koblas]

I'm trying to bring up a new Datacenter - while I probably could have 
brought things up in another way I've now got a DC that has a ready 
Cassandra with keys allocated.  The problem is that I cannot get a 
repair to complete due since it appears that some part of my network 
decides to restart all connections twice a day (6am and 2pm - ok 5 
minutes before).


So when I start a repair job, it usually get's a ways into things before 
one of the nodes goes DOWN, then back up.  What I don't see is the 
repair restarting, it just stops.


Is there a workaround for this case, or is there something else I could 
be doing?


--david

Re: Repair Failing due to bad network

[Jim Cistaro]

I am not aware of any built-in mechanism for retrying repairs.  I believe
you will have to build that into your process.

As for reducing the time of each repair command to fit in your windows:

If you have multiple reasonable size column families, and are not already
doing this, one approach might be to do repairs on a per cf basis.  This
will break your repairs up into smaller chunks that might fit in the
window.

If you are not doing -pr (primary range), using that on each node causes
the repair command to only repair the primary range on the node (not the
ones it is replicating).

Depending on your version, there is also
https://issues.apache.org/jira/browse/CASSANDRA-3912 which might help you
- but I have no experience using this feature.

jc

On 10/11/12 4:09 PM, "David Koblas"  wrote:

>I'm trying to bring up a new Datacenter - while I probably could have
>brought things up in another way I've now got a DC that has a ready
>Cassandra with keys allocated.  The problem is that I cannot get a
>repair to complete due since it appears that some part of my network
>decides to restart all connections twice a day (6am and 2pm - ok 5
>minutes before).
>
>So when I start a repair job, it usually get's a ways into things before
>one of the nodes goes DOWN, then back up.  What I don't see is the
>repair restarting, it just stops.
>
>Is there a workaround for this case, or is there something else I could
>be doing?
>
>--david
>

Re: Perlcassa - Perl Cassandra 'Client'

[Watanabe Maki]

Perl never die... As Fortran

From iPhone


On 2012/10/12, at 3:09, Michael Kjellman  wrote:

> Hi- A few months back I wrote a Perl client for Cassandra and I realized I 
> never sent it out to this list. While I realize that while Perl is not the 
> language du jour hopefully this will help someone else out. :) Code is 
> periodically thrown up on CPAN but look at 
> http://github.com/mkjellman/perlcassa for the most current version. It 
> supports serialization/deserialization of validation classes, composite 
> columns, as well as connection pooling (without using ResourcePool which 
> fails miserably when run under mod_perl).
> 
> Best,
> michael
> 
> -- 
> 'Like' us on Facebook for exclusive content and other resources on all 
> Barracuda Networks solutions.
> Visit http://barracudanetworks.com/facebook
>   ­­  

Cassandra nodes loaded unequally

[Ben Kaehne]

Good morning,

I am running a simple 3 node cluster.
Each node is relatively powerful. (16 core, lots of ram, good disk etc).
All machines are identical.
Token ranges are all equal (generated with the python script on datastax
website)

Although under heavy load I can see the "unix load" be rather uneven
between the machines. (ie one machine may be 1/5th of the load of the
others double digits vs single digits).

Where would you recommend me to start tracking this down?

Regards,

-- 
-Ben

Re: 1.1.1 is "repair" still needed ?

[Watanabe Maki]

Oh sorry. It's pretty nice to know that.


On 2012/10/12, at 0:18, "B. Todd Burruss"  wrote:

> as of 1.0 (CASSANDRA-2034) hints are generated for nodes that timeout.
> 
> On Thu, Oct 11, 2012 at 3:55 AM, Watanabe Maki  
> wrote:
>> Even if HH works fine, HH will not be created until the failure detector 
>> marks  the node is dead.
>> HH will not be created for partially timeouted mutation request ( but meets 
>> CL ) also... In my understanding...
>> 
>> 
>> On 2012/10/11, at 5:55, Rob Coli  wrote:
>> 
>>> On Tue, Oct 9, 2012 at 12:56 PM, Oleg Dulin  wrote:
 My understanding is that the repair has to happen within gc_grace period.
 [ snip ]
 So the question is, is this still needed ? Do we even need to run nodetool
 repair ?
>>> 
>>> If Hinted Handoff works in your version of Cassandra, and that version
>>> is > 1.0, you "should" not need to repair if no node has crashed or
>>> been down for longer than max_hint_window_in_ms. This is because after
>>> 1.0, any failed write to a remote replica results in a hint, so any
>>> DELETE should eventually be fully replicated.
>>> 
>>> However hinted handoff is meaningfully broken between 1.1.0 and 1.1.6
>>> (unreleased) so you cannot rely on the above heuristic for
>>> consistency. In these versions, you have to repair (or read repair
>>> 100% of keys) once every GCGraceSeconds to prevent the possibility of
>>> zombie data. If it were possible to repair on a per-columnfamily
>>> basis, you could get a significant win by only repairing
>>> columnfamilies which take DELETE traffic.
>>> 
>>> https://issues.apache.org/jira/browse/CASSANDRA-4772
>>> 
>>> =Rob
>>> 
>>> --
>>> =Robert Coli
>>> AIM>ALK - rc...@palominodb.com
>>> YAHOO - rcoli.palominob
>>> SKYPE - rcoli_palominodb

Re: Option for ordering columns by timestamp in CF

[Ertio Lew]

"Make column timestamps optional"- kidding me, right ?:)  I do understand
that this wont be possible as then cassandra wont be able to distinguish
the latest among several copies of same column. I dont mean that. I just
want the while ordering the columns, Cassandra(in an optional mode per CF)
should not look at column names(they will exist though but for retrieval
purposes not for ordering) but instead Cassandra would order the columns by
looking at the timestamp values(timestamps would exist!). So the change
would be just to provide a mode in which cassandra, while ordering, uses
timestamps instead of column names.

On Fri, Oct 12, 2012 at 2:26 AM, Tyler Hobbs  wrote:

> Without thinking too deeply about it, this is basically equivalent to
> disabling timestamps for a column family and using timestamps for column
> names, though in a very indirect (and potentially confusing) manner.  So,
> if you want to open a ticket, I would suggest framing it as "make column
> timestamps optional".
>
>
> On Wed, Oct 10, 2012 at 4:44 AM, Ertio Lew  wrote:
>
>> I think Cassandra should provide an configurable option on per column
>> family basis to do columns sorting by time-stamp rather than column names.
>> This would be really helpful to maintain time-sorted columns without using
>> up the column name as time-stamps which might otherwise be used to store
>> most relevant column names useful for retrievals. Very frequently we need
>> to store data sorted in time order. Therefore I think this may be a very
>> general requirement & not specific to just my use-case alone.
>>
>> Does it makes sense to create an issue for this ?
>>
>>
>>
>>
>> On Fri, Mar 25, 2011 at 2:38 AM, aaron morton wrote:
>>
>>> If you mean order by the column timestamp (as passed by the client) that
>>> it not possible.
>>>
>>> Can you use your own timestamps as the column name and store them as
>>> long values ?
>>>
>>> Aaron
>>>
>>> On 25 Mar 2011, at 09:30, Narendra Sharma wrote:
>>>
>>> > Cassandra 0.7.4
>>> > Column names in my CF are of type byte[] but I want to order columns
>>> by timestamp. What is the best way to achieve this? Does it make sense for
>>> Cassandra to support ordering of columns by timestamp as option for a
>>> column family irrespective of the column name type?
>>> >
>>> > Thanks,
>>> > Naren
>>>
>>>
>>
>
>
> --
> Tyler Hobbs
> DataStax 
>
>