Re: help needed interpreting Read/Write latency in cfstats and cfhistograms output

[aaron morton]

ms is for microseconds

To get a handle on what is happening I would run them both first to reset the 
recent counts. Then run them and see if they make sense.

Cheers

-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 4/10/2011, at 9:57 AM, Ramesh Natarajan wrote:

> Thanks Aaron. The ms in the latency is it microseconds or milliseconds? 
> 
> I ran the 2 commands at the same time. I was expecting the values to be in 
> the some what similar but from my output earlier ,  you can see the median in 
> read latency in histogram output is about 10 milliseconds whereas the cfstats 
> showed 5 ms.  Is this normal?
> 
> thanks
> Ramesh
> 
> 
> On Mon, Oct 3, 2011 at 3:40 PM, aaron morton  wrote:
> Hi Rameash,
> 
>Both tools output the "recent" latency, and while they do this 
> slightly differently, the result is that it's the latency since the last time 
> it was checked. Also the two tools use different counters, so using cfstats 
> will not update cfhistogram.
> 
> 
>So when you see
> >  Read Latency: 5.086 ms.
> >  Write Latency: 0.018 ms.
> It means since you last checked the average latency for requests was 5.086 
> and 0.018ms
> 
> When you see
> > Offset  SSTables Write Latency  Read Latency  Row Size  
> > Column Count
> > 14148086 9198896 0  
> >0
> 
> it means that 198,896 read requests were completed in 1 *microsecond* and 9 
> write requests completed n 1 microsecond.
> 
> Cheers
> 
> 
> -
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 4/10/2011, at 4:58 AM, Ramesh Natarajan wrote:
> 
> > I am running a cassandra 0.8.6 cluster. I started a clean test setup and 
> > run my tests for a while. Later when I run cfstats and cfhistograms ( both 
> > ran at the same time )
> > the values for Read/Write latency doesn't match.   As per  cfstats  the 
> > latency for read and write are 5.086 and 0.018 ms respectively. However per 
> > cfhistogram output the
> > latency doesn't look correct. Attached are the output.. Can someone explain 
> > how to correlate the data?
> >
> > Thanks
> > Ramesh
> >
> >
> > cfstats
> > Column Family: uid
> > SSTable count: 10
> > Space used (live): 7453864915
> > Space used (total): 7453864915
> > Number of Keys (estimate): 2669184
> > Memtable Columns Count: 6864
> > Memtable Data Size: 9254197
> > Memtable Switch Count: 1037
> > Read Count: 353627031
> > Read Latency: 5.086 ms.
> > Write Count: 325803780
> > Write Latency: 0.018 ms.
> > Pending Tasks: 0
> > Key cache capacity: 200
> > Key cache size: 200
> > Key cache hit rate: 0.8106968059650433
> > Row cache: disabled
> > Compacted row minimum size: 104
> > Compacted row maximum size: 11864
> > Compacted row mean size: 2629
> >
> >
> > cfhistograms for uid
> > MSA/uid histograms
> > Offset  SSTables Write Latency  Read Latency  Row Size  
> > Column Count
> > 14148086 9198896 0  
> >0
> > 28680130 2993805 0  
> >0
> > 3   17138720 2   2487034 0  
> >0
> > 4   29039539 3   4712246 0  
> >0
> > 5   4192539220   7805708 0  
> >0
> > 6   52669945   126  11641747 0  
> >0
> > 7   57474130   457  15812298 0  
> >0
> > 8   53613212  1034  19846340 0  
> >0
> > 10  67641689  5463  48797478 0  
> >0
> > 12  19456795 15703  52875124 0  
> >0
> > 14   1841556 35196  47573455 0  
> >0
> > 17  3095102787  51065577 0  
> >0
> > 20 0145706  27439942 0  
> >0
> > 24 0196614  14573201 0  
> >0
> > 29 0237579   4983641 0  
> >0
> > 35 0489150   2167481 0  
> >0
> > 4

Re: Running on Windows

[aaron morton]

I'm guessing things here without checking, but some issues may be:

* pretty sure JNA mlockall() to lock the cassandra memory from swapping is not 
available https://issues.apache.org/jira/browse/CASSANDRA-1214

* Creating hard links for the snapshots will be different, not sure exactly how 
different 
https://github.com/apache/cassandra/blob/cassandra-0.8.6/src/java/org/apache/cassandra/utils/CLibrary.java#L160

* features that try to avoid polluting the os disk cache will not work 

* off heap row cache ??

* You will have a greater chance of finding help on *nix than windows.   

Hope that helps. 

-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 4/10/2011, at 10:31 AM, Bryce Godfrey wrote:

> I’m wondering what the consensus is for running a Cassandra cluster on top of 
> Windows boxes?  We are currently running a small 5 node cluster on top of 
> CentOS without problems, so I have no desire to move.  But we are a windows 
> shop, and I have an IT department that is scared of Linux since they don’t 
> know how to manage it.
> 
> My primary thoughts of why not, was just community support (haven’t seen or 
> heard of  anybody else doing it on Windows), performance, and stability.  The 
> last two are mostly guesses by me, but my thoughts was that java on windows 
> just does not perform as well.  We have a very high right load, and are 
> adding about 5 GB a day of data with a 3 month retention.
>  
> I really don’t want to move a stable system onto an unknown just out of fear 
> of the unknown from my IT department, so looking for some ammo.  Thanks J
>  
> ~Bryce

[RELEASE CANDIDATE] Apache Cassandra 1.0.0-rc2 released

[Sylvain Lebresne]

The first release candidate happened to have some important regressions, so we
decided to release a new release candidate. So here it is: Apache
Cassandra 1.0.0-rc2.

This is still *not* the final release and is thus not yet considered ready for
production.

As always, your help in testing this release candidate would be highly
appreciated and while doing so, please report any problem you may
encounter[3,4]. The changes since the rc1 can be found in the change log[1]
and see the release notes[2] to find what Cassandra 1.0 is made of.

Apache Cassandra 1.0.0-rc2[5] is available as usual from the cassandra
website:

 http://cassandra.apache.org/download/

Thank you for your help in testing and have fun with it. Hopefully, Cassandra
1.0 final is next.

[1]: http://goo.gl/PxL3V (CHANGES.txt)
[2]: http://goo.gl/R5eVn (NEWS.txt)
[3]: https://issues.apache.org/jira/browse/CASSANDRA
[4]: user@cassandra.apache.org
[5]: https://svn.apache.org/repos/asf/cassandra/tags/cassandra-1.0.0-rc2

Re: nodetool cfstats on 1.0.0-rc1 throws an exception

[aaron morton]

That row has a size of 819 peta bytes, so something is odd there. The error is 
a result of that value been so huge. When you rant he same script on 0.8.6 what 
was the max size of the Migrations CF ?

As Jonathan says, it's unlikely anyone would have tested creating 5000 CF's. 
Most people only create a few 10's of CF's at most.

either use fewer CF's or…

* dump the Migrations CF using sstable2json to take a look around 
* work out steps to reproduce and report it on Jira

Hope that helps. 

-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 4/10/2011, at 11:30 AM, Ramesh Natarajan wrote:

> We recreated the schema using the same input file on both clusters and they 
> are running identical load.  
> 
> Isn't the exception thrown in the system CF?
> 
> this line looks strange:
> 
> Compacted row maximum size: 9223372036854775807
> 
> thanks
> Ramesh
> 
> On Mon, Oct 3, 2011 at 5:26 PM, Jonathan Ellis  wrote:
> Looks like you have unexpectedly large rows in your 1.0 cluster but
> not 0.8.  I guess you could use sstable2json to manually check your
> row sizes.
> 
> On Mon, Oct 3, 2011 at 5:20 PM, Ramesh Natarajan  wrote:
> > It happens all the time on 1.0. It doesn't happen on 0.8.6.  Is there any
> > thing I can do to check?
> > thanks
> > Ramesh
> >
> > On Mon, Oct 3, 2011 at 5:15 PM, Jonathan Ellis  wrote:
> >>
> >> My suspicion would be that it has more to do with "rare case when
> >> running with 5000 CFs" than "1.0 regression."
> >>
> >> On Mon, Oct 3, 2011 at 5:00 PM, Ramesh Natarajan 
> >> wrote:
> >> > We have about 5000 column family and when we run the nodetool cfstats it
> >> > throws out this exception...  this is running 1.0.0-rc1
> >> > This seems to work on 0.8.6.  Is this a bug in 1.0.0?
> >> >
> >> > thanks
> >> > Ramesh
> >> > Keyspace: system
> >> > Read Count: 28
> >> > Read Latency: 5.8675 ms.
> >> > Write Count: 3
> >> > Write Latency: 0.166 ms.
> >> > Pending Tasks: 0
> >> > Column Family: Schema
> >> > SSTable count: 4
> >> > Space used (live): 4293758276
> >> > Space used (total): 4293758276
> >> > Number of Keys (estimate): 5376
> >> > Memtable Columns Count: 0
> >> > Memtable Data Size: 0
> >> > Memtable Switch Count: 0
> >> > Read Count: 3
> >> > Read Latency: NaN ms.
> >> > Write Count: 0
> >> > Write Latency: NaN ms.
> >> > Pending Tasks: 0
> >> > Key cache capacity: 53
> >> > Key cache size: 2
> >> > Key cache hit rate: NaN
> >> > Row cache: disabled
> >> > Compacted row minimum size: 104
> >> > Compacted row maximum size: 1955666
> >> > Compacted row mean size: 1508515
> >> > Column Family: HintsColumnFamily
> >> > SSTable count: 0
> >> > Space used (live): 0
> >> > Space used (total): 0
> >> > Number of Keys (estimate): 0
> >> > Memtable Columns Count: 0
> >> > Memtable Data Size: 0
> >> > Memtable Switch Count: 0
> >> > Read Count: 5
> >> > Read Latency: NaN ms.
> >> > Write Count: 0
> >> > Write Latency: NaN ms.
> >> > Pending Tasks: 0
> >> > Key cache capacity: 1
> >> > Key cache size: 0
> >> > Key cache hit rate: NaN
> >> > Row cache: disabled
> >> > Compacted row minimum size: 0
> >> > Compacted row maximum size: 0
> >> > Compacted row mean size: 0
> >> > Column Family: LocationInfo
> >> > SSTable count: 1
> >> > Space used (live): 6947
> >> > Space used (total): 6947
> >> > Number of Keys (estimate): 128
> >> > Memtable Columns Count: 0
> >> > Memtable Data Size: 0
> >> > Memtable Switch Count: 2
> >> > Read Count: 20
> >> > Read Latency: NaN ms.
> >> > Write Count: 3
> >> > Write Latency: NaN ms.
> >> > Pending Tasks: 0
> >> > Key cache capacity: 1
> >> > Key cache size: 1
> >> > Key cache hit rate: NaN
> >> > Row cache: disabled
> >> > Compacted row minimum size: 73
> >> > Compacted row maximum size: 258
> >> > Compacted row mean size: 185
> >> > Column Family: Migrations
> >> > SSTable count: 4
> >> > Space used (live): 4315909643
> >> > Space used (total): 43159096

Re: Weird problem with empty CF

[aaron morton]

Yes that's the slice query skipping past the tombstone columns. 

Cheers

-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 4/10/2011, at 4:24 PM, Daning Wang wrote:

> Lots of SliceQueryFilter in the log, is that handling tombstone?
> 
> DEBUG [ReadStage:49] 2011-10-03 20:15:07,942 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317582939743663:true:4@1317582939933000
> DEBUG [ReadStage:50] 2011-10-03 20:15:07,942 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317573253148778:true:4@1317573253354000
> DEBUG [ReadStage:43] 2011-10-03 20:15:07,942 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317669552951428:true:4@1317669553018000
> DEBUG [ReadStage:33] 2011-10-03 20:15:07,942 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317581886709261:true:4@1317581886957000
> DEBUG [ReadStage:52] 2011-10-03 20:15:07,942 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317568165152246:true:4@1317568165482000
> DEBUG [ReadStage:36] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317567265089211:true:4@1317567265405000
> DEBUG [ReadStage:53] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317674324843122:true:4@1317674324946000
> DEBUG [ReadStage:38] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317571990078721:true:4@1317571990141000
> DEBUG [ReadStage:57] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317671855234221:true:4@1317671855239000
> DEBUG [ReadStage:54] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317558305262954:true:4@1317558305337000
> DEBUG [RequestResponseStage:11] 2011-10-03 20:15:07,941 
> ResponseVerbHandler.java (line 48) Processing response on a callback from 
> 12347@/10.210.101.104
> DEBUG [RequestResponseStage:9] 2011-10-03 20:15:07,941 
> AbstractRowResolver.java (line 66) Preprocessed data response
> DEBUG [RequestResponseStage:13] 2011-10-03 20:15:07,941 
> AbstractRowResolver.java (line 66) Preprocessed digest response
> DEBUG [ReadStage:58] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317581337972739:true:4@1317581338044000
> DEBUG [ReadStage:64] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317582656796332:true:4@131758265697
> DEBUG [ReadStage:55] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317569432886284:true:4@1317569432984000
> DEBUG [ReadStage:45] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317572658687019:true:4@1317572658718000
> DEBUG [ReadStage:47] 2011-10-03 20:15:07,940 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317582281617755:true:4@1317582281717000
> DEBUG [ReadStage:48] 2011-10-03 20:15:07,940 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 1317549607869226:true:4@1317549608118000
> DEBUG [ReadStage:34] 2011-10-03 20:15:07,940 SliceQueryFilter.java (line 123) 
> collecting 0 of 1: 
> On Thu, Sep 29, 2011 at 2:17 PM, aaron morton  wrote:
> As with any situation involving the un-dead, it really is the number of 
> Zombies, Mummies or Vampires that is the concern.  
> 
> If you delete data there will always be tombstones. If you have a delete 
> heavy workload there will be more tombstones. This is why implementing a 
> queue with cassandra is a bad idea.
> 
> gc_grace_seconds (and column TTL) are the *minimum* about of time the 
> tombstones will stay in the data files, there is no maximum. 
> 
> Your read performance also depends on the number of SSTables the row is 
> spread over, see http://thelastpickle.com/2011/04/28/Forces-of-Write-and-Read/
> 
> If you really wanted to purge them then yes a repair and then major 
> compaction would be the way to go. Also consider if it's possible to design 
> the data model around the problem, e.g. partitioning rows by date. IMHO I 
> would look to make data model changes before implementing a compaction 
> policy, or consider if cassandra is the right store if you have a delete 
> heavy workload.
> 
> Cheers
> 
>  
> -
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 30/09/2011, at 3:27 AM, Daning Wang wrote:
> 
>> Jonathan/Aaron,
>> 
>> Thank you guy's reply, I will change GCGracePeriod to 1 day to see what will 
>> happen.
>> 
>> Is there a way to purge tombstones at anytime? because if tombstones affect 
>> performance, we want them to be purged right away, not after GCGracePeriod. 
>> We know all the nodes are up, and we can do repair first to make sure the 
>> consistency before purging.
>> 
>> Thanks,
>> 
>> Daning
>> 
>> 
>> On Wed, Sep 28, 2011 at 5:22 PM, aaron morton  
>> wrote:
>> if I had to guess I would say it was spending time handling tombstones. If 
>> you see it happen again, and are interested, turn the logging up to DEBUG 
>> 

RE: invalid column name length 0

[Desimpel, Ignace]

I run the application with the JVM -ea option, so assertions are enabled.

I insert records using the StorageProxy.mutate function. The elements are 
created  as specified below. 
Below : The arForwardFuncValueBytes and arReverseFuncValueBytes are tested for 
null or length = 0 by my code. The oTokenColumnName bytebuffer is created each 
time, but is reused in the two QueryPaths. I assume this is allowed.

QueryPath oPathtoInsert = new QueryPath( sForwardColumnFamToAdd, 
ByteBuffer.wrap(arForwardFuncValueBytes), oTokenColumnName);
oForwardRowMut.add(oPathtoInsert, oTokenPos, lUpdateTimeStamp);

oPathtoInsert = new QueryPath(sReverseColumnFamToAdd, 
ByteBuffer.wrap(arReverseFuncValueBytes), oTokenColumnName);
oReverseRowMut.add(oPathtoInsert, oTokenPos, lUpdateTimeStamp);

Anyway, I will do a test inserting the same data, but via thrift and with 
Cassandra in a separate jvm.

Ignace

-Original Message-
From: Sylvain Lebresne [mailto:sylv...@datastax.com] 
Sent: maandag 3 oktober 2011 18:02
To: user@cassandra.apache.org
Subject: Re: invalid column name length 0

On the 'invalid column name length 0' exception, since you're embedding the 
Cassandra server, it could be that you modify a column ByteBuffer that you feed 
to Cassandra (that's fairly easy to do with ByteBuffer by calling some relative 
get method of ByteBuffer). Or more generally that you feed a zero length 
ByteBuffer as a column name (maybe by using a relative put without a 
rewind/reset afterwards).

Which leads me to a question: do you run your server without assertions enabled 
? (I suspect you do).
If so I suggest you turn them on (to help you find the problem). It turns out 
that we detect zero length column name at write time in an assertion, while we 
detect them at read time using a good old 'if'. So if you do feed a zero length 
column name to Cassandra throught the StorageService interface, you'd only get 
the exception you get at read time.

Now I don't know how much those exceptions are related to the timeoutException 
you're seeing, but such error would typically produce timeout on reads whatever 
the rpc_timeout value is.

--
Sylvain

On Mon, Oct 3, 2011 at 4:58 PM, Desimpel, Ignace  
wrote:
> I did an extra test, again starting from scratch but with replication factor 
> 1.
> I still get the dead/up messages and timeout exceptions, but the system keeps 
> running and storing. However I ran out of disk space, logically producing a 
> lot of other errors.
> Then I restarted the Cassandra servers, so they were able to cleanup and 
> restart without errors.
> Then I did some queries I normally do and got again exceptions like " invalid 
> column name length 0", but also other like " Corrupt (negative) value length 
> encountered".
> Exception : see below.
>
> With this test, I run Cassandra embedded, so a lot of processing ( and object 
> allocations ) are done within the same JVM. I will modify the code so that 
> 'my processing/allcations' are done outside and the Cassandra jvm only has to 
> store the records. But that's for tomorrow.
>
> Did anyone ran into this type of error? And what was the reason? Any help?
>
> 2011-10-03 11:49:21.035 Fatal exception in thread 
> Thread[ReadStage:623,5,main]
> java.io.IOError: java.io.IOException: Corrupt (negative) value length 
> encountered
>        at 
> org.apache.cassandra.io.util.ColumnIterator.deserializeNext(ColumnSort
> edMap.java:265) ~[apache-cassandra-0.8.6.jar:0.8.6]
>        at 
> org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:
> 281) ~[apache-cassandra-0.8.6.jar:0.8.6]
>        at 
> org.apache.cassandra.io.util.ColumnIterator.next(ColumnSortedMap.java:
> 236) ~[apache-cassandra-0.8.6.jar:0.8.6]
>        at 
> java.util.concurrent.ConcurrentSkipListMap.buildFromSorted(ConcurrentS
> kipListMap.java:1493) ~[na:1.6.0_24]
>        at 
> java.util.concurrent.ConcurrentSkipListMap.(ConcurrentSkipListMa
> p.java:1443) ~[na:1.6.0_24]
>        at 
> org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.
> java:445) ~[apache-cassandra-0.8.6.jar:0.8.6]
>        at 
> org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.
> java:428) ~[apache-cassandra-0.8.6.jar:0.8.6]
>        at 
> org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.
> java:418) ~[apache-cassandra-0.8.6.jar:0.8.6]
>        at 
> org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.
> java:380) ~[apache-cassandra-0.8.6.jar:0.8.6]
>        at 
> org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlock
> Fetcher.getNextBlock(IndexedSliceReader.java:179) 
> ~[apache-cassandra-0.8.6.jar:0.8.6]
>        at 
> org.apache.cassandra.db.columniterator.IndexedSliceReader.computeNext(
> IndexedSliceReader.java:121) ~[apache-cassandra-0.8.6.jar:0.8.6]
>        at 
> org.apache.cassandra.db.columniterator.IndexedSliceReader.computeNext(
> IndexedSliceReader.java:49) ~[apache-cassandra-0.8.6.jar:0.8.6]
>        at 
> com.goog

Does anybody know why Twitter stop integrate Cassandra as Twitter store?

[ruslan usifov]

http://engineering.twitter.com/2010/07/cassandra-at-twitter-today.html

As said in this post Twiter stop working on using Cassandra as a store for
Tweets, but there nothing said why they made this decision? Does anybody
have mo information

How to speed up "Waiting for schema agreement" for a single node Cassandra cluster?

[Joseph Norton]

Hello.

For unit test purposes, I have a single node Cassandra cluster.  I need to drop 
and re-create several keyspaces between each test iteration.  This process 
takes approximately 10 seconds for a single node installation.

Can you recommend any tricks or recipes to reduce the time required for such 
operations and/or for "Waiting for schema agreement" to complete?

regards,

- Joe N.




$ time ./setupDB.sh 
Deleteing cassandra keyspaces
Connected to: "Foo" on 127.0.0.1/9160
ed9c7fc0-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
ee8c36f0-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
eeb14b20-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
Insert data
Creating cassandra keyspaces
Connected to: "Foo" on 127.0.0.1/9160
ef1a6d30-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
Authenticated to keyspace: Bars
ef4af310-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
ef9bab20-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
efbceec0-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
f00e4310-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
f0589280-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
f0821380-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
f0c44ca0-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
Authenticated to keyspace: Baz
f121d5f0-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
f1619e10-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
f18b4620-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
Authenticated to keyspace: Buz
f1debd50-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
f20690a0-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
f25043d0-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
f29a1e10-ee91-11e0--534d24a6e7f7
Waiting for schema agreement...
... schemas agree across the cluster
Inserting data in cassandra
Connected to: "Foo" on 127.0.0.1/9160
Authenticated to keyspace: Boo
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.
Value inserted.

real0m9.554s
user0m2.729s
sys 0m0.194s


Joseph Norton
nor...@alum.mit.edu

Re: How to speed up "Waiting for schema agreement" for a single node Cassandra cluster?

[Jonathan Ellis]

Truncate is faster than drop + recreate.

On Tue, Oct 4, 2011 at 9:15 AM, Joseph Norton  wrote:
>
> Hello.
>
> For unit test purposes, I have a single node Cassandra cluster.  I need to 
> drop and re-create several keyspaces between each test iteration.  This 
> process takes approximately 10 seconds for a single node installation.
>
> Can you recommend any tricks or recipes to reduce the time required for such 
> operations and/or for "Waiting for schema agreement" to complete?
>
> regards,
>
> - Joe N.
>
>
>
>
> $ time ./setupDB.sh
> Deleteing cassandra keyspaces
> Connected to: "Foo" on 127.0.0.1/9160
> ed9c7fc0-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> ee8c36f0-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> eeb14b20-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> Insert data
> Creating cassandra keyspaces
> Connected to: "Foo" on 127.0.0.1/9160
> ef1a6d30-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> Authenticated to keyspace: Bars
> ef4af310-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> ef9bab20-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> efbceec0-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> f00e4310-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> f0589280-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> f0821380-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> f0c44ca0-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> Authenticated to keyspace: Baz
> f121d5f0-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> f1619e10-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> f18b4620-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> Authenticated to keyspace: Buz
> f1debd50-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> f20690a0-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> f25043d0-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> f29a1e10-ee91-11e0--534d24a6e7f7
> Waiting for schema agreement...
> ... schemas agree across the cluster
> Inserting data in cassandra
> Connected to: "Foo" on 127.0.0.1/9160
> Authenticated to keyspace: Boo
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
> Value inserted.
>
> real    0m9.554s
> user    0m2.729s
> sys     0m0.194s
>
>
> Joseph Norton
> nor...@alum.mit.edu
>
>
>
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: How to speed up "Waiting for schema agreement" for a single node Cassandra cluster?

[Joseph Norton]

I didn't consider using truncate because a set of potentially random Column 
Families are created dynamically during the test.

Are there any configuration knobs that could be adjusted for drop + recreate?

thanks in advance,

- Joe N


Joseph Norton
nor...@alum.mit.edu



On Oct 4, 2011, at 11:19 PM, Jonathan Ellis wrote:

> Truncate is faster than drop + recreate.
> 
> On Tue, Oct 4, 2011 at 9:15 AM, Joseph Norton  
> wrote:
>> 
>> Hello.
>> 
>> For unit test purposes, I have a single node Cassandra cluster.  I need to 
>> drop and re-create several keyspaces between each test iteration.  This 
>> process takes approximately 10 seconds for a single node installation.
>> 
>> Can you recommend any tricks or recipes to reduce the time required for such 
>> operations and/or for "Waiting for schema agreement" to complete?
>> 
>> regards,
>> 
>> - Joe N.
>> 
>> 
>> 
>> 
>> $ time ./setupDB.sh
>> Deleteing cassandra keyspaces
>> Connected to: "Foo" on 127.0.0.1/9160
>> ed9c7fc0-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> ee8c36f0-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> eeb14b20-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> Insert data
>> Creating cassandra keyspaces
>> Connected to: "Foo" on 127.0.0.1/9160
>> ef1a6d30-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> Authenticated to keyspace: Bars
>> ef4af310-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> ef9bab20-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> efbceec0-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> f00e4310-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> f0589280-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> f0821380-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> f0c44ca0-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> Authenticated to keyspace: Baz
>> f121d5f0-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> f1619e10-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> f18b4620-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> Authenticated to keyspace: Buz
>> f1debd50-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> f20690a0-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> f25043d0-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> f29a1e10-ee91-11e0--534d24a6e7f7
>> Waiting for schema agreement...
>> ... schemas agree across the cluster
>> Inserting data in cassandra
>> Connected to: "Foo" on 127.0.0.1/9160
>> Authenticated to keyspace: Boo
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> Value inserted.
>> 
>> real0m9.554s
>> user0m2.729s
>> sys 0m0.194s
>> 
>> 
>> Joseph Norton
>> nor...@alum.mit.edu
>> 
>> 
>> 
>> 
> 
> 
> 
> -- 
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com

Re: How to speed up "Waiting for schema agreement" for a single node Cassandra cluster?

[Jonathan Ellis]

Hmm...  Maybe disable compaction, since that can block schema changes.
 Otherwise the big win will be in
https://issues.apache.org/jira/browse/CASSANDRA-1391.

On Tue, Oct 4, 2011 at 9:33 AM, Joseph Norton  wrote:
>
> I didn't consider using truncate because a set of potentially random Column 
> Families are created dynamically during the test.
>
> Are there any configuration knobs that could be adjusted for drop + recreate?
>
> thanks in advance,
>
> - Joe N
>
>
> Joseph Norton
> nor...@alum.mit.edu
>
>
>
> On Oct 4, 2011, at 11:19 PM, Jonathan Ellis wrote:
>
>> Truncate is faster than drop + recreate.
>>
>> On Tue, Oct 4, 2011 at 9:15 AM, Joseph Norton  
>> wrote:
>>>
>>> Hello.
>>>
>>> For unit test purposes, I have a single node Cassandra cluster.  I need to 
>>> drop and re-create several keyspaces between each test iteration.  This 
>>> process takes approximately 10 seconds for a single node installation.
>>>
>>> Can you recommend any tricks or recipes to reduce the time required for 
>>> such operations and/or for "Waiting for schema agreement" to complete?
>>>
>>> regards,
>>>
>>> - Joe N.
>>>
>>>
>>>
>>>
>>> $ time ./setupDB.sh
>>> Deleteing cassandra keyspaces
>>> Connected to: "Foo" on 127.0.0.1/9160
>>> ed9c7fc0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> ee8c36f0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> eeb14b20-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> Insert data
>>> Creating cassandra keyspaces
>>> Connected to: "Foo" on 127.0.0.1/9160
>>> ef1a6d30-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> Authenticated to keyspace: Bars
>>> ef4af310-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> ef9bab20-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> efbceec0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f00e4310-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f0589280-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f0821380-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f0c44ca0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> Authenticated to keyspace: Baz
>>> f121d5f0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f1619e10-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f18b4620-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> Authenticated to keyspace: Buz
>>> f1debd50-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f20690a0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f25043d0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f29a1e10-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> Inserting data in cassandra
>>> Connected to: "Foo" on 127.0.0.1/9160
>>> Authenticated to keyspace: Boo
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>>
>>> real    0m9.554s
>>> user    0m2.729s
>>> sys     0m0.194s
>>>
>>>
>>> Joseph Norton
>>> nor...@alum.mit.edu
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Re: Weird problem with empty CF

[Daning]

Thanks Aaron.  How about I set the gc_grace_seconds to 0 or like 2 
hours? I like to clean up tomebstone sooner, I don't care losing some 
data and all my columns have ttl.


If one node is down longer than gc_grace_seconds, and I got tombstone 
removed, once the node is up, from my understanding deleted data will be 
synced back. In this case my data will be processed twice and it will 
not be a big deal to me.


Thanks,

Daning


On 10/04/2011 01:27 AM, aaron morton wrote:

Yes that's the slice query skipping past the tombstone columns.

Cheers

-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 4/10/2011, at 4:24 PM, Daning Wang wrote:


Lots of SliceQueryFilter in the log, is that handling tombstone?

DEBUG [ReadStage:49] 2011-10-03 20:15:07,942 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317582939743663:true:4@1317582939933000
DEBUG [ReadStage:50] 2011-10-03 20:15:07,942 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317573253148778:true:4@1317573253354000
DEBUG [ReadStage:43] 2011-10-03 20:15:07,942 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317669552951428:true:4@1317669553018000
DEBUG [ReadStage:33] 2011-10-03 20:15:07,942 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317581886709261:true:4@1317581886957000
DEBUG [ReadStage:52] 2011-10-03 20:15:07,942 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317568165152246:true:4@1317568165482000
DEBUG [ReadStage:36] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317567265089211:true:4@1317567265405000
DEBUG [ReadStage:53] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317674324843122:true:4@1317674324946000
DEBUG [ReadStage:38] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317571990078721:true:4@1317571990141000
DEBUG [ReadStage:57] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317671855234221:true:4@1317671855239000
DEBUG [ReadStage:54] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317558305262954:true:4@1317558305337000
DEBUG [RequestResponseStage:11] 2011-10-03 20:15:07,941 
ResponseVerbHandler.java (line 48) Processing response on a callback 
from 12347@/10.210.101.104 
DEBUG [RequestResponseStage:9] 2011-10-03 20:15:07,941 
AbstractRowResolver.java (line 66) Preprocessed data response
DEBUG [RequestResponseStage:13] 2011-10-03 20:15:07,941 
AbstractRowResolver.java (line 66) Preprocessed digest response
DEBUG [ReadStage:58] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317581337972739:true:4@1317581338044000
DEBUG [ReadStage:64] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317582656796332:true:4@131758265697
DEBUG [ReadStage:55] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317569432886284:true:4@1317569432984000
DEBUG [ReadStage:45] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317572658687019:true:4@1317572658718000
DEBUG [ReadStage:47] 2011-10-03 20:15:07,940 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317582281617755:true:4@1317582281717000
DEBUG [ReadStage:48] 2011-10-03 20:15:07,940 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317549607869226:true:4@1317549608118000
DEBUG [ReadStage:34] 2011-10-03 20:15:07,940 SliceQueryFilter.java 
(line 123) collecting 0 of 1:
On Thu, Sep 29, 2011 at 2:17 PM, aaron morton 
mailto:aa...@thelastpickle.com>> wrote:


As with any situation involving the un-dead, it really is the
number of Zombies, Mummies or Vampires that is the concern.

If you delete data there will always be tombstones. If you have a
delete heavy workload there will be more tombstones. This is why
implementing a queue with cassandra is a bad idea.

gc_grace_seconds (and column TTL) are the *minimum* about of time
the tombstones will stay in the data files, there is no maximum.

Your read performance also depends on the number of SSTables the
row is spread over, see
http://thelastpickle.com/2011/04/28/Forces-of-Write-and-Read/

If you really wanted to purge them then yes a repair and then
major compaction would be the way to go. Also consider if it's
possible to design the data model around the problem, e.g.
partitioning rows by date. IMHO I would look to make data model
changes before implementing a compaction policy, or consider if
cassandra is the right store if you have a delete heavy workload.

Cheers


-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com 

On 30/09/2011, at 3:27 AM, Daning Wang wrote:


Jonathan/Aaron,

Thank you guy's reply, I will change GCGracePeriod to 1 day to
see what will happen.

Is there a way to purge to

Re: Does anybody know why Twitter stop integrate Cassandra as Twitter store?

[Paul Loy]

Did you read the article you posted?

"*We believe that this isn't the time to make large scale migration to a new
technology*. We will focus our Cassandra work on new projects that we
wouldn't be able to ship without a large-scale data store.

"*We're investing in Cassandra every day. It'll be with us for a long time
and our usage of it will only grow.*"


On Tue, Oct 4, 2011 at 5:07 AM, ruslan usifov wrote:

> http://engineering.twitter.com/2010/07/cassandra-at-twitter-today.html
>
> As said in this post Twiter stop working on using Cassandra as a store for
> Tweets, but there nothing said why they made this decision? Does anybody
> have mo information
>



-- 
-
Paul Loy
p...@keteracel.com
http://uk.linkedin.com/in/paulloy

Re: Does anybody know why Twitter stop integrate Cassandra as Twitter store?

[ruslan usifov]

Hello

2011/10/4 Paul Loy 

> Did you read the article you posted?
>
Yes


> "*We believe that this isn't the time to make large scale migration to a
> new technology*. We will focus our Cassandra work on new projects that we
> wouldn't be able to ship without a large-scale data store.



There was big boom in network about, that Tweeter will migrate they tweets
to cassandra, but than they reject this plans. This explanation sounds very
vague. Why they have changed the mind? I find only one article about this:

http://highscalability.com/blog/2010/7/11/so-why-is-twitter-really-not-using-cassandra-to-store-tweets.html

Re: help needed interpreting Read/Write latency in cfstats and cfhistograms output

[Brandon Williams]

On Mon, Oct 3, 2011 at 3:57 PM, Ramesh Natarajan  wrote:
> Thanks Aaron. The ms in the latency is it microseconds or milliseconds?
> I ran the 2 commands at the same time. I was expecting the values to be in
> the some what similar but from my output earlier ,  you can see the median
> in read latency in histogram output is about 10 milliseconds whereas the
> cfstats showed 5 ms.  Is this normal?
> thanks
> Ramesh

You should be aware of https://issues.apache.org/jira/browse/CASSANDRA-3222

-Brandon

dedicated gossip lan

[Sorin Julean]

Hi,

 Did anyone used a dedicated interfaces and LAN / VLAN for gossip traffic ?

 Any benefits in such approach ?


Cheers,
Sorin

Re: Does anybody know why Twitter stop integrate Cassandra as Twitter store?

[Paul Loy]

yup, and again it gives a perfectly adequate reason: "Twitter is busy
fighting other fires
and they
don't have the time to retrofit something that is (more or less) working,
namely their MySQL based tweet storage, with a completely new technology
based on Cassandra.*"
*
If I was in charge of platform at twitter I'd have probably made the same
call. If it aint broke, don't spend $100ks fixing it. Push out new features
that help keep you ahead of the competition.

I really don't see anything in the closet here. It's just a simple resource
management issue.

On Tue, Oct 4, 2011 at 11:43 AM, ruslan usifov wrote:

> Hello
>
> 2011/10/4 Paul Loy 
>
>> Did you read the article you posted?
>>
> Yes
>
>
>> "*We believe that this isn't the time to make large scale migration to a
>> new technology*. We will focus our Cassandra work on new projects that we
>> wouldn't be able to ship without a large-scale data store.
>
>
>
> There was big boom in network about, that Tweeter will migrate they tweets
> to cassandra, but than they reject this plans. This explanation sounds very
> vague. Why they have changed the mind? I find only one article about this:
>
>
> http://highscalability.com/blog/2010/7/11/so-why-is-twitter-really-not-using-cassandra-to-store-tweets.html
>
>
>
>


-- 
-
Paul Loy
p...@keteracel.com
http://uk.linkedin.com/in/paulloy

ApacheCon meetup?

[Chris Burroughs]

ApacheCon NA is coming up next month.  I suspect there will be at least
a few Cassandra users there (yeah new release!).  Would anyone be
interested in getting together and sharing some stories?  This could
either be a "official" [1] meetup.  Or grabbing food together sometime.

[1] http://wiki.apache.org/apachecon/ApacheMeetupsNa11

Re: dedicated gossip lan

[Brandon Williams]

On Tue, Oct 4, 2011 at 2:00 PM, Sorin Julean  wrote:
> Hi,
>
>  Did anyone used a dedicated interfaces and LAN / VLAN for gossip traffic ?
>
>  Any benefits in such approach ?

I don't think there is any substantial benefit to doing this, but also
it's impossible: gossip is not separate from the storage protocol.  Of
course, I am assuming you mean just gossip, but if what you actually
mean is the entire storage protocol (listen_address) then yes, there
is benefit to having a dedicated network for that.

-Brandon

Re: dedicated gossip lan

[Sorin Julean]

Sorry for not being clear.
Indeed I mean a separate LAN and interfaces for "listen_address".

- sorin

On Tue, Oct 4, 2011 at 10:49 PM, Brandon Williams  wrote:

> On Tue, Oct 4, 2011 at 2:00 PM, Sorin Julean 
> wrote:
> > Hi,
> >
> >  Did anyone used a dedicated interfaces and LAN / VLAN for gossip traffic
> ?
> >
> >  Any benefits in such approach ?
>
> I don't think there is any substantial benefit to doing this, but also
> it's impossible: gossip is not separate from the storage protocol.  Of
> course, I am assuming you mean just gossip, but if what you actually
> mean is the entire storage protocol (listen_address) then yes, there
> is benefit to having a dedicated network for that.
>
> -Brandon
>

Re: Weird problem with empty CF

[aaron morton]

I would not get gc_grace seconds to 0, set to to something small. 

gc_grace_seconds or ttl is only the minimum amount of time the column will stay 
in the data files. The columns are only purged when compaction runs some time 
after that timespan has ended. 

If you are seeing issues where a heavy delete workload is having an noticeably 
adverse effect on read performance then you should look at the data model. 
Consider ways to spread the write / read / delete workload over multiple rows.

If you cannot get away from it then experiment with reducing the 
min_compactioon_threshold of the CF's so that compaction kicks in quicker, and 
(potentially) tombstones are purged faster. 

Chees

 
-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 5/10/2011, at 6:03 AM, Daning wrote:

> Thanks Aaron.  How about I set the gc_grace_seconds to 0 or like 2 hours? I 
> like to clean up tomebstone sooner, I don't care losing some data and all 
> my columns have ttl. 
> 
> If one node is down longer than gc_grace_seconds, and I got tombstone 
> removed, once the node is up, from my understanding deleted data will be 
> synced back. In this case my data will be processed twice and it will not be 
> a big deal to me.
> 
> Thanks,
> 
> Daning
> 
> 
> On 10/04/2011 01:27 AM, aaron morton wrote:
>> 
>> Yes that's the slice query skipping past the tombstone columns. 
>> 
>> Cheers
>> 
>> -
>> Aaron Morton
>> Freelance Cassandra Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>> 
>> On 4/10/2011, at 4:24 PM, Daning Wang wrote:
>> 
>>> Lots of SliceQueryFilter in the log, is that handling tombstone?
>>> 
>>> DEBUG [ReadStage:49] 2011-10-03 20:15:07,942 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317582939743663:true:4@1317582939933000
>>> DEBUG [ReadStage:50] 2011-10-03 20:15:07,942 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317573253148778:true:4@1317573253354000
>>> DEBUG [ReadStage:43] 2011-10-03 20:15:07,942 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317669552951428:true:4@1317669553018000
>>> DEBUG [ReadStage:33] 2011-10-03 20:15:07,942 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317581886709261:true:4@1317581886957000
>>> DEBUG [ReadStage:52] 2011-10-03 20:15:07,942 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317568165152246:true:4@1317568165482000
>>> DEBUG [ReadStage:36] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317567265089211:true:4@1317567265405000
>>> DEBUG [ReadStage:53] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317674324843122:true:4@1317674324946000
>>> DEBUG [ReadStage:38] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317571990078721:true:4@1317571990141000
>>> DEBUG [ReadStage:57] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317671855234221:true:4@1317671855239000
>>> DEBUG [ReadStage:54] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317558305262954:true:4@1317558305337000
>>> DEBUG [RequestResponseStage:11] 2011-10-03 20:15:07,941 
>>> ResponseVerbHandler.java (line 48) Processing response on a callback from 
>>> 12347@/10.210.101.104
>>> DEBUG [RequestResponseStage:9] 2011-10-03 20:15:07,941 
>>> AbstractRowResolver.java (line 66) Preprocessed data response
>>> DEBUG [RequestResponseStage:13] 2011-10-03 20:15:07,941 
>>> AbstractRowResolver.java (line 66) Preprocessed digest response
>>> DEBUG [ReadStage:58] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317581337972739:true:4@1317581338044000
>>> DEBUG [ReadStage:64] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317582656796332:true:4@131758265697
>>> DEBUG [ReadStage:55] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317569432886284:true:4@1317569432984000
>>> DEBUG [ReadStage:45] 2011-10-03 20:15:07,941 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317572658687019:true:4@1317572658718000
>>> DEBUG [ReadStage:47] 2011-10-03 20:15:07,940 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317582281617755:true:4@1317582281717000
>>> DEBUG [ReadStage:48] 2011-10-03 20:15:07,940 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 1317549607869226:true:4@1317549608118000
>>> DEBUG [ReadStage:34] 2011-10-03 20:15:07,940 SliceQueryFilter.java (line 
>>> 123) collecting 0 of 1: 
>>> On Thu, Sep 29, 2011 at 2:17 PM, aaron morton  
>>> wrote:
>>> As with any situation involving the un-dead, it really is the number of 
>>> Zombies, Mummies or Vampires that is the concern.  
>>> 
>>> If you delete data there will always be tombstones. If you have a delete 
>>> heavy workload there will be more tombstones. This is why implementing a 
>>> queue with cassandra is a bad idea.
>>> 
>>> gc_grace_seconds (a

Re: Does anybody know why Twitter stop integrate Cassandra as Twitter store?

[aaron morton]

If you want to see just how much Twitter uses Cassandra watch Chris Goffinet's 
awesome presentation at this years Cassandra SF meeting 
http://www.datastax.com/events/cassandrasf2011/presentations

Cheers
   
-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 5/10/2011, at 8:16 AM, Paul Loy wrote:

> yup, and again it gives a perfectly adequate reason: "Twitter is busy 
> fighting other fires and they don't have the time to retrofit something that 
> is (more or less) working, namely their MySQL based tweet storage, with a 
> completely new technology based on Cassandra."
> 
> If I was in charge of platform at twitter I'd have probably made the same 
> call. If it aint broke, don't spend $100ks fixing it. Push out new features 
> that help keep you ahead of the competition.
> 
> I really don't see anything in the closet here. It's just a simple resource 
> management issue.
> 
> On Tue, Oct 4, 2011 at 11:43 AM, ruslan usifov  
> wrote:
> Hello
> 
> 2011/10/4 Paul Loy 
> Did you read the article you posted?
> Yes
>  
> "We believe that this isn't the time to make large scale migration to a new 
> technology. We will focus our Cassandra work on new projects that we wouldn't 
> be able to ship without a large-scale data store.
> 
> 
> There was big boom in network about, that Tweeter will migrate they tweets to 
> cassandra, but than they reject this plans. This explanation sounds very 
> vague. Why they have changed the mind? I find only one article about this:
> 
> http://highscalability.com/blog/2010/7/11/so-why-is-twitter-really-not-using-cassandra-to-store-tweets.html
>  
> 
> 
> 
> 
> 
> -- 
> -
> Paul Loy
> p...@keteracel.com
> http://uk.linkedin.com/in/paulloy

EC2 raid0 disks ?

[Yang]

it seems that how many virtual disks you can have is fixed:

on m2.4xlarge you have 2 disks, while on m2.2xlarge you have only 1,
so I can't setup a raid0 on m2.2xlarge

am I correct?

Thanks
Yang

Re: Does anybody know why Twitter stop integrate Cassandra as Twitter store?

[Chris Goffinet]

At the time of that project, there wasn't enough resources and dedicated
team. Since then we changed that (based on the presentation I gave). We
decided to focus on other areas, and newer projects. We spent a lot of time
with the community improving failure conditions, performance, etc. We chose
to focus on projects that were lower tier SLAs at first, and work our way
up. Now we have Cassandra running on our highest tier SLA (Cuckoo -- our
monitoring and alerting infrastructure for Twitter).


On Tue, Oct 4, 2011 at 1:37 PM, aaron morton wrote:

> If you want to see just how much Twitter uses Cassandra watch Chris
> Goffinet's awesome presentation at this years Cassandra SF meeting
> http://www.datastax.com/events/cassandrasf2011/presentations
>
> Cheers
>
> -
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 5/10/2011, at 8:16 AM, Paul Loy wrote:
>
> yup, and again it gives a perfectly adequate reason: "Twitter is busy
> fighting other fires
> and
> they don't have the time to retrofit something that is (more or less)
> working, namely their MySQL based tweet storage, with a completely new
> technology based on Cassandra.*"
> *
> If I was in charge of platform at twitter I'd have probably made the same
> call. If it aint broke, don't spend $100ks fixing it. Push out new features
> that help keep you ahead of the competition.
>
> I really don't see anything in the closet here. It's just a simple resource
> management issue.
>
> On Tue, Oct 4, 2011 at 11:43 AM, ruslan usifov wrote:
>
>> Hello
>>
>> 2011/10/4 Paul Loy 
>>
>>> Did you read the article you posted?
>>>
>> Yes
>>
>>
>>> "*We believe that this isn't the time to make large scale migration to a
>>> new technology*. We will focus our Cassandra work on new projects that
>>> we wouldn't be able to ship without a large-scale data store.
>>
>>
>>
>> There was big boom in network about, that Tweeter will migrate they tweets
>> to cassandra, but than they reject this plans. This explanation sounds very
>> vague. Why they have changed the mind? I find only one article about this:
>>
>>
>> http://highscalability.com/blog/2010/7/11/so-why-is-twitter-really-not-using-cassandra-to-store-tweets.html
>>
>>
>>
>>
>
>
> --
> -
> Paul Loy
> p...@keteracel.com
> http://uk.linkedin.com/in/paulloy
>
>
>

Shrinking cluster with counters ...

[Ian Danforth]

All,

 If I have a 3 node cluster storing counters and RF3, is it possible to
shrink back down to a single node cluster? If so should I change replication
factor, disable a node, wait for streaming to complete, and repeat for the
other node? Should I assume that the cluster will be unavailable during this
process?

Thanks in advance!

Ian

Re: How to speed up "Waiting for schema agreement" for a single node Cassandra cluster?

[Joseph Norton]

Thanks for the pointers.  For this type of functional unit testing, I suppose 
what I really want is a mock Cassandra (or Thrift Server) node for quickly 
running lots of tests for an application's logic.

thanks,

- Joe N.


Joseph Norton
nor...@alum.mit.edu



On Oct 5, 2011, at 12:01 AM, Jonathan Ellis wrote:

> Hmm...  Maybe disable compaction, since that can block schema changes.
> Otherwise the big win will be in
> https://issues.apache.org/jira/browse/CASSANDRA-1391.
> 
> On Tue, Oct 4, 2011 at 9:33 AM, Joseph Norton  
> wrote:
>> 
>> I didn't consider using truncate because a set of potentially random Column 
>> Families are created dynamically during the test.
>> 
>> Are there any configuration knobs that could be adjusted for drop + recreate?
>> 
>> thanks in advance,
>> 
>> - Joe N
>> 
>> 
>> Joseph Norton
>> nor...@alum.mit.edu
>> 
>> 
>> 
>> On Oct 4, 2011, at 11:19 PM, Jonathan Ellis wrote:
>> 
>>> Truncate is faster than drop + recreate.
>>> 
>>> On Tue, Oct 4, 2011 at 9:15 AM, Joseph Norton  
>>> wrote:
 
 Hello.
 
 For unit test purposes, I have a single node Cassandra cluster.  I need to 
 drop and re-create several keyspaces between each test iteration.  This 
 process takes approximately 10 seconds for a single node installation.
 
 Can you recommend any tricks or recipes to reduce the time required for 
 such operations and/or for "Waiting for schema agreement" to complete?
 
 regards,
 
 - Joe N.
 
 
 
 
 $ time ./setupDB.sh
 Deleteing cassandra keyspaces
 Connected to: "Foo" on 127.0.0.1/9160
 ed9c7fc0-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 ee8c36f0-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 eeb14b20-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 Insert data
 Creating cassandra keyspaces
 Connected to: "Foo" on 127.0.0.1/9160
 ef1a6d30-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 Authenticated to keyspace: Bars
 ef4af310-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 ef9bab20-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 efbceec0-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 f00e4310-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 f0589280-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 f0821380-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 f0c44ca0-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 Authenticated to keyspace: Baz
 f121d5f0-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 f1619e10-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 f18b4620-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 Authenticated to keyspace: Buz
 f1debd50-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 f20690a0-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 f25043d0-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 f29a1e10-ee91-11e0--534d24a6e7f7
 Waiting for schema agreement...
 ... schemas agree across the cluster
 Inserting data in cassandra
 Connected to: "Foo" on 127.0.0.1/9160
 Authenticated to keyspace: Boo
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 Value inserted.
 
 real0m9.554s
 user0m2.729s
 sys 0m0.194s
 
 
 Joseph Norton
 nor...@alum.mit.edu
 
 
 
 
>>> 
>>> 
>>> 
>>> --
>>> Jonathan Ellis
>>> Project Chair, Apache Cassandra
>>> co-founder of DataStax, the source for professional Cassandra support
>>> http://www.datastax.com
>> 
>> 
> 
> 
> 
> -- 
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http:/

Re: Weird problem with empty CF

[Daning]

Thanks.  Do you have plan to improve this? I think tombstone should be 
separated with live data since it serves different purpose, built in 
separate SSTable or indexed differently. It is pretty costly to do 
filtering while reading.


Daning

On 10/04/2011 01:34 PM, aaron morton wrote:

I would not get gc_grace seconds to 0, set to to something small.

gc_grace_seconds or ttl is only the minimum amount of time the column 
will stay in the data files. The columns are only purged when 
compaction runs some time after that timespan has ended.


If you are seeing issues where a heavy delete workload is having an 
noticeably adverse effect on read performance then you should look at 
the data model. Consider ways to spread the write / read / delete 
workload over multiple rows.


If you cannot get away from it then experiment with reducing the 
min_compactioon_threshold of the CF's so that compaction kicks in 
quicker, and (potentially) tombstones are purged faster.


Chees


-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com

On 5/10/2011, at 6:03 AM, Daning wrote:

Thanks Aaron.  How about I set the gc_grace_seconds to 0 or like 2 
hours? I like to clean up tomebstone sooner, I don't care losing some 
data and all my columns have ttl.


If one node is down longer than gc_grace_seconds, and I got tombstone 
removed, once the node is up, from my understanding deleted data will 
be synced back. In this case my data will be processed twice and it 
will not be a big deal to me.


Thanks,

Daning


On 10/04/2011 01:27 AM, aaron morton wrote:

Yes that's the slice query skipping past the tombstone columns.

Cheers

-
Aaron Morton
Freelance Cassandra Developer
@aaronmorton
http://www.thelastpickle.com 

On 4/10/2011, at 4:24 PM, Daning Wang wrote:


Lots of SliceQueryFilter in the log, is that handling tombstone?

DEBUG [ReadStage:49] 2011-10-03 20:15:07,942 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317582939743663:true:4@1317582939933000
DEBUG [ReadStage:50] 2011-10-03 20:15:07,942 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317573253148778:true:4@1317573253354000
DEBUG [ReadStage:43] 2011-10-03 20:15:07,942 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317669552951428:true:4@1317669553018000
DEBUG [ReadStage:33] 2011-10-03 20:15:07,942 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317581886709261:true:4@1317581886957000
DEBUG [ReadStage:52] 2011-10-03 20:15:07,942 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317568165152246:true:4@1317568165482000
DEBUG [ReadStage:36] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317567265089211:true:4@1317567265405000
DEBUG [ReadStage:53] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317674324843122:true:4@1317674324946000
DEBUG [ReadStage:38] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317571990078721:true:4@1317571990141000
DEBUG [ReadStage:57] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317671855234221:true:4@1317671855239000
DEBUG [ReadStage:54] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317558305262954:true:4@1317558305337000
DEBUG [RequestResponseStage:11] 2011-10-03 20:15:07,941 
ResponseVerbHandler.java (line 48) Processing response on a 
callback from 12347@/10.210.101.104 
DEBUG [RequestResponseStage:9] 2011-10-03 20:15:07,941 
AbstractRowResolver.java (line 66) Preprocessed data response
DEBUG [RequestResponseStage:13] 2011-10-03 20:15:07,941 
AbstractRowResolver.java (line 66) Preprocessed digest response
DEBUG [ReadStage:58] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317581337972739:true:4@1317581338044000
DEBUG [ReadStage:64] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317582656796332:true:4@131758265697
DEBUG [ReadStage:55] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317569432886284:true:4@1317569432984000
DEBUG [ReadStage:45] 2011-10-03 20:15:07,941 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317572658687019:true:4@1317572658718000
DEBUG [ReadStage:47] 2011-10-03 20:15:07,940 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317582281617755:true:4@1317582281717000
DEBUG [ReadStage:48] 2011-10-03 20:15:07,940 SliceQueryFilter.java 
(line 123) collecting 0 of 1: 1317549607869226:true:4@1317549608118000
DEBUG [ReadStage:34] 2011-10-03 20:15:07,940 SliceQueryFilter.java 
(line 123) collecting 0 of 1:
On Thu, Sep 29, 2011 at 2:17 PM, aaron morton 
mailto:aa...@thelastpickle.com>> wrote:


As with any situation involving the un-dead, it really is the
number of Zombies, Mummies or Vampires that is the concern.

If you delete data there will always be tombstones. If you have
a delete heavy workload there w

Re: EC2 raid0 disks ?

[Yi Yang]

AFAIK it's around 450G per ephemeral disk.
BTW randomly you can get high performance EBS drives as well. Performance are 
good for DB but are random in IOps.
--Original Message--
From: Yang
To: user@cassandra.apache.org
ReplyTo: user@cassandra.apache.org
Subject: EC2 raid0 disks ?
Sent: Oct 5, 2011 5:01 AM

it seems that how many virtual disks you can have is fixed:

on m2.4xlarge you have 2 disks, while on m2.2xlarge you have only 1,
so I can't setup a raid0 on m2.2xlarge

am I correct?

Thanks
Yang

?? BlackBerry?0?3 ?o???b??

Re: EC2 raid0 disks ?

[Joaquin Casares]

Correct. Not with ephemeral storage.

Here's a complete list of the drives that are attached:
http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?InstanceStorage.html#StorageOnInstanceTypes

Joaquin Casares
DataStax
Software Engineer/Support



On Tue, Oct 4, 2011 at 4:01 PM, Yang  wrote:

> it seems that how many virtual disks you can have is fixed:
>
> on m2.4xlarge you have 2 disks, while on m2.2xlarge you have only 1,
> so I can't setup a raid0 on m2.2xlarge
>
> am I correct?
>
> Thanks
> Yang
>

Re: EC2 raid0 disks ?

[Joaquin Casares]

Hello again,

Also, EBS volumes can be attached, but the performance issues cause other
issues when running a healthy cluster. From experience running clusters on
EBS volumes bring their own set of unique problems and are harder to debug.

Here's a quick link that provides a bit more background information on why
it's not the best fit for Cassandra.
http://www.mail-archive.com/user@cassandra.apache.org/msg11022.html

Thanks,
Joaquin Casares
DataStax
Software Engineer/Support



2011/10/4 Yi Yang 

> AFAIK it's around 450G per ephemeral disk.
> BTW randomly you can get high performance EBS drives as well. Performance
> are good for DB but are random in IOps.
> --Original Message--
> From: Yang
> To: user@cassandra.apache.org
> ReplyTo: user@cassandra.apache.org
> Subject: EC2 raid0 disks ?
> Sent: Oct 5, 2011 5:01 AM
>
> it seems that how many virtual disks you can have is fixed:
>
> on m2.4xlarge you have 2 disks, while on m2.2xlarge you have only 1,
> so I can't setup a raid0 on m2.2xlarge
>
> am I correct?
>
> Thanks
> Yang
>
> 從我的 BlackBerry(R) 無線裝置

Re: EC2 raid0 disks ?

[Yang]

Thanks guys.

btw, what is the performance difference between doing a raid0 on the
multiple ephemeral drives available, and then assign it to cassandra
data directory, vs creating a mount on each of these drives, and then
specify all of these to cassandra's data directory list?

since these drives are all virtual, would there be any benefit at all
in doing a raid0 ?

Yang

2011/10/4 Joaquin Casares :
> Hello again,
> Also, EBS volumes can be attached, but the performance issues cause other
> issues when running a healthy cluster. From experience running clusters on
> EBS volumes bring their own set of unique problems and are harder to debug.
> Here's a quick link that provides a bit more background information on why
> it's not the best fit for Cassandra.
> http://www.mail-archive.com/user@cassandra.apache.org/msg11022.html
> Thanks,
> Joaquin Casares
> DataStax
> Software Engineer/Support
>
>
> 2011/10/4 Yi Yang 
>>
>> AFAIK it's around 450G per ephemeral disk.
>> BTW randomly you can get high performance EBS drives as well. Performance
>> are good for DB but are random in IOps.
>> --Original Message--
>> From: Yang
>> To: user@cassandra.apache.org
>> ReplyTo: user@cassandra.apache.org
>> Subject: EC2 raid0 disks ?
>> Sent: Oct 5, 2011 5:01 AM
>>
>> it seems that how many virtual disks you can have is fixed:
>>
>> on m2.4xlarge you have 2 disks, while on m2.2xlarge you have only 1,
>> so I can't setup a raid0 on m2.2xlarge
>>
>> am I correct?
>>
>> Thanks
>> Yang
>>
>> 從我的 BlackBerry(R) 無線裝置
>

Re: EC2 raid0 disks ?

[Joaquin Casares]

Not a problem!

We ran a test a few months back and know it's better to use RAID0 vs. just
one mount directory. It slips my mind whether the test was also run for
multiple directories vs. RAID0 as well.

We chose the RAID0 method for the AMI as to avoid confusion and allow for
all the sstables to be on one drive and easier to find. This is also in line
with many setups that we see in the wild.

Thanks,

Joaquin Casares
DataStax
Software Engineer/Support



2011/10/4 Yang 

> Thanks guys.
>
> btw, what is the performance difference between doing a raid0 on the
> multiple ephemeral drives available, and then assign it to cassandra
> data directory, vs creating a mount on each of these drives, and then
> specify all of these to cassandra's data directory list?
>
> since these drives are all virtual, would there be any benefit at all
> in doing a raid0 ?
>
> Yang
>
> 2011/10/4 Joaquin Casares :
> > Hello again,
> > Also, EBS volumes can be attached, but the performance issues cause other
> > issues when running a healthy cluster. From experience running clusters
> on
> > EBS volumes bring their own set of unique problems and are harder to
> debug.
> > Here's a quick link that provides a bit more background information on
> why
> > it's not the best fit for Cassandra.
> > http://www.mail-archive.com/user@cassandra.apache.org/msg11022.html
> > Thanks,
> > Joaquin Casares
> > DataStax
> > Software Engineer/Support
> >
> >
> > 2011/10/4 Yi Yang 
> >>
> >> AFAIK it's around 450G per ephemeral disk.
> >> BTW randomly you can get high performance EBS drives as well.
> Performance
> >> are good for DB but are random in IOps.
> >> --Original Message--
> >> From: Yang
> >> To: user@cassandra.apache.org
> >> ReplyTo: user@cassandra.apache.org
> >> Subject: EC2 raid0 disks ?
> >> Sent: Oct 5, 2011 5:01 AM
> >>
> >> it seems that how many virtual disks you can have is fixed:
> >>
> >> on m2.4xlarge you have 2 disks, while on m2.2xlarge you have only 1,
> >> so I can't setup a raid0 on m2.2xlarge
> >>
> >> am I correct?
> >>
> >> Thanks
> >> Yang
> >>
> >> 從我的 BlackBerry(R) 無線裝置
> >
>

Re: How to speed up "Waiting for schema agreement" for a single node Cassandra cluster?

[Jeremiah Jordan]

But truncate is still slow, especially if it can't use JNA (windows) as it 
snapshots.  Depending on how much data you are inserting during your unit 
tests, just paging through all the keys and then deleting them is the fastest 
way, though if you use timestamps besides "now" this won't work, as the 
timestamps need to be increasing between test runs.


On Oct 4, 2011, at 9:33 AM, Joseph Norton wrote:

> 
> I didn't consider using truncate because a set of potentially random Column 
> Families are created dynamically during the test.
> 
> Are there any configuration knobs that could be adjusted for drop + recreate?
> 
> thanks in advance,
> 
> - Joe N
> 
> 
> Joseph Norton
> nor...@alum.mit.edu
> 
> 
> 
> On Oct 4, 2011, at 11:19 PM, Jonathan Ellis wrote:
> 
>> Truncate is faster than drop + recreate.
>> 
>> On Tue, Oct 4, 2011 at 9:15 AM, Joseph Norton  
>> wrote:
>>> 
>>> Hello.
>>> 
>>> For unit test purposes, I have a single node Cassandra cluster.  I need to 
>>> drop and re-create several keyspaces between each test iteration.  This 
>>> process takes approximately 10 seconds for a single node installation.
>>> 
>>> Can you recommend any tricks or recipes to reduce the time required for 
>>> such operations and/or for "Waiting for schema agreement" to complete?
>>> 
>>> regards,
>>> 
>>> - Joe N.
>>> 
>>> 
>>> 
>>> 
>>> $ time ./setupDB.sh
>>> Deleteing cassandra keyspaces
>>> Connected to: "Foo" on 127.0.0.1/9160
>>> ed9c7fc0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> ee8c36f0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> eeb14b20-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> Insert data
>>> Creating cassandra keyspaces
>>> Connected to: "Foo" on 127.0.0.1/9160
>>> ef1a6d30-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> Authenticated to keyspace: Bars
>>> ef4af310-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> ef9bab20-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> efbceec0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f00e4310-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f0589280-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f0821380-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f0c44ca0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> Authenticated to keyspace: Baz
>>> f121d5f0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f1619e10-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f18b4620-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> Authenticated to keyspace: Buz
>>> f1debd50-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f20690a0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f25043d0-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> f29a1e10-ee91-11e0--534d24a6e7f7
>>> Waiting for schema agreement...
>>> ... schemas agree across the cluster
>>> Inserting data in cassandra
>>> Connected to: "Foo" on 127.0.0.1/9160
>>> Authenticated to keyspace: Boo
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> Value inserted.
>>> 
>>> real0m9.554s
>>> user0m2.729s
>>> sys 0m0.194s
>>> 
>>> 
>>> Joseph Norton
>>> nor...@alum.mit.edu
>>> 
>>> 
>>> 
>>> 
>> 
>> 
>> 
>> -- 
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
> 

Re: How to speed up "Waiting for schema agreement" for a single node Cassandra cluster?

[Roshan Dawrani]

On Wed, Oct 5, 2011 at 7:42 AM, Jeremiah Jordan <
jeremiah.jor...@morningstar.com> wrote:

> But truncate is still slow, especially if it can't use JNA (windows) as it
> snapshots.  Depending on how much data you are inserting during your unit
> tests, just paging through all the keys and then deleting them is the
> fastest way


Exactly the thing we also found out (and hence ditched "truncate" for DB
cleanup between tests):
http://roshandawrani.wordpress.com/2011/09/30/grails-cassandra-giving-each-test-a-clean-db-to-work-with/
.
Worked much better for us this way.

-- 
Roshan
Blog: http://roshandawrani.wordpress.com/
Twitter: @roshandawrani 
Skype: roshandawrani

ByteOrderedPartitioner token generation

[Masoud Moshref Javadi]

I need to insert a large amount of data to Cassandra cluster in a short 
time. So I want the interaction among Cassandra servers be minimum. I 
think that the best way to do this is to use ByteOrderedPartitioner and 
generate ID of new data based on the InitialToken of servers and send 
data to the corresponding server from the webserver. Am I right?


Now my question is if I have some data ranging from 1-100 and want to 
put 1-25 in server1, 26-50 in server 2 and so on, what should be the 
Initial Token of the servers?


Thanks in advance