Re: Why my Cassandra is compacting like mad

2012-10-16 Thread Alain RODRIGUEZ
"I don't what hardware information may help" A low memory server will produce an intensive use of CPU and disk IO. It will write a lot of small SSTables which will need to be compacted very often. That was one of my thought when I asked. Alain 2012/10/15 Manu Zhang > I use default option for c

Performance problems & Unbalanced disk io on my cassandra cluster

2012-10-16 Thread Tamar Fraenkel
Hi! *Problem* I have one node which seems to be in a bad situation, with lots of dropped reads for a long time. *My cluster* I have 3 node cluster on Amazon m1.large DataStax AMI with cassandra 1.08. RF=3, RCL=WCL=QUORUM I use Hector which should be doing round robin of the requests between the n

Re: Missing non composite column

2012-10-16 Thread Sylvain Lebresne
On Tue, Oct 16, 2012 at 11:57 AM, Vivek Mishra wrote: > --- > RowKey: Jayne Cobb > => (column=2012-07-24:2:alliance_involvement, value=false, > timestamp=135038100502) > => (column=2012-07-24:2:energy_used, value=4.6, timestamp=1350381005020001) > > > Not sure, why is it not ge

Re: Missing non composite column

2012-10-16 Thread Vivek Mishra
Thanks Sylvain. I missed it. If i try to access these via thrift API, what will be the column names? -Vivek If i try to access this row via thrift API, what would be the column names returned? On Tue, Oct 16, 2012 at 3:52 PM, Sylvain Lebresne wrote: > On Tue, Oct 16, 2012 at 11:57 AM, Vivek Mish

Re: Why my Cassandra is compacting like mad

2012-10-16 Thread Manu Zhang
only 1/8 of heap is used. Only system-schema_columns cf is compacted. The weird thing is that it never stops itself. On Tue, Oct 16, 2012 at 5:14 PM, Alain RODRIGUEZ wrote: > "I don't what hardware information may help" > > A low memory server will produce an intensive use of CPU and disk IO. It

Re: what happens while node is bootstrapping?

2012-10-16 Thread Tyler Hobbs
On Mon, Oct 15, 2012 at 3:50 PM, Andrey Ilinykh wrote: > Does it mean that during bootstrapping process only replicas serve > read requests for new node range? In other words, replication factor > is RF-1? > No. The bootstrapping node will writes for its new range while bootstrapping as consist

Using Cassandra to store binary files?

2012-10-16 Thread Vasileios Vlachos
Hello All, We need to store about 40G of binary files in a redundant way and since we are already using Cassandra for other applications we were thinking that we could just solve that problem using the same Cassandra cluster. Each individual File will be approximately 1MB. We are thinking that th

Re: Using Cassandra to store binary files?

2012-10-16 Thread Hiller, Dean
Astyanax provides a streaming file feature and was written by netflix who is storing probably a huge amount of files with that feature. I was going to use that feature for one product but I never got around to creating the product…..but I still use astyanax under the hood of PlayOrm (we kind o

Re: Using Cassandra to store binary files?

2012-10-16 Thread Michael Kjellman
When we started with Cassandra almost 2 years ago in production originally it was for the sole purpose storing blobs in a redundant way. I ignored the warnings as my own tests showed it would be okay (and two years later it is "ok"). If you plan on using Cassandra later (as we now as as features

Re: what happens while node is bootstrapping?

2012-10-16 Thread Andrey Ilinykh
> > > No. The bootstrapping node will writes for its new range while > bootstrapping as consistency optimization (more or less), but does not > contribute to the replication factor or consistency level; all of the > original replicas for that range still receive writes, serve reads, and are > the

Re: Missing non composite column

2012-10-16 Thread Sylvain Lebresne
On Tue, Oct 16, 2012 at 12:31 PM, Vivek Mishra wrote: > Thanks Sylvain. I missed it. If i try to access these via thrift API, what > will be the column names? I'm not sure I understand the question. The cli output is pretty much what you get via the thrift API. -- Sylvain

Re: what happens while node is bootstrapping?

2012-10-16 Thread Michael Kjellman
Correct. Also, there is a new feature in 1.1+ that lets you play with live traffic on new nodes before they actually join the ring http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-1-live-traffic-sa mpling On 10/16/12 9:42 AM, "Andrey Ilinykh" wrote: >> >> >> No. The bootstrapping nod

Re: Using Cassandra to store binary files?

2012-10-16 Thread Hiller, Dean
Yes, astyanax stores the file in many rows so it reads from many disks giving you a performance advantage vs. storing each file in one row….well at least from my understanding so read performance "should" be really really good in that case. Dean From: Michael Kjellman mailto:mkjell...@barracud

Re: Using Cassandra to store binary files?

2012-10-16 Thread Michael Kjellman
Ah, so they just wrote chunking into Astyanax? Do they create an index somewhere so they know how to reassemble the file on the way out? On 10/16/12 10:36 AM, "Hiller, Dean" wrote: >Yes, astyanax stores the file in many rows so it reads from many disks >giving you a performance advantage vs. sto

Re: Is Anti Entropy repair idempotent with respect to transferred data?

2012-10-16 Thread Andrey Ilinykh
> In my experience running repair on some counter data, the size of > streamed data is much bigger than the cluster could possibly have lost > messages or would be due to snapshotting at different times. > > I know the data will eventually be in sync on every repair, but I'm > more interested in wh

Re: Using Cassandra to store binary files?

2012-10-16 Thread Hiller, Dean
I am not sure. If I were to implement it myself though, I would have probably… postfixed the rows with 1,2,3,4,… and then stored the lastValue in the first row so then my program knows all the rows. Ie. Not sure an index is really needed in that case. Dean On 10/16/12 11:45 AM, "Michael Kjellm

ApacheCon EU -- Hackathon anyone?

2012-10-16 Thread Eric Evans
Hi all, ApacheCon EU[1] is in 3 weeks, and Monday the 5th[2] is reserved for hackathons[3]. Who else is planning to be in Sinsheim on the 5th, and, is there any interest in a Cassandra hackathon? One idea for a hackathon would be to focus on testing (writing, running, manual), and bug squashing.

Re: Is Anti Entropy repair idempotent with respect to transferred data?

2012-10-16 Thread Omid Aladini
Thanks Andrey. Also found this ticket regarding this issue: https://issues.apache.org/jira/browse/CASSANDRA-2698 On Tue, Oct 16, 2012 at 8:00 PM, Andrey Ilinykh wrote: >> In my experience running repair on some counter data, the size of >> streamed data is much bigger than the cluster could poss

Re: Cassandra nodes loaded unequally

2012-10-16 Thread Ben Kaehne
I checked this and all the numbers seemed to be about the same. Although the files would compact from time to time. There was nothing to suggest why 1 node, ongoingly had less load then the others. Regards, On Fri, Oct 12, 2012 at 7:22 PM, Alexey Zotov wrote: > Hi Ben, > > I suggest you to compa

Re: Cassandra nodes loaded unequally

2012-10-16 Thread Ben Kaehne
Not connecting to the same node every time. Using Hector to ensure an even distribution of connections accross the cluster. Regards, On Sat, Oct 13, 2012 at 4:15 AM, B. Todd Burruss wrote: > are you connecting to the same node every time? if so, spread out > your connections across the ring >

Re: Cassandra nodes loaded unequally

2012-10-16 Thread Andrey Ilinykh
With your environment (3 nodes, RF=3) it is very difficult to get uneven load. Each node receives the same number of read/write requests. Probably something is wrong on low level, OS or VM. Do you see anything unusual in log files? Andrey On Tue, Oct 16, 2012 at 3:40 PM, Ben Kaehne wrote: > Not

Re: Cassandra nodes loaded unequally

2012-10-16 Thread Ben Kaehne
Nothing unusual. All servers are exactly the same. Nothing unusual in the log files. Is there any level of logging that I should be turning on? Regards, On Wed, Oct 17, 2012 at 9:51 AM, Andrey Ilinykh wrote: > With your environment (3 nodes, RF=3) it is very difficult to get > uneven load. Eac

Java 7 support?

2012-10-16 Thread Edward Sargisson
Hi all, The Datastax documentation says that Java 7 is not recommended[1]. However, Java 6 is due to EOL in Feb 2013 so what is the reasoning behind that comment? Is it something we should be still concerned about? Cheers, Edward Links: [1] http://www.datastax.com/docs/1.1/install/install_de

Re: Java 7 support?

2012-10-16 Thread Rob Coli
On Tue, Oct 16, 2012 at 4:45 PM, Edward Sargisson wrote: > The Datastax documentation says that Java 7 is not recommended[1]. However, > Java 6 is due to EOL in Feb 2013 so what is the reasoning behind that > comment? I've asked this approximate question here a few times, with no official respons

Re: Missing non composite column

2012-10-16 Thread Vivek Mishra
column name will be "2012-07-24:2:alliance_involvement" or "alliance_involvement"? -Vivek On Tue, Oct 16, 2012 at 10:25 PM, Sylvain Lebresne wrote: > On Tue, Oct 16, 2012 at 12:31 PM, Vivek Mishra > wrote: > > Thanks Sylvain. I missed it. If i try to access these via thrift API, > what > > will