Re: Cassandra counters replication uses more traffic than client increments?

2013-01-08 Thread Sylvain Lebresne
Since you're asking about counters, I'll note too that the internal representation of counters is pretty fat. In you RF=2 case, each counter is probably about 64 bytes internally, while on the client side you send only a 8 bytes value for each increment. So I don't think there is anything unexpecte

Re: Date Index?

2013-01-08 Thread aaron morton
There has to be one equality clause in there, and thats the thing to cassandra uses to select of disk. The others are in memory filters. So if you have one on the year+month you can have a simple select clause and it limits the amount of data that has to be read. If you have like many 10's to

Re: How long does it take for a write to actually happen?

2013-01-08 Thread aaron morton
> EC2 m1.large node You will have a much happier time if you use a m1.xlarge. > We set MAX_HEAP_SIZE="6G" and HEAP_NEWSIZE="400M" Thats a pretty low new heap size. > checks for new entries (in "Entries" CF, with indexed column status=1), > processes them, and sets the status to 2, when done T

Re: Helenos 1.3 released

2013-01-08 Thread aaron morton
I was going to say update the Wiki, but it's already there. Thanks for contributing. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 9/01/2013, at 1:02 AM, Tomek Kuprowski wrote: > Dear all, > > I'd like to annou

Re: Cassandra counters replication uses more traffic than client increments?

2013-01-08 Thread aaron morton
Can you measure the incoming client traffic on the nodes in DC 1 on port 9160 ? That would be more of an Apples to Apples comparison. >> I've taken a look at some of the captured packets and it looks like >> there's much more service information in DC-to-DC traffic compared to >> client-to-serve

Re: The solution of the write-in load of Cassandra?

2013-01-08 Thread aaron morton
> - Owing to what does a log continue coming out? (Is it presumed?) Were you noticing file times changing? The log files are recycled so it may have been that or from the 10 second commit log fsync. Can you provide more details on what you saw? > > The data of the contents of SSTABLE after a

JIRA for native IAuthorizer and IAuthenticator ?

2013-01-08 Thread Frank Hsueh
I am very interested in the native IAuthorizer and IAuthenticator implementation. However, I can't find a JIRA entry to follow in the 1.2.1 [1] or 1.2.2 [2] issues page. does anybody know about it ? thanks ! [1] https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%20%221.2.1%22%20AND%20pr

remote datacentre consistency

2013-01-08 Thread Jabbar
I'm a bit confused about how a two datacentre apache cassandra cluster keeps the data consistent. >From what I understand a client application in datacentre1 contacts a coordinator node which sends the data to the local replicas and it also sends the updates to the remote coordinator in the remote

Re: inconsistent hadoop/cassandra results

2013-01-08 Thread aaron morton
Assuming their were no further writes, running repair or using CL all should have fixed it. Can you describe the inconsistency between runs? Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 8/01/2013, at 2:16 AM, Br

Re: Script to load sstables from v1.0.x to v 1.1.x

2013-01-08 Thread Rob Coli
On Tue, Jan 8, 2013 at 11:56 AM, Todd Nine wrote: > Our current production > cluster is still on 1.0.x, so we can either fire up a 1.0.x cluster, then > upgrade every node to accomplish this, or just use the script. No 1.0 cluster is required to restore 1.0 directory structure to a 1.1 cluster an

Re: Script to load sstables from v1.0.x to v 1.1.x

2013-01-08 Thread Todd Nine
Our use case is for testing migrations in our data, as well as stress testing outside our production environment. To do this, we load our backups into a fresh cluster, then perform our testing. Our current production cluster is still on 1.0.x, so we can either fire up a 1.0.x cluster, then upg

Re: Script to load sstables from v1.0.x to v 1.1.x

2013-01-08 Thread Michael Kjellman
I thought this was to load between separate clusters not to upgrade within the same cluster. No? On Jan 8, 2013, at 11:29 AM, "Rob Coli" wrote: > On Tue, Jan 8, 2013 at 8:41 AM, Todd Nine wrote: >> I have recently been trying to restore backups from a v1.0.x cluster we >> have into a 1.1.7 cl

Re: Script to load sstables from v1.0.x to v 1.1.x

2013-01-08 Thread Rob Coli
On Tue, Jan 8, 2013 at 8:41 AM, Todd Nine wrote: > I have recently been trying to restore backups from a v1.0.x cluster we > have into a 1.1.7 cluster. This has not been as trivial as I expected, and > I've had a lot of help from the IRC channel in tackling this problem. As a > way of saying t

Re: help turning compaction..hours of run to get 0% compaction....

2013-01-08 Thread B. Todd Burruss
i'll second edward's comment. cassandra is designed to scale horizontally, so if disk I/O is slowing you down then you must scale On Tue, Jan 8, 2013 at 7:10 AM, Jim Cistaro wrote: > One metric to watch is pending compactions (via nodetool > compactionstats). This count will give you some id

Date Index?

2013-01-08 Thread Stephen.M.Thompson
Hi folks - Question about secondary indexes. How are people doing date indexes?I have a date column in my tables in RDBMS that we use frequently, such as look at all records recorded in the last month. What is the best practice for being able to do such a query? It seems like there could

Script to load sstables from v1.0.x to v 1.1.x

2013-01-08 Thread Todd Nine
Hi all, I have recently been trying to restore backups from a v1.0.x cluster we have into a 1.1.7 cluster. This has not been as trivial as I expected, and I've had a lot of help from the IRC channel in tackling this problem. As a way of saying thanks, I'd like to contribute the updated ruby scr

Re: about validity of recipe "A node join using external data copy methods"

2013-01-08 Thread Edward Capriolo
It has been true since about 0.8. in the old days ANTI-COMPACTION stunk and many weird errors would cause node joins to have to be retried N times. Now node moves/joins seem to work near 100% of the time (in 1.0.7) they are also very fast and efficient. If you want to move a node to new hardware

Re: Astyanax

2013-01-08 Thread Brian O'Neill
Not sure where you are on the learning curve, but I've put a couple "getting started" projects out on github: https://github.com/boneill42/astyanax-quickstart And the latest from the webinar is here: https://github.com/boneill42/naughty-or-nice http://brianoneill.blogspot.com/2013/01/creating-your

Re: CQL3 Frame Length

2013-01-08 Thread Ben Hood
Hey Sylvain, Thanks for explaining the rationale. When you look at from the perspective of the use cases you mention, it makes sense to be able to supply the reader with the frame size up front. I've opted to go for serializing the frame into a buffer. Although this could materialize an arbitrari

Re: CQL3 Frame Length

2013-01-08 Thread Sylvain Lebresne
Mostly this is because having the frame length is convenient to have in practice. Without pretending that there is only one way to write a server, it is common to separate the phase "read a frame from the network" from the phase "decode the frame" which is often simpler if you can read the frame u

RE: about validity of recipe "A node join using external data copy methods"

2013-01-08 Thread DE VITO Dominique
" Now streaming is very efficient rarely fails and there is no need to do it this way anymore" I guess it's true in v1.2. Is it true also in v1.1 ? Thanks. Dominique De : Edward Capriolo [mailto:edlinuxg...@gmail.com] Envoyé : mardi 8 janvier 2013 16:01 À : user@cassandra.apache.org Objet : R

Re: Astyanax

2013-01-08 Thread Radek Gruchalski
Hi, We are using astyanax and we found out that github wiki with stackoverflow is the most comprehensive set of documentation. Do you have any specific questions? Kind regards, Radek Gruchalski On 8 Jan 2013, at 15:46, Everton Lima wrote: > I was studing by there, but I would to know if anyo

Re: help turning compaction..hours of run to get 0% compaction....

2013-01-08 Thread Jim Cistaro
One metric to watch is pending compactions (via nodetool compactionstats). This count will give you some idea of whether you are falling behind with compactions. The other measure is how long you are compacting after your inserts have stopped. If I understand correctly, since you never update

Re: about validity of recipe "A node join using external data copy methods"

2013-01-08 Thread Edward Capriolo
Basically this recipe is from the old days when we had anti-compaction. Now streaming is very efficient rarely fails and there is no need to do it this way anymore. This recipe will be abolished from the second edition. It still likely works except when using counters. Edward On Tue, Jan 8, 2013

Re: Astyanax

2013-01-08 Thread Everton Lima
I was studing by there, but I would to know if anyone knows other sources. 2013/1/8 Markus Klems > The wiki? https://github.com/Netflix/astyanax/wiki > > > On Tue, Jan 8, 2013 at 2:44 PM, Everton Lima wrote: > >> Hi, >> Someone has or could indicate some good tutorial or book to learn >> Astyana

Re: Astyanax

2013-01-08 Thread Markus Klems
The wiki? https://github.com/Netflix/astyanax/wiki On Tue, Jan 8, 2013 at 2:44 PM, Everton Lima wrote: > Hi, > Someone has or could indicate some good tutorial or book to learn Astyanax? > > Thanks > > -- > Everton Lima Aleixo > Mestrando em Ciência da Computação pela UFG > Programador no LUPA >

about validity of recipe "A node join using external data copy methods"

2013-01-08 Thread DE VITO Dominique
Hi, Edward Capriolo described in his Cassandra book a faster way [1] to start new nodes if the cluster size doubles, from N to 2 *N. It's about splitting in 2 parts each token range taken in charge, after the split, with 2 nodes: the existing one, and a new one. And for starting a new node, on

CQL3 Frame Length

2013-01-08 Thread Ben Hood
Hi, I've read the CQL wire specification and naively, I can't see how the frame length length header is used. To me, it looks like on the read side, you know which type of structures to expect based on the opcode and each structure is TLV encoded. On the write side, you need to encode TLV struct

The solution of the write-in load of Cassandra?

2013-01-08 Thread hiroshi.kise.rk
Dear everyone I am troubled by the solution of the write-in load of Cassandra. Please teach, if there is a good solution. Cassandra is working only by one server and an action is Cassandra-1.1.7. Cassandra-1.1.8 was the same. 1. Where a certain amount of load is applied from a thrift cli

Re: Cassandra counters replication uses more traffic than client increments?

2013-01-08 Thread Sergey Olefir
So with the holidays hopefully being over, I thought I'd ask again :) Could someone please help with answers to the two questions: - Is it reasonable to expect that cross-datacenter node-to-node replication traffic is greater than actual client-to-server traffic that generates this activity? Speci