Re: Moving to a new cluster

2011-09-25 Thread aaron morton
sounds like it. A - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 25/09/2011, at 6:10 PM, Yan Chunlu wrote: > thanks! is that similar problem described in this thread? > > > http://cassandra-user-incubator-apache-org.3065146.n2.nabbl

Re: Moving to a new cluster

2011-09-24 Thread Yan Chunlu
thanks! is that similar problem described in this thread? http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/nodetool-repair-caused-high-disk-space-usage-td6695542.html On Sun, Sep 25, 2011 at 11:33 AM, aaron morton wrote: > It can result in a lot of data on the node you run repa

Re: Moving to a new cluster

2011-09-24 Thread aaron morton
It can result in a lot of data on the node you run repair on. Where a lot means perhaps 2 or more times more data. My unscientific approach is to repair one CF at a time so you can watch the disk usage and repair the smaller CF's first. After the repair compact if you need to. I think the a

Re: Moving to a new cluster

2011-09-24 Thread aaron morton
No, Run it on one node at a time. Looks like I was a little off on my previous statement. Perhaps it would have been better to say if you have more than RF nodes run it on every RF'th node at the same time. But make sure you run it on all nodes eventually. Aaron - Aaron Mort

Re: Moving to a new cluster

2011-09-24 Thread aaron morton
Thanks sylvain, will look into the new stuff. - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 22/09/2011, at 9:09 PM, Sylvain Lebresne wrote: > 2011/9/22 Jonas Borgström : >> On 09/22/2011 01:25 AM, aaron morton wrote: >> *snip* >>> Wh

Re: Moving to a new cluster

2011-09-22 Thread Zhu Han
On Thu, Sep 22, 2011 at 11:04 PM, Yan Chunlu wrote: > > hi Aaron: > > could you explain more about the issue about repair make space usage going > crazy? > I guess repair defers the compaction progress as it brings a lot of load. For update-heavy workload, the space usage goes higher and higher

Re: Moving to a new cluster

2011-09-22 Thread Yan Chunlu
hi Aaron: could you explain more about the issue about repair make space usage going crazy? I am planning to upgrade my cluster from 0.7.4 to 0.8.6, which is because the repair never works on 0.7.4 for me. more specifically, CASSANDRA-2280 an

Re: Moving to a new cluster

2011-09-22 Thread Philippe
The current load on the nodes is around 300g. Le 22 sept. 2011 11:08, "aaron morton" a écrit : > the new nodes will have 1TB of data disk, but how much data will you put on them? > > A > > - > Aaron Morton > Freelance Cassandra Developer > @aaronmorton > http://www.thelastpickle.co

Re: Moving to a new cluster

2011-09-22 Thread Sylvain Lebresne
2011/9/22 Jonas Borgström : > On 09/22/2011 01:25 AM, aaron morton wrote: > *snip* >> When you start a repair it will repair will the other nodes it >> replicates data with. So you only need to run it every RF nodes. Start >> it one one, watch the logs to see who it talks to and then start it on >>

Re: Moving to a new cluster

2011-09-22 Thread aaron morton
the new nodes will have 1TB of data disk, but how much data will you put on them? A - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 22/09/2011, at 7:23 PM, Philippe wrote: > Hi Aaron > Thanks for the reply > > I should hhave mentionned

Re: Moving to a new cluster

2011-09-22 Thread Jonas Borgström
On 09/22/2011 01:25 AM, aaron morton wrote: *snip* > When you start a repair it will repair will the other nodes it > replicates data with. So you only need to run it every RF nodes. Start > it one one, watch the logs to see who it talks to and then start it on > the first node it does not talk to.

Re: Moving to a new cluster

2011-09-22 Thread Philippe
Hi Aaron Thanks for the reply I should hhave mentionned that all current nodes are running 0.8.4. All current and future services have 2TB disks of which i have allocated only half. I don't expect any issues here. Should I? Le 22 sept. 2011 01:26, "aaron morton" a écrit : > How much data is on th

Re: Moving to a new cluster

2011-09-21 Thread aaron morton
How much data is on the nodes in cluster 1 and how much disk space on cluster 2 ? Be aware that Cassandra 0.8 has an issue where repair can go crazy and use a lot of space. If you are not regularly running repair I would also repair before the move. The repair after the copy is a good idea but

Moving to a new cluster

2011-09-21 Thread Philippe
Hello, We're currently running on a 3-node RF=3 cluster. Now that we have a better grip on things, we want to replace it with a 12-node RF=3 cluster of "smaller" servers. So I wonder what the best way to move the data to the new cluster would be. I can afford to stop writing to the current cluster