Re: I don't understand shuffle progress

2013-09-19 Thread Juan Manuel Formoso
Thanks. I did this and I finished rebuilding the new cluster in about 8 hours... much better option than shuffle (you have to have the hardware for duplicating your environment though) On Thu, Sep 19, 2013 at 7:21 PM, Jeremiah D Jordan < jeremiah.jor...@gmail.com> wrote: > > http://www.datastax.

Re: I don't understand shuffle progress

2013-09-19 Thread Jeremiah D Jordan
http://www.datastax.com/documentation/cassandra/1.2/webhelp/index.html#cassandra/configuration/configVnodesProduction_t.html On Sep 18, 2013, at 9:41 AM, Chris Burroughs wrote: > http://www.datastax.com/documentation/cassandra/1.2/webhelp/index.html#cassandra/operations/ops_add_dc_to_cluster_t.h

Re: I don't understand shuffle progress

2013-09-18 Thread Juan Manuel Formoso
Awesome, thanks! A few final questions: 1) Can I change the Snitch in the live source cluster? I'm using SimpleSnitch, I'd change it to GossipingPropertyFileSnitch (in preparation for changing the replication strategy when the new cluster is up and running). 2) Can I have different Partitioners on

Re: I don't understand shuffle progress

2013-09-18 Thread Chris Burroughs
http://www.datastax.com/documentation/cassandra/1.2/webhelp/index.html#cassandra/operations/ops_add_dc_to_cluster_t.html This is a basic outline. On 09/18/2013 10:32 AM, Juan Manuel Formoso wrote: I really like this idea. I can create a new cluster and have it replicate the old one, after it f

Re: I don't understand shuffle progress

2013-09-18 Thread Juan Manuel Formoso
I really like this idea. I can create a new cluster and have it replicate the old one, after it finishes I can remove the original. Any good resource that explains how to add a new datacenter to a live single dc cluster that anybody can recommend? On Wed, Sep 18, 2013 at 9:58 AM, Chris Burroughs

Re: I don't understand shuffle progress

2013-09-18 Thread Chris Burroughs
On 09/17/2013 09:41 PM, Paulo Motta wrote: So you're saying the only feasible way of enabling VNodes on an upgraded C* 1.2 is by doing fork writes to a brand new cluster + bulk load of sstables from the old cluster? Or is it possible to succeed on shuffling, even if that means waiting some weeks

Re: I don't understand shuffle progress

2013-09-17 Thread Juan Manuel Formoso
I have been trying to make it work non-stop since Friday afternoon. I officially gave up today and I'm going to go the sstableloader route. I wrote a little of what I tried here: http://seniorgeek.com.ar/blog/2013/09/16/tips-for-running-cassandra-shuffle/ (I have yet to update it with the fact tha

Re: I don't understand shuffle progress

2013-09-17 Thread David McNelis
As Rob mentioned, no one (myself included) has successfully used shuffle in the wild (that I've heard of). Shuffle is *supposed* to be a transparent background process... and is designed, in theory, to take a long time to run (weeks is the right way to think of it). Be sure to keep an eye on your

Re: I don't understand shuffle progress

2013-09-17 Thread Paulo Motta
That is very disappointing to hear. Vnodes support is one of the main reasons we're upgrading from 1.1.X to 1.2.X. So you're saying the only feasible way of enabling VNodes on an upgraded C* 1.2 is by doing fork writes to a brand new cluster + bulk load of sstables from the old cluster? Or is it p

Re: I don't understand shuffle progress

2013-09-17 Thread Robert Coli
On Tue, Sep 17, 2013 at 4:00 PM, Juan Manuel Formoso wrote: > Any better alternatives than creating a small application that reads from > one cluster and inserts in the new one that anybody can suggest? > > http://www.palominodb.com/blog/2012/09/25/bulk-loading-options-cassandra In theory if you

Re: I don't understand shuffle progress

2013-09-17 Thread Juan Manuel Formoso
Will the new cluster be evenly balanced? Remember that the old one was pre 1.2.X, so I had no vnodes I haven't used that tool, will look it up. Thanks for the suggestion! On Tuesday, September 17, 2013, David McNelis wrote: > Stable loader is the way to go to load up the new cluster. > > On Tue

Re: I don't understand shuffle progress

2013-09-17 Thread David McNelis
Stable loader is the way to go to load up the new cluster. On Tuesday, September 17, 2013, Juan Manuel Formoso wrote: > > If your shuffle succeeds, you will be the first reported case of > shuffle succeeding on a non-test cluster. > > Awesome! :O > > I'll try to migrate to a new cluster then. > >

Re: I don't understand shuffle progress

2013-09-17 Thread Juan Manuel Formoso
> If your shuffle succeeds, you will be the first reported case of shuffle succeeding on a non-test cluster. Awesome! :O I'll try to migrate to a new cluster then. Any better alternatives than creating a small application that reads from one cluster and inserts in the new one that anybody can su

Re: I don't understand shuffle progress

2013-09-17 Thread Robert Coli
On Tue, Sep 17, 2013 at 12:13 PM, Juan Manuel Formoso wrote: > I am running shuffle on a cluster after upgrading to 1.2.X, and I don't > understand how to check progress. > If your shuffle succeeds, you will be the first reported case of shuffle succeeding on a non-test cluster. Until I hear a re

I don't understand shuffle progress

2013-09-17 Thread Juan Manuel Formoso
I am running shuffle on a cluster after upgrading to 1.2.X, and I don't understand how to check progress. I'm counting the lines of cassandra-shuffle ls, and it decreases VERY slowly. Sometimes not at all after 24 hours of processing. Is that value accurate? Does the shuffle operation supports di