Changing replication factor from 2 to 3

2010-06-02 Thread Eric Halpern
We'd like to double our cluster size from 4 to 8 and increase our replication factor from 2 to 3. Is there any special procedure we need to follow to increase replication? Is it sufficient to just start the new nodes with the replication factor of 3 and then reconfigure the existing nodes to the

Re: Nodes dropping out of cluster due to GC

2010-06-02 Thread Eric Halpern
Ryan King wrote: > > Why run with so few nodes? > > -ryan > > On Tue, Jun 1, 2010 at 4:20 PM, Eric Halpern wrote: >> >> Hello, >> >> We're running a 4 node cluster on beefy EC2 virtual instances (8 core, 32 >> GB) using EBS storage with

Re: Nodes dropping out of cluster due to GC

2010-06-02 Thread Eric Halpern
Oleg Anastasjev wrote: > >> >> Has anyone experienced this sort of problem? It would be great to hear >> from >> anyone who has had experience with this sort of issue and/or suggestions >> for >> how to deal with it. >> >> Thanks, Eric > > Yes, i did. Symptoms you described point to concur

Nodes dropping out of cluster due to GC

2010-06-01 Thread Eric Halpern
Hello, We're running a 4 node cluster on beefy EC2 virtual instances (8 core, 32 GB) using EBS storage with 8 GB of heap allocated to the JVM. Every couple of hours, each of the nodes does a concurrent mark/sweep that takes around 30 seconds to complete. During that GC, the node temporarily dro