Re: Transitioning to incremental repair

2015-12-02 Thread Marcus Eriksson
Bryan, this should be improved with https://issues.apache.org/jira/browse/CASSANDRA-10768 - could you try it out? On Tue, Dec 1, 2015 at 10:58 PM, Bryan Cheng wrote: > Sorry if I misunderstood, but are you asking about the LCS case? > > Based on our experience, I would absolutely recommend you c

Re: Cassandra compaction stuck? Should I disable?

2015-12-02 Thread PenguinWhispererThe .
So it seems I found the problem. The node opening a stream is waiting for the other node to respond but that node never responds due to a broken pipe which makes Cassandra wait forever. It's basically this issue: https://issues.apache.org/jira/browse/CASSANDRA-8472 And this is the workaround/fix:

Re: Transitioning to incremental repair

2015-12-02 Thread Bryan Cheng
Ah Marcus, that looks very promising- unfortunately we have already switched back to full repairs and our test cluster has been re-purposed for other tasks atm. I will be sure to apply the patch/try a fixed version of Cassandra if we attempt to migrate to incremental repair again.

cassandra reads are unbalanced

2015-12-02 Thread Walsh, Stephen
Hey all, Thanks for taking the time to help. So we have 6 cassandra nodes in 2 Data Centers. Both Data Centers have a replication of 3 - so all nodes have all the data. Over the last 2 days we've noticed that data reads / writes has shifted from balanced to unbalanced (Nodetool status still sho

Restoring a snapshot into a new cluster - thoughts on replica placement

2015-12-02 Thread Peer, Oded
I read the documentation for restoring a snapshot into a new cluster. It got me thinking about replica placement in that context. "NetworkTopologyStrategy places replicas in the same data center by walking the ring clockwise until reaching the first node in another rack." It seems it is not enoug

Re: cassandra reads are unbalanced

2015-12-02 Thread DuyHai Doan
Which Consistency level do you use for reads ? ONE ? Are you reading from only DC1 or from both DC ? What is the LoadBalancingStrategy you have configured for your driver ? TokenAware wrapped on DCAwareRoundRobin ? On Wed, Dec 2, 2015 at 3:36 PM, Walsh, Stephen wrote: > Hey all, > > > > Tha

RE: cassandra reads are unbalanced

2015-12-02 Thread Walsh, Stephen
Very good questions. We have reads and writes at LOCAL_ONE. There are 2 application (1 for each DC) who read and write at the same rate to their local DC (All reads / writes started all perfectly even and degraded over time) We use DCAwareRoundRobin policy On update on the nodetool cleanup – it

Re: cassandra reads are unbalanced

2015-12-02 Thread DuyHai Doan
If you're using the Java driver with LOCAL_ONE and the default load balancing strategy (TokenAware wrapped on DCAwareRoundRobin), the driver will always select the primary replica. To change this behavior and introduce some randomness so that non primary replicas get a chance to serve a read: new

Want to run repair on a node without it taking traffic

2015-12-02 Thread K F
Hi Folks, How can I run repair on a node without it taking any coordinator/client traffic. So, I can complete the repair on the node without it taking any traffic, except the streams from other nodes. Is that possible? Thanks.

Re: Want to run repair on a node without it taking traffic

2015-12-02 Thread Robert Coli
On Wed, Dec 2, 2015 at 8:54 AM, K F wrote: > How can I run repair on a node without it taking any coordinator/client > traffic. So, I can complete the repair on the node without it taking any > traffic, except the streams from other nodes. Is that possible? > In general you should probably just

Re: Restoring a snapshot into a new cluster - thoughts on replica placement

2015-12-02 Thread Robert Coli
On Wed, Dec 2, 2015 at 5:06 AM, Peer, Oded wrote: > It seems it is not enough to restore the token ranges on an equal-size > cluster since you also need to restore the rack information. > Yep, if you're using a rack-aware snitch, that is correct. Because in that case, rack determines replica pla

Unable to add nodes / awaiting patch.

2015-12-02 Thread Jeff Ferland
Looks like we’re hit by https://issues.apache.org/jira/browse/CASSANDRA-10012 . Not knowing a better place to ask, when will the next version of 2.1.x Cassandra be cut and the following DSE fix cut from there? Could DSE cut an in-between ver

Re: Unable to add nodes / awaiting patch.

2015-12-02 Thread Michael Shuler
On 12/02/2015 01:54 PM, Jeff Ferland wrote: > Looks like we’re hit > by https://issues.apache.org/jira/browse/CASSANDRA-10012. Not knowing a > better place to ask, when will the next version of 2.1.x Cassandra be > cut and the following DSE fix cut from there? Could DSE cut an > in-between version

Re: Issues on upgrading from 2.2.3 to 3.0

2015-12-02 Thread Bryan Cheng
Has your configuration changed? This is a new check- https://issues.apache.org/jira/browse/CASSANDRA-10242. It seems likely either your snitch changed, your properties changed, or something caused Cassandra to think one of the two happened... What's your node layout? On Fri, Nov 27, 2015 at 6:45

Re: Issues on upgrading from 2.2.3 to 3.0

2015-12-02 Thread Carlos A
Bryan, thanks for replying. I had that figured out already few days ago. The issue was that the snitch method was also changed on the configuration hence the problem. Now, if you change from rackInferingSnitch to PropertiesFileSnitch it will not run as the data has to be migrated. Is that correct?