Re: 2.0.10 to 2.0.11 upgrade and immediate ParNew and CMS GC storm

2014-10-29 Thread Peter Haggerty
the default). > > Ciao, Duncan. > > > On 29/10/14 01:08, Peter Haggerty wrote: >> >> On a 3 node test cluster we recently upgraded one node from 2.0.10 to >> 2.0.11. This is a cluster that had been happily running 2.0.10 for >> weeks and that has very little load an

2.0.10 to 2.0.11 upgrade and immediate ParNew and CMS GC storm

2014-10-28 Thread Peter Haggerty
On a 3 node test cluster we recently upgraded one node from 2.0.10 to 2.0.11. This is a cluster that had been happily running 2.0.10 for weeks and that has very little load and very capable hardware. The upgrade was just your typical package upgrade: $ dpkg -s cassandra | egrep '^Ver|^Main' Mainta

Should a node that is bootstrapping be receiving writes in addition to the streams it is receiving?

2014-10-15 Thread Peter Haggerty
We bootstrapped a node with replace_address and noticed that the WriteCount for each CF stayed at 0 until the bootstrap was complete. After the bootstrap completed cfstats reported the expected values for WriteCount on each CF. The node wrote gigs of data to various CFs during the bootstrap so it

When CHANGES and JIRA "Fix Versions" disagree which should we believe?

2014-09-08 Thread Peter Haggerty
When the CHANGES file shows an issue as being in a particular release but the JIRA for the issue shows a different version in "Fix Versions" which one is right? All four of these are listed in 2.0.10 in the CHANGES file: https://github.com/apache/cassandra/blob/cassandra-2.0/CHANGES.txt https://i

Re: Bootstrap failures: unable to find sufficient sources for streaming range

2014-08-15 Thread Peter Haggerty
Peter On Fri, Aug 15, 2014 at 3:36 AM, Mark Reddy wrote: > Hi Peter, > > At the time of the IllegalStateException, do you see the node that it should > be streaming from marked as down by the failure detector? > > > Mark > > > On Fri, Aug 15, 2014 at 5:45 AM, Pe

Re: Bootstrap failures: unable to find sufficient sources for streaming range

2014-08-15 Thread Peter Haggerty
ion, do you see the node that it should > be streaming from marked as down by the failure detector? > > > Mark > > > On Fri, Aug 15, 2014 at 5:45 AM, Peter Haggerty > wrote: >> >> When adding nodes via bootstrap to a 27 node 2.0.9 cluster with a >> cluster-wide p

Bootstrap failures: unable to find sufficient sources for streaming range

2014-08-14 Thread Peter Haggerty
When adding nodes via bootstrap to a 27 node 2.0.9 cluster with a cluster-wide phi_convict_threshold of 12 the nodes fail to bootstrap. This worked a half dozen times in the past few weeks as we've scaled this cluster from 21 to 24 and then to 27 nodes. There have been no configuration or Cassandra

Is there guidance about compaction thresholds and setting minthreshold to 2?

2013-04-04 Thread Peter Haggerty
The default minthreshold for compactions is 4: http://www.datastax.com/docs/1.1/references/nodetool#nodetool-setcompactionthreshold Is there a reason that this value is not "2", the lowest possible value? If we change this to 2 what should we expect to see? Should we see less growth in storage lo