Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

Thanks again guys, this has been a major blocker for us and I think we've made some major progress with your advice. We have gone ahead with Lerh's suggestion and the cluster is operating much more smoothly while the new node compacts. We read at quorum, so in the event that we don't make it withi

Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

> > > Kurt - We're on 3.7, and our approach was to try thorttling compaction > throughput as much as possible rather than the opposite. I had found some > resources that suggested unthrottling to let it get it over with, but > wasn't sure if this would really help in our situation since the I/O pip

Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

Hi Paul, Agh, I don't have any experience with sstableofflinerelevel. Maybe Kurt does, sorry. Also, if it wasn't obvious, to add back the node to the cluster once it is done would be the 3 commands, with enable substituted for disable. It feels like it will take some time to get through all the c

Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

Unsubscribe On Mon, Sep 11, 2017 at 4:48 PM, Paul Pollack wrote: > Hi, > > We run 48 node cluster that stores counts in wide rows. Each node is using > roughly 1TB space on a 2TB EBS gp2 drive for data directory and > LeveledCompactionStrategy. We have been trying to bootstrap new nodes that > u

Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

Thanks for the responses Lerh and Kurt! Lerh - We had been considering those particular nodetool commands but were hesitant to perform them on a production node without either testing adequately in a dev environment or getting some feedback from someone who knew what they were doing (such as yours

Re: load distribution that I can't explain

Your first query will effectively have to perform table scans to satisfy what you are asking. If a query requires ALLOW FILTERING to be specified, it means that Cassandra can't really optimise that query in any way and it's going to have to query a lot of data (all of it...) to satisfy the result.

Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

What version are you using? There are improvements to streaming with LCS in 2.2. Also, are you unthrottling compaction throughput while the node is bootstrapping? ​

Re: Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

Hi Paul, The new node will certainly have a lot of compactions to deal with being LCS. Have you tried performing the following on the new node once it has joined? *nodetool disablebinary && nodetool disablethrift && nodetooldisablegossip* This will disconnect Cassandra from the cluster, but not

Re: Cassandra downgrade of 2.1.15 to 2.1.12

On 09/11/2017 06:29 PM, Mark Furlong wrote: > I have a requirement to test a downgrade of 2.1.15 to 2.1.12. Can > someone please identify how to achieve this? Downgrades have never been officially supported, but this is a relatively small step. Testing it out is definitely a good thing. Since prot

Bootstrapping node on Cassandra 3.7 causes cluster-wide performance issues

Hi, We run 48 node cluster that stores counts in wide rows. Each node is using roughly 1TB space on a 2TB EBS gp2 drive for data directory and LeveledCompactionStrategy. We have been trying to bootstrap new nodes that use a raid0 configuration over 2 1TB EBS drives to increase I/O throughput cap f

Cassandra downgrade of 2.1.15 to 2.1.12

I have a requirement to test a downgrade of 2.1.15 to 2.1.12. Can someone please identify how to achieve this? Thanks, Mark Furlong Sr. Database Administrator mfurl...@ancestry.com M: 801-859-7427 O: 801-705-7115 1300 W Traverse Pkwy Lehi, UT 84043 ​[http://c.

load distribution that I can't explain

Hi every one So I have a 2 node( node1, node2 ) cassandra 3.11 cluster on which I have a keyspace with a replication factor of 2. this keyspace has only this table: CREATE KEYSPACE myks WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '2'} AND durable_writes = true; CR

Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+

Hi, Is this bug fixed in dse 5.1.3? As I understand calling jmx getTombStoneRatio trigers that bug. We are using opscenter as well and do you have any idea whether opscenter using/calling this method? Thanks On Aug 29, 2017 6:35 AM, "Jeff Jirsa" wrote: > I shouldn't actually say I don't think

Re: Cassandra 3.7 repair error messages

Thanks Erick, and sorry it took me so long to respond, I had to turn my attention to other things. It definitely looks like there had been some network blips going on with that node for a while before we saw it marked down from every other node's perspective. Additionally, my original comment that

Re: Weird error (unable to start cassandra)

I had to do brew upgrade jemalloc to fix this issue. On Mon, Sep 11, 2017 at 4:25 AM, Kant Kodali wrote: > Hi All, > > I am trying to start cassandra 3.11 on Mac OS Sierra 10.12.6. when invoke > cassandra binary I get the following error > > java(2981,0x7fffedb763c0) malloc: *** malloc_zone_unre

Weird error (unable to start cassandra)

Hi All, I am trying to start cassandra 3.11 on Mac OS Sierra 10.12.6. when invoke cassandra binary I get the following error java(2981,0x7fffedb763c0) malloc: *** malloc_zone_unregister() failed for 0x7fffedb6c000 I have xcode version 8.3.3 installed (latest). Any clue ? Thanks!

Re: Self-healing data integrity?

Agree A tricky detail about streaming is that: 1) On the sender side, the node just send the SSTable (without any other components like CRC files, partition index, partition summary etc...) 2) The sender does not even bother to de-serialize the SSTable data, it is just sending the stream of byte