My first thought was that you were running into the merkle tree depth problem, but the details on the ticket don’t seem to confirm that.
It does look like eden is too small. C* lives in Java’s GC pain point, a lot of medium-lifetime objects. If you haven’t already done so, you’ll want to configure as many things to be off-heap as you can, but I’d definitely look at improving the ratio of eden to old gen, and see if you can get the young gen GC activity to be more successful at sweeping away the medium-lived objects. All that really comes to mind is if you’re getting to a point where GC isn’t coping. That can be hard to sometimes spot on metrics with coarse granularity. Per-second metrics might show CPU cores getting pegged. I’m not sure that GC tuning eliminates this problem, but if it isn’t being caused by that, GC tuning may at least improve the visibility of the underlying problem. From: "Steinmaurer, Thomas" <thomas.steinmau...@dynatrace.com> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org> Date: Wednesday, November 6, 2019 at 11:27 AM To: "user@cassandra.apache.org" <user@cassandra.apache.org> Subject: Cassandra 3.0.18 went OOM several hours after joining a cluster Message from External Sender Hello, after moving from 2.1.18 to 3.0.18, we are facing OOM situations after several hours a node has successfully joined a cluster (via auto-bootstrap). I have created the following ticket trying to describe the situation, including hprof / MAT screens: https://issues.apache.org/jira/browse/CASSANDRA-15400<https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_CASSANDRA-2D15400&d=DwMF-g&c=9Hv6XPedRSA-5PSECC38X80c1h60_XWA4z1k_R1pROA&r=OIgB3poYhzp3_A7WgD7iBCnsJaYmspOa2okNpf6uqWc&m=lnQdpMrbVjmjj_af9BwSn1ftI8H2uSyvAya3887aDLk&s=BEeQbrRZS6Z1i25NSdwRmQVpQ36AvSNz_i8Y9ks5UmA&e=> Would be great if someone could have a look. Thanks a lot. Thomas The contents of this e-mail are intended for the named addressee only. It contains information that may be confidential. Unless you are the named addressee or an authorized designee, you may not copy or use it, or disclose it to anyone else. If you received it in error please notify us immediately and then destroy it. Dynatrace Austria GmbH (registration number FN 91482h) is a company registered in Linz whose registered office is at 4040 Linz, Austria, Freistädterstraße 313
