Re: ***UNCHECKED*** Re: Memory Recommendations for G1GC

2019-11-04 Thread Ben Mills
Thanks again Reid - another great response - you have pointed me in the right direction. On Mon, Nov 4, 2019 at 12:23 PM Reid Pinchback wrote: > It’s not a setting I’ve played with at all. I understand the gist of it > though, essentially it’ll let you automatically adjust your JVM size > relat

Re: ***UNCHECKED*** Re: Memory Recommendations for G1GC

2019-11-04 Thread Reid Pinchback
It’s not a setting I’ve played with at all. I understand the gist of it though, essentially it’ll let you automatically adjust your JVM size relative to whatever you allocated to the cgroup. Unfortunately I’m not a K8s developer (that may change shortly, but atm the case). What you need to a

Re: ***UNCHECKED*** Re: Memory Recommendations for G1GC

2019-11-04 Thread Ben Mills
Hi Reid, Many thanks for this thoughtful response - very helpful and much appreciated. No doubt some additional experimentation will pay off as you noted. One additional question: we currently use this heap setting: -XX:MaxRAMFraction=2 I realize every environment and its tuning goals are diff

Re: ***UNCHECKED*** Re: Memory Recommendations for G1GC

2019-11-04 Thread Reid Pinchback
Hi Ben, just catching up over the weekend. The typical advice, per Sergio’s link reference, is an obvious starting point. We use G1GC and normally I’d treat 8gig as the minimal starting point for a heap. What sometimes doesn’t get talked about in the myriad of tunings, is that you have to hav

Re: ***UNCHECKED*** Re: Memory Recommendations for G1GC

2019-11-04 Thread Ben Mills
Hi (yet again) Sergio, Finally, note that we use this sidecar for shipping metrics to Stackdriver. It runs as a second container within our Prometheus stateful set. On Mon, Nov 4, 2019 at 8:46 AM Ben Mills wrote: > Hi (again) Serg

Re: ***UNCHECKED*** Re: Memory Recommendations for G1GC

2019-11-04 Thread Ben Mills
Hi (again) Sergio, I forgot to note that along with Prometheus, we use Grafana (with Prometheus as its data source) as well as Stackdriver for monitoring. As Stackdriver is still developing (i.e. does not have all the features we need), we tend to use it for the basics (i.e. monitoring and alerti

Re: ***UNCHECKED*** Re: Memory Recommendations for G1GC

2019-11-04 Thread Ben Mills
Hi Sergio, Thanks for this and sorry for the slow reply. We are indeed still running Java 8 and so it's very helpful. This Cassandra cluster has been running reliably in Kubernetes for several years, and while we've had some repair-related issues, they are not related to container orchestration