Another issue you'll need to consider is how the JVM allocates resources towards GC, especially if you're using G1 with a pause time goal. Specifically, if you let it pick it's own numbers for ParallelGCThreads & ConcGCThreads they'll be based on the total number of CPUs, not the number you've restricted access to, and you'll end up with GC taking way longer than it should. Anything that auto sizes based on available cores will also be affected. Set -XX:ParallelGCThreads=n and -XX:ConcGCThreads=n to be no more than the number of cores you're going to allocate to each JVM (by default CMS uses ConcGCThreads=8+(# proc-8)*(5/8)).
This is also a problem when working with containers, the JVM sees all the cores not the resource limit. Some other good reading on the topic: https://xmlandmore.blogspot.com/2014/06/oracle-has-published-some-tuning-guides.html Jon On Thu, Oct 25, 2018 at 7:51 AM Jeff Jirsa <jji...@gmail.com> wrote: > I don't have time to reply to your stackoverflow post, but what you > proposed is a great idea for a server that size. > > You can use taskset or numactl to bind each JVM to the appropriate > cores/zones. > Setup a data directory on each SSD for the data > > There are two caveats you need to think about: > > 1) You'll need an IP per instance - so bind a second IP to each NIC and > you should be good to go. > 2) You need to make sure you dont have 2 replicas on the same host. > Cassandra has "rack aware" snitches that can help with this, but you need > to make sure that you treat each server as it's own rack to force replicas > to land on different physical machines. > > If you do both of those things, you'll probably be happy with the result. > > > > > On Wed, Oct 24, 2018 at 12:33 PM Bobbie Haynes <haynes30...@gmail.com> > wrote: > >> I have three Physical servers. I want to run cassandra on multiple JVM's >> i.e each physical node contains 2 cassandra nodes so that i could able to >> run 6 node cluster.Could anyone help me pointing setup guide. >> >> Each Physical node Configuration:- RAM -256 GB --- I want to assign each >> JVM(64GB) cores --64 --- 32 cores each node. 6TB SSD .. seperate SSD disk >> for each node 3TB each. >> >> >> >> https://stackoverflow.com/questions/52976646/cassandra-running-multiple-jvms >> > -- Jon Haddad http://www.rustyrazorblade.com twitter: rustyrazorblade
