Major changes in 3.0, but the big ones involve storage - parent ticket is https://issues.apache.org/jira/browse/CASSANDRA-9427
64G cms heap with 4G new gen is somewhat atypical - would expect g1 at that size, or at the very least a larger new gen to try to avoid promotion which will bring you that parnew pause. -- Jeff Jirsa > On Sep 19, 2016, at 11:12 PM, Dikang Gu <dikan...@gmail.com> wrote: > > In our 2.1 cluster, I find that hints handoff is using a lot of memory on > our proxy nodes, when delivering hints to a data node that was dead for 3+ > hours (our hints window is 3 hours). It makes the young gen GC time as long > as 2 secs. > > I'm using 64G max heap size, and 4G young gen size. I'm considering > increasing the young gen size, I'm wondering any improvements to hints > handoff in newer version? > > some logs: > 2016-09-20_05:04:30.67928 INFO 05:04:30 [Service Thread]: ParNew GC in > 2200ms. CMS Old Gen: 49112623080 -> 50052787792; Par Eden Space: > 2863398912 -> 0; > > mem usage: > 2016-09-20T01:25:23.522+0000 Process summary > process cpu=0.00% > application cpu=140.36% (user=132.28% sys=8.08%) > other: cpu=-140.36% > heap allocation rate 1500mb/s > [001343] user=90.42% sys= 0.01% alloc= 1439mb/s - HintedHandoff:3 > [290234] user=18.85% sys= 5.14% alloc= 40mb/s - CompactionExecutor:176226 > [064445] user=17.57% sys= 1.09% alloc= 10mb/s - RMI TCP > Connection(148376)-127.0.0.1 > [290439] user= 3.20% sys= 1.10% alloc= 4607kb/s - RMI TCP > Connection(148380)-127.0.0.1 > [000016] user= 0.75% sys=-0.03% alloc= 2321kb/s - ScheduledTasks:1 > [000149] user= 0.32% sys=-0.04% alloc= 1516kb/s - GossipStage:1 > [001370] user= 0.11% sys= 0.04% alloc= 420kb/s - GossipTasks:1 > > Thanks > Dikang.
