Thanks! But getting back to the original issue: I think the GC itself is not the root cause for such a long pause. I remember having had issues with 1 minute GCs in the beginning. I also tried around with larger and smaller heap sizes and different GCs (G1, CMS), different settings but what helped in the end (as far as I remember - please nail me down on that) was to increase memtable flush writers. I could explain it like that: If available mem is getting fuller and fuller, the GC has to ran more often and longer to reclaim the last available bit that is currently required. Memtables use a considerable amount of it and if they can't be flushed in time, they grow and use more and more memory putting more and more pressure on the GC - also known as the GC death spiral. In my case I never ran in an OOM crash but the node became totally unresponsive.
I don't tell this must be the case here but it is one possible case. P.S.: In my case memtable flush writers default was 2 AFAIR as I had only one SSD but the node could easily handle many more with 8 real cores and an SSD. 2016-11-26 7:52 GMT+01:00 Work <jrother...@codojo.me>: > I'm not affiliated with them, I've just been impressed by them. They have > done amazing work in performance measurement. They discovered a major flaw > in most performance testing ... I've never seen their pricing. But, > recently, they made their product available for testing by developers. And > the assured me that pricing is on a sliding scale depending upon > utilization, and not ridiculous. > > - James > > Sent from my iPhone > > On Nov 25, 2016, at 10:40 PM, Benjamin Roth <benjamin.r...@jaumo.com> > wrote: > > This sounds amazing but also expensive - I don't see pricing on their > page. Are you able and allowed to tell a rough pricing range? > > Am 26.11.2016 04:33 schrieb "Harikrishnan Pillai" <hpil...@walmartlabs.com > >: > >> We are running azul zing in prod with 1 million reads/s and 100 K >> writes/s with azul .we never had a major gc above 10 ms . >> >> Sent from my iPhone >> >> > On Nov 25, 2016, at 3:49 PM, Martin Schröder <mar...@oneiros.de> wrote: >> > >> > 2016-11-25 23:38 GMT+01:00 Kant Kodali <k...@peernova.com>: >> >> I would also restate the following sentence "java GC pauses are pretty >> much >> >> a fact of life" to "Any GC based system pauses are pretty much a fact >> of >> >> life". >> >> >> >> I would be more than happy to see if someone can counter prove. >> > >> > Azul disagrees. >> > https://www.azul.com/products/zing/pgc/ >> > >> > Best >> > Martin >> > -- Benjamin Roth Prokurist Jaumo GmbH · www.jaumo.com Wehrstraße 46 · 73035 Göppingen · Germany Phone +49 7161 304880-6 · Fax +49 7161 304880-1 AG Ulm · HRB 731058 · Managing Director: Jens Kammerer
