Ignite cluster stability problems under heavy load

2021-10-06 Thread Piotr Jagielski
Hi, We experience stability problems on our Ignite cluster (2.10) under heavy load. Our cluster nodes are 3x 8 CPU, 32GB RAM. We mainly use 2 persistent caches: - aggregates - only updates, around 6K records / sec, ~70 mln records total, stored mostly on disk (dataRegion maxSize = 4GB) - custo

Re: Ignite cluster stability problems under heavy load

2021-10-06 Thread Anton Kurbanov
Hello Piotr, Please share the configuration and the logs, preferably with thread dumps attached during the time when system_worker_blocked message pops up. It is difficult to diagnose an issue with only metrics involved, as for example, the checkpoint itself has several phases which might be long

Re: Re[2]: What does "First 10 long running cache futures" ?

2021-10-06 Thread John Smith
Ok. For now I rebooted all nodes... But it's fairly easy to reproduce. On Wed., Oct. 6, 2021, 2:17 a.m. Zhenya Stanilovsky, wrote: > > Ok, seems something goes wrong on node with > id=36edbfd5-4feb-417e-b965-bdc34a0a6f4f If you still have a problem, can u > send here or directly by me these logs

Ignite Summit Cloud Edition Event Page/Community

2021-10-06 Thread Kevin Corbett
Hello Igniters! Ignite Summit Cloud Edition is coming up pretty soon and I thought it would be great to help setup an event page on LinkedIn to spread awareness. When someone signs up, a notification is sent to the person’s network which helps encourage more people to sign up to attend as well