Hello, Maxim. I can confirm this itching issue. It also happens when some custom Security plugin throws an exception while processing a communication message.
``` UUID newSecSubjId = secSubjId != null ? secSubjId : nodeId; try (OperationSecurityContext s = ctx.security().withContext(newSecSubjId)) { lsnr.onMessage(nodeId, msg, plc); } finally { if (change) CUR_PLC.set(oldPlc); } ``` If an exception thrown from `withContext` it is hidden from the user. > 4 мая 2020 г., в 19:15, Maxim Muzafarov <mmu...@apache.org> написал(а): > > Igniters, > > > I've spent a couple of days in debug mode and found that some of the > configured ExecutorServices of an Ignite instance completely hide > assertion errors without any logging and node killing. This may > violate some internal guarantees and hide serious errors. > > Let's consider, for instance, GridDhtPartitionsExchangeFuture [1]. It > has three places of submitting some Runnable on system executor > service. If an assertion error (or even any uncached exception) occurs > in the code block below it will be swallowed without logging, exchange > future completion or node stoping. > > cctx.kernalContext().getSystemExecutorService().submit(new Runnable() { > @Override public void run() { > sendPartitions(newCrd); > } > }); > > I've checked that these executor services and most of them configured > to catch only OutOfMemoryError. > > Should we consider catching AssertionErrors as well and treat them as > CRITICAL_ERRORS for the Failure Handler? > Should we log uncached errors on each of them? > > > The most important list of executor services configured with OOM handler only: > execSvc > svcExecSvc > sysExecSvc > p2pExecSvc > restExecSvc > utilityCacheExecSvc > affExecSvc > qryExecSvc > > [1] > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java#L4848