Hi Maxim, I recently had similar troubles related to ExecutorService and exception handling. It was an unexpected behavior for me that exceptions of any kind* thrown from Runnable in a following code: executor.submit(() -> throw new RuntimeException()) were silently ignored.
I seemed to me really unexpected for me at first. But later on I figured out the cause. There is a subtle detail which I forgot. ExecutorService.submit returns a Future and leaves a responsibility to handle exceptions of any kind to a caller of Future.get. And it was really subtle for me, because I did not use a returned Future at all. Actually I supposed ExecutorService.execute when used ExecutorService.submit. Is it related to the subject? Can ExecutorService.execute help here? * In my case there were exceptions caused by missing classes on a classpath. 2020-05-21 22:22 GMT+03:00, Maxim Muzafarov <mmu...@apache.org>: > Folks, > > I've created an issue. > https://issues.apache.org/jira/browse/IGNITE-13055 > > On Wed, 6 May 2020 at 10:28, Nikolay Izhikov <nizhi...@apache.org> wrote: >> >> Hello, Maxim. >> >> I can confirm this itching issue. >> It also happens when some custom Security plugin throws an exception while >> processing a communication message. >> >> ``` >> UUID newSecSubjId = secSubjId != null ? secSubjId : nodeId; >> >> try (OperationSecurityContext s = >> ctx.security().withContext(newSecSubjId)) { >> lsnr.onMessage(nodeId, msg, plc); >> } >> finally { >> if (change) >> CUR_PLC.set(oldPlc); >> } >> ``` >> >> If an exception thrown from `withContext` it is hidden from the user. >> >> > 4 мая 2020 г., в 19:15, Maxim Muzafarov <mmu...@apache.org> написал(а): >> > >> > Igniters, >> > >> > >> > I've spent a couple of days in debug mode and found that some of the >> > configured ExecutorServices of an Ignite instance completely hide >> > assertion errors without any logging and node killing. This may >> > violate some internal guarantees and hide serious errors. >> > >> > Let's consider, for instance, GridDhtPartitionsExchangeFuture [1]. It >> > has three places of submitting some Runnable on system executor >> > service. If an assertion error (or even any uncached exception) occurs >> > in the code block below it will be swallowed without logging, exchange >> > future completion or node stoping. >> > >> > cctx.kernalContext().getSystemExecutorService().submit(new Runnable() { >> > @Override public void run() { >> > sendPartitions(newCrd); >> > } >> > }); >> > >> > I've checked that these executor services and most of them configured >> > to catch only OutOfMemoryError. >> > >> > Should we consider catching AssertionErrors as well and treat them as >> > CRITICAL_ERRORS for the Failure Handler? >> > Should we log uncached errors on each of them? >> > >> > >> > The most important list of executor services configured with OOM handler >> > only: >> > execSvc >> > svcExecSvc >> > sysExecSvc >> > p2pExecSvc >> > restExecSvc >> > utilityCacheExecSvc >> > affExecSvc >> > qryExecSvc >> > >> > [1] >> > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java#L4848 >> > -- Best regards, Ivan Pavlukhin
