On 18/01/2024 09:30, Johannes Spangenberg wrote:
Hello,
I have a question about dealing with side effects caused by ForkJoinPool. I am
not certain if this mailing list is the appropriate platform, but I hope it is.
The issue I am facing is that running code within a ForkJoinPool may change the
behavior of the code. These changes in behavior have resulted in
non-deterministic test failures in one of the repositories I work on. JUnit
uses a ForkJoinPool when you enable parallel test execution. I would like to
disable or avoid the side effects of various methods if called from a
ForkJoinPool. So far, I haven't found a solution, except for completely
replacing the use of the ForkJoinPool in JUnit with some custom scheduler which
is not a ForkJoinPool.
Is there a solution to force an "unmanaged" block (as opposed to
ForkJoinPool.managedBlock)? Is there alternatively a good approach to transfer CPU bound
subtasks to another thread pool while blocking the ForkJoinWorkerThread without
compensation? I have implemented a workaround which I explain below, but I am not sure if
this will remain effective in future JDK versions. I am currently using JDK 17 but were
also able to reproduce the issues with JDK 21.
I have observed the following side-effects caused by managed blocks or similar
mechanisms:
1. Parallel stream methods execute another task (i.e. JUnit test) from the pool
recursively, which is particularly problematic if your code utilizes any
ThreadLocal.
2. Blocking methods spawn around 256 threads in total to "compensate" for the
blocking operation. Consequently, you end up with up to 256 tests running concurrently,
each of them might or might not be CPU bound (unknown to the ForkJoinPool).
3. Blocking methods may throw a RejectedExecutionException when the thread
limit is reached. This is effectively a race condition which may lead to
exceptions.
I have not been able to determine under which circumstances each behavior
occurs. I am unaware of any thorough documentation that clearly outlines the
expected behavior in different scenarios with different blocking methods. While
(1.) and (3.) have caused test failures, (2.) simply causes JUnit to run 256
tests in parallel instead of the intended 12. I attached a JUnit test to
reproduce (1.) and (3.), but it might not fail on every run.
Many of the blocking methods of the standard library include a check if the current
thread is an instance of ForkJoinWorkerThread. My current workaround involves wrapping
the code that makes blocking calls into a FutureTask which is executed on another thread
and then joining this task afterwards. As of now, FutureTask.get() seems not to implement
any of the side-effects. As the missing instanceof-check in FutureTask makes it
inconsistent with other Futures like CompletableFuture, I fear it might be considered a
"bug". I would like to know a safe solution which is specified to continue to
work in future JDKs.
I think it would be useful to understand how JUnit creates the
ForkJoinPool. The reason is that it controls the parallelism and "max
pool size". If there is interference due to managedBlocker then JUnit
can set maxPoolSize to the same value as parallelism.
On the REE, this is also controlled by JUnit when it creates the FJP.
The saturate parameter is the predicate that is determines if REE is
thrown or the pool continues without an additional thread.
-Alan
