Igniters,
We have one well-known limitation: one distributed opeartion cannot be
executed from within another if they both end up in the same pool, because
it will lead to starvation and hang. Most notable example - running data
streamer from compute task.
Users with complex logic will have the same issue if they want to execute
one job from another. I think it could be resolved with introduction of
custom executors and routing computing jobs to them.
Config:
ExecutorConfiguration execCfg = new
ExecutorConfiguration().setName("myExecutor").setThreads(8);
Starting MR:
ignite.compute().withExecutor("myExecutor").execute(...);
Another improvement we can apply here is managed blockers, similar to
ForkJoinPool.ManagedBlocker [1]. If user is to execute distributed
operation, he can infrom us about it, and we will compensate blocked thread
with a new one.
Thoughts?
[1]
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ForkJoinPool.ManagedBlocker.html
