[
https://issues.apache.org/jira/browse/CAMEL-21671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-21671:
--------------------------------
Fix Version/s: 4.11.0
(was: 4.x)
> Issue with Camel Splitter and Asynchronous Processing in Parallel Mode
> ----------------------------------------------------------------------
>
> Key: CAMEL-21671
> URL: https://issues.apache.org/jira/browse/CAMEL-21671
> Project: Camel
> Issue Type: Improvement
> Components: came-core
> Affects Versions: 4.8.0
> Reporter: Soheila Esmaeili
> Assignee: Claus Ibsen
> Priority: Minor
> Fix For: 4.11.0
>
> Attachments: SplitterParallelAsyncProcessorTest.java
>
>
> We are currently facing an issue with a Camel route that uses a splitter with
> parallelProcessing. The route includes an asynchronous processor and a delay
> within the splitter. However, the problem is that the split threads do not
> wait for the asynchronous process to complete before proceeding further.
> Although the max. number of threads for split processing is set to 2, we have
> no control over thread usage due to the async processor. In *Camel 3.14* and
> {*}Camel 4.8{*}, the 2 threads will forward the message to the async
> processor, then continue with the next 2 split parts, which are also handed
> over to another thread pool and so on. This can lead to hundreds of parallel
> threads performing the operations defined within the split-block. If these
> operations are long-running and memory-intensive, this can lead to further
> problems. {*}In the older (unmaintained – 2.x) Camel version{*}, this was not
> the case. Although the threads changed due to the async processor, the
> “split-related threads” were in a blocking/waiting state (I assume). At least
> there was no “unbounded” upscaling in the number of threads.
> Could you please confirm if this is a known issue or if there are specific
> configurations needed in newer Camel versions to achieve the expected
> behavior?
> A simplified unit test can be found in the appendix.
> *Result in Camel 2.24:*
> Before AsyncProcessor - Thread: Camel (camel-1) thread #2 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #1 - Split
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-1
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-2
> Before AsyncProcessor - Thread: Camel (camel-1) thread #2 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #1 - Split
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-1
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-2
> Before AsyncProcessor - Thread: Camel (camel-1) thread #2 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #1 - Split
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-2
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-1
> Before AsyncProcessor - Thread: Camel (camel-1) thread #1 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #2 - Split
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-2
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-1
> Before AsyncProcessor - Thread: Camel (camel-1) thread #1 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #2 - Split
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-2
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-1
>
> *Result in 3.14 and 4.8:*
> Before AsyncProcessor - Thread: Camel (camel-1) thread #2 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #3 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #2 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #2 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #3 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #2 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #2 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #2 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #2 - Split
> Before AsyncProcessor - Thread: Camel (camel-1) thread #3 - Split
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-6
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-3
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-5
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-10
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-8
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-2
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-9
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-7
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-1
> After AsyncProcessor - Thread: ForkJoinPool.commonPool-worker-4
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
