[
https://issues.apache.org/jira/browse/CAMEL-23975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18095884#comment-18095884
]
Claus Ibsen commented on CAMEL-23975:
-------------------------------------
This is not a camel problem as told. You need to investgate this on your own,
or buy consultancy from companies that can offer this kind of help. Its an
architecture and in this case a lifecycle between your app and hazelcast or
whatever.
Also today AIs are good at helping so try use them first.
If still a problem you can use the camel mailing list to get help if anyone is
willing to spend time helping.
> java.util.concurrent.RejectedExecutionException: CamelContext is stopped :
> at runTime (Hazelcast MapLoader Calls Route with producer Template))
> ------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-23975
> URL: https://issues.apache.org/jira/browse/CAMEL-23975
> Project: Camel
> Issue Type: Bug
> Affects Versions: 4.10.9
> Environment: Prod
> Reporter: NITESH KUMAR
> Priority: Major
>
> Hi Team,
> I am facing an issue with an Apache Camel application running on Spring Boot
> in a clustered environment and would appreciate your guidance.
> *🔧* *Setup Details*
> * *Framework:* Spring Boot(3.5x) with Apache Camel (4.10.9)
> * *Deployment:* 5 EC2 instances running the same service
> * *Caching:* Embedded Hazelcast cluster (5.5.0)
> * *Usage:* Hazelcast distributed IMap with a custom MapLoader implementation
> *⚠️* *Issue Description*
> During application startup, specifically while Hazelcast performs {*}initial
> cache population{*}, we intermittently encounter the following exception:
> java.util.concurrent.RejectedExecutionException: CamelContext is stopped
>
> But My IMaps are LAZY and executes at the time of message processing
> indicates routes & camel Context are up
> Note : This Exception We Also gets intermittently when cache reloads after
> eviction.
> *🔍* *Stack Trace (Relevant Section)*
> at
> com.hazelcast.map.impl.MapStoreWrapper.lambda$load$7(MapStoreWrapper.java:176)
> at
> com.hazelcast.internal.namespace.NamespaceUtil.callWithClassLoader(NamespaceUtil.java:200)
> at
> com.hazelcast.internal.namespace.NamespaceUtil.callWithOwnClassLoader(NamespaceUtil.java:152)
> at
> com.hazelcast.map.impl.MapStoreWrapper.load(MapStoreWrapper.java:176)
> at
> com.hazelcast.map.impl.mapstore.writethrough.WriteThroughStore.load(WriteThroughStore.java:88)
> at
> com.hazelcast.map.impl.mapstore.writethrough.WriteThroughStore.load(WriteThroughStore.java:31)
> at
> com.hazelcast.map.impl.recordstore.DefaultRecordStore.loadValueWithExpiry(DefaultRecordStore.java:494)
> at
> com.hazelcast.map.impl.operation.steps.GetOpSteps$2.runStep(GetOpSteps.java:58)
> at
> com.hazelcast.map.impl.operation.steps.GetOpSteps$2.runStep(GetOpSteps.java:48)
> at
> com.hazelcast.map.impl.operation.steps.engine.StepSupplier.runStepWithState(StepSupplier.java:203)
> at
> com.hazelcast.map.impl.operation.steps.engine.StepSupplier$1.run(StepSupplier.java:146)
> at
> com.hazelcast.internal.namespace.impl.NoOpUserCodeNamespaceService.runWithNamespace(NoOpUserCodeNamespaceService.java:82)
> at
> com.hazelcast.internal.namespace.NamespaceUtil.runWithNamespace(NamespaceUtil.java:106)
> at
> com.hazelcast.map.impl.operation.steps.engine.StepRunner.runDirect(StepRunner.java:234)
> at
> com.hazelcast.map.impl.operation.steps.engine.StepRunner.run(StepRunner.java:176)
> at
> com.hazelcast.internal.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:220)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.base/java.lang.Thread.run(Thread.java:833)
> at
> com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76)
> at
> com.hazelcast.internal.util.executor.PoolExecutorThreadFactory$ManagedThread.executeRun(PoolExecutorThreadFactory.java:74)
> at
> com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:111)
> Caused by: java.util.concurrent.RejectedExecutionException: CamelContext is
> stopped
> at
> org.apache.camel.support.cache.DefaultProducerCache.send(DefaultProducerCache.java:155)
> at
> org.apache.camel.impl.engine.DefaultProducerTemplate.send(DefaultProducerTemplate.java:175)
> at
> org.apache.camel.impl.engine.DefaultProducerTemplate.send(DefaultProducerTemplate.java:171)
> at
> org.apache.camel.impl.engine.DefaultProducerTemplate.requestBody(DefaultProducerTemplate.java:413)
> ... 25 common frames omitted
> *🔍* *Key Implementation Detail*
> We are using a custom Hazelcast MapLoader, where:
> * loadAllKeys() is triggered during first message processing as maps are
> lazy. (indirectly camel is fully UP)
> * Inside loadAllKeys(), we invoke a Camel route using ProducerTemplate
> *Example:*
> Java
> @Override
> public Iterable<K> loadAllKeys() {
> producerTemplate.requestBody("direct:loadKeysRoute", payload);
> return keys;
> }
>
> *🔍* *Observations*
> * Issue occurs primarily during *initial cache loading*
> * Sometimes observed during steady-state runtime
> * Occurs across multiple EC2 nodes concurrently
> * No explicit shutdown or stop is triggered from application code
> * CamelContext stops automatically after the exception
> * Suspected causes:
> * Interaction between Hazelcast-managed threads and Camel execution
>
> *📉* *Impact*
> * Message processing is interrupted
>
> *✅* *Request*
> Kindly help with:
> * Root cause analysis
> * Recommended configuration or architectural changes
> * Preventive measures to avoid CamelContext shutdown in such scenarios
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
