[
https://issues.apache.org/jira/browse/CAMEL-20486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen resolved CAMEL-20486.
---------------------------------
Resolution: Invalid
You need to make sure Artemis is configured for large streaming, and reach out
to this community first to ensure your broker is setup correctly.
The stacktrace below is not triggering Camel but only Spring JMS and Artemis.
Also its not good to send big payloads over JMS brokers - that is not what they
are intend for.
> JMS OutOfMemoryError in Streaming Mode
> --------------------------------------
>
> Key: CAMEL-20486
> URL: https://issues.apache.org/jira/browse/CAMEL-20486
> Project: Camel
> Issue Type: Bug
> Components: camel-jms
> Affects Versions: 3.22.0
> Environment: *Environment: Platform: MacOS, Camel 3.22.0, Spring
> Boot: v2.7.18, Java 17.0.5*
> Reporter: Michael Rambichler
> Priority: Minor
>
> We are struggling with big messages sent over jms in streaming fashion in
> production.
> *Expected:* Messages are transferred in streaming mode without the need of
> the equivalent amount of memory as the size of the file.
>
> *Currently: Java.lang.OutOfMemoryError: Java heap space*
> *Environment: Platform: MacOS, Camel 3.22.0*
> And started some analysis locally with the following outcome:
>
> We setup a simple route wich receives a message from jms.
> {code:java}
> from("jms:testjms_q?streamMessageTypeEnabled=true&artemisStreamingEnabled=true&jmsMessageType=Bytes")
> .log("Start streaming from jms")
> .to("stream:out")
> .log("Finished"); {code}
> For testing purpose we restricted the Java Heap Size to 700MB (-Xmx700m)
>
> After some analysis we see the
> {code:java}
> io.netty.util.internal.PlatformDependent.allocateUninitializedArray(int size)
> {
> return UNINITIALIZED_ARRAY_ALLOCATION_THRESHOLD < 0 ||
> UNINITIALIZED_ARRAY_ALLOCATION_THRESHOLD > size ?
> new byte[size] : PlatformDependent0.allocateUninitializedArray(size);
> }{code}
> Where size is equivalent with the message size awaiting in the JMS queue
> (around 1GB)
> Which of course lead to a OutOfMemoryError
> {code:java}
> /:: Spring Boot :: (v2.7.18)
> 2024-02-28 19:00:54.857 INFO 36343 --- [ main]
> org.example.MySpringBootApplication : Starting MySpringBootApplication
> using Java 17.0.5 on atca-052347 with PID 36343
> (/Users/at00191173/dev/src/playground/helloJms/target/classes started by
> at00191173 in /Users/at00191173/dev/src/playground/helloJms)2024-02-28
> 19:00:54.859 INFO 36343 --- [ main]
> org.example.MySpringBootApplication : No active profile set, falling
> back to 1 default profile: "default"2024-02-28 19:00:55.518 WARN 36343 --- [
> main] io.undertow.websockets.jsr : UT026010: Buffer
> pool was not set on WebSocketDeploymentInfo, the default pool will be
> used2024-02-28 19:00:55.528 INFO 36343 --- [ main]
> io.undertow.servlet : Initializing Spring embedded
> WebApplicationContext2024-02-28 19:00:55.528 INFO 36343 --- [
> main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext:
> initialization completed in 645 ms2024-02-28 19:00:56.079 DEBUG 36343 --- [
> main] i.netty.util.internal.PlatformDependent : sun.misc.Unsafe:
> available2024-02-28 19:00:56.080 DEBUG 36343 --- [ main]
> i.netty.util.internal.PlatformDependent : -Dio.netty.tmpdir:
> /var/folders/1j/85cl6ywn0tbg9tg4z_kbnhsr0000gr/T (java.io.tmpdir)2024-02-28
> 19:00:56.080 DEBUG 36343 --- [ main]
> i.netty.util.internal.PlatformDependent : -Dio.netty.bitMode: 64
> (sun.arch.data.model)2024-02-28 19:00:56.080 DEBUG 36343 --- [
> main] i.netty.util.internal.PlatformDependent : Platform: MacOS2024-02-28
> 19:00:56.080 DEBUG 36343 --- [ main]
> i.netty.util.internal.PlatformDependent : -Dio.netty.maxDirectMemory: -1
> bytes2024-02-28 19:00:56.080 DEBUG 36343 --- [ main]
> i.netty.util.internal.PlatformDependent :
> -Dio.netty.uninitializedArrayAllocationThreshold: -12024-02-28 19:00:56.081
> DEBUG 36343 --- [ main] i.netty.util.internal.PlatformDependent :
> -Dio.netty.noPreferDirect: false2024-02-28 19:00:56.181 INFO 36343 --- [
> main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 2 endpoint(s)
> beneath base path '/actuator'2024-02-28 19:00:56.227 INFO 36343 --- [
> main] io.undertow : starting server: Undertow
> - 2.2.28.Final2024-02-28 19:00:56.231 INFO 36343 --- [ main]
> org.xnio : XNIO version 3.8.7.Final2024-02-28
> 19:00:56.236 INFO 36343 --- [ main] org.xnio.nio
> : XNIO NIO Implementation Version 3.8.7.Final2024-02-28
> 19:00:56.256 INFO 36343 --- [ main] org.jboss.threads
> : JBoss Threads version 3.1.0.Final2024-02-28 19:00:56.276 INFO
> 36343 --- [ main] o.s.b.w.e.undertow.UndertowWebServer :
> Undertow started on port(s) 8080 (http)2024-02-28 19:00:56.398 DEBUG 36343
> --- [ main] o.a.camel.component.jms.JmsEndpoint : Using
> destinationName: testjms_q on listenerContainer:
> org.apache.camel.component.jms.DefaultJmsMessageListenerContainer@68454ed52024-02-28
> 19:00:56.400 INFO 36343 --- [ main]
> o.a.c.impl.engine.AbstractCamelContext : Apache Camel 3.22.0 (MyCamel) is
> starting2024-02-28 19:00:56.404 INFO 36343 --- [ main]
> o.a.c.i.e.DefaultStreamCachingStrategy : StreamCaching in use with spool
> directory: /Users/at00191173/temp and rules: [Spool > 9K body size]2024-02-28
> 19:00:56.408 TRACE 36343 --- [ main]
> o.a.camel.component.jms.JmsConsumer : Starting listener container
> org.apache.camel.component.jms.DefaultJmsMessageListenerContainer@68454ed5 on
> destination testjms_q2024-02-28 19:00:56.481 DEBUG 36343 --- [
> main] i.netty.util.internal.PlatformDependent :
> org.jctools-core.MpscChunkedArrayQueue: available2024-02-28 19:00:56.585
> DEBUG 36343 --- [ main] c.c.j.DefaultJmsMessageListenerContainer :
> Established shared JMS Connection2024-02-28 19:00:56.586 DEBUG 36343 --- [
> main] c.c.j.DefaultJmsMessageListenerContainer : Resumed paused task:
> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker@5bf523e32024-02-28
> 19:00:56.586 DEBUG 36343 --- [ main]
> o.a.camel.component.jms.JmsConsumer : Started listener container
> org.apache.camel.component.jms.DefaultJmsMessageListenerContainer@68454ed5 on
> destination testjms_q2024-02-28 19:00:56.586 TRACE 36343 ---
> [umer[testjms_q]] c.c.j.DefaultJmsMessageListenerContainer : runningAllowed()
> -> true2024-02-28 19:00:56.587 INFO 36343 --- [ main]
> o.a.c.impl.engine.AbstractCamelContext : Routes startup
> (started:1)2024-02-28 19:00:56.587 INFO 36343 --- [ main]
> o.a.c.impl.engine.AbstractCamelContext : Started route1
> (jms://testjms_q)2024-02-28 19:00:56.587 INFO 36343 --- [ main]
> o.a.c.impl.engine.AbstractCamelContext : Apache Camel 3.22.0 (MyCamel)
> started in 296ms (build:18ms init:92ms start:186ms)2024-02-28 19:00:56.592
> INFO 36343 --- [ main] org.example.MySpringBootApplication :
> Started MySpringBootApplication in 1.906 seconds (JVM running for
> 2.423)2024-02-28 19:00:57.061 INFO 36343 --- [(3)-10.84.6.176]
> io.undertow.servlet : Initializing Spring
> DispatcherServlet 'dispatcherServlet'2024-02-28 19:00:57.062 INFO 36343 ---
> [(3)-10.84.6.176] o.s.web.servlet.DispatcherServlet : Initializing
> Servlet 'dispatcherServlet'2024-02-28 19:00:57.063 INFO 36343 ---
> [(3)-10.84.6.176] o.s.web.servlet.DispatcherServlet : Completed
> initialization in 1 ms2024-02-28 19:01:01.682 WARN 36343 ---
> [umer[testjms_q]] c.c.j.DefaultJmsMessageListenerContainer : Setup of JMS
> message listener invoker failed for destination 'testjms_q' - trying to
> recover. Cause: Java heap space
> java.lang.OutOfMemoryError: Java heap space at
> io.netty.util.internal.PlatformDependent.allocateUninitializedArray(PlatformDependent.java:323)
> ~[netty-common-4.1.101.Final.jar:4.1.101.Final] at
> io.netty.buffer.UnpooledUnsafeHeapByteBuf.allocateArray(UnpooledUnsafeHeapByteBuf.java:39)
> ~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeHeapByteBuf.allocateArray(UnpooledByteBufAllocator.java:144)
> ~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
> io.netty.buffer.UnpooledHeapByteBuf.<init>(UnpooledHeapByteBuf.java:59)
> ~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
> io.netty.buffer.UnpooledUnsafeHeapByteBuf.<init>(UnpooledUnsafeHeapByteBuf.java:34)
> ~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeHeapByteBuf.<init>(UnpooledByteBufAllocator.java:139)
> ~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
> io.netty.buffer.UnpooledByteBufAllocator.newHeapBuffer(UnpooledByteBufAllocator.java:82)
> ~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
> io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:169)
> ~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
> io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:160)
> ~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
> io.netty.buffer.Unpooled.buffer(Unpooled.java:119)
> ~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
> org.apache.activemq.artemis.core.message.impl.CoreMessage.initBuffer(CoreMessage.java:150)
> ~[artemis-core-client-2.19.1.jar:2.19.1] at
> org.apache.activemq.artemis.core.client.impl.ClientLargeMessageImpl.checkBuffer(ClientLargeMessageImpl.java:153)
> ~[artemis-core-client-2.19.1.jar:2.19.1] at
> org.apache.activemq.artemis.core.client.impl.ClientLargeMessageImpl.checkCompletion(ClientLargeMessageImpl.java:82)
> ~[artemis-core-client-2.19.1.jar:2.19.1] at
> org.apache.activemq.artemis.jms.client.ActiveMQMessage.doBeforeReceive(ActiveMQMessage.java:799)
> ~[artemis-jms-client-2.19.1.jar:2.19.1] at
> org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.getMessage(ActiveMQMessageConsumer.java:230)
> ~[artemis-jms-client-2.19.1.jar:2.19.1] at
> org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:134)
> ~[artemis-jms-client-2.19.1.jar:2.19.1] at
> org.springframework.jms.support.destination.JmsDestinationAccessor.receiveFromConsumer(JmsDestinationAccessor.java:132)
> ~[spring-jms-5.3.31.jar:5.3.31] at
> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:431)
> ~[spring-jms-5.3.31.jar:5.3.31] at
> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:316)
> ~[spring-jms-5.3.31.jar:5.3.31] at
> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:270)
> ~[spring-jms-5.3.31.jar:5.3.31] at
> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1237)
> ~[spring-jms-5.3.31.jar:5.3.31] at
> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1227)
> ~[spring-jms-5.3.31.jar:5.3.31] at
> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1120)
> ~[spring-jms-5.3.31.jar:5.3.31] at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> ~[na:na] at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> ~[na:na] at java.base/java.lang.Thread.run(Thread.java:833) ~[na:na]{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
