Michael Rambichler created CAMEL-20486:
------------------------------------------
Summary: JMS OutOfMemoryError in Streaming Mode
Key: CAMEL-20486
URL: https://issues.apache.org/jira/browse/CAMEL-20486
Project: Camel
Issue Type: Bug
Components: camel-jms
Affects Versions: 3.22.0
Environment: *Environment: Platform: MacOS, Camel 3.22.0, Spring Boot:
v2.7.18, Java 17.0.5*
Reporter: Michael Rambichler
We are struggling with big messages sent over jms in streaming fashion in
production.
*Expected:* Messages are transferred in streaming mode without the need of the
equivalent amount of memory as the size of the file.
*Currently: Java.lang.OutOfMemoryError: Java heap space*
*Environment: Platform: MacOS, Camel 3.22.0*
And started some analysis locally with the following outcome:
We setup a simple route wich receives a message from jms.
{code:java}
from("jms:testjms_q?streamMessageTypeEnabled=true&artemisStreamingEnabled=true&jmsMessageType=Bytes")
.log("Start streaming from jms")
.to("stream:out")
.log("Finished"); {code}
For testing purpose we restricted the Java Heap Size to 700MB (-Xmx700m)
After some analysis we see the
{code:java}
io.netty.util.internal.PlatformDependent.allocateUninitializedArray(int size) {
return UNINITIALIZED_ARRAY_ALLOCATION_THRESHOLD < 0 ||
UNINITIALIZED_ARRAY_ALLOCATION_THRESHOLD > size ?
new byte[size] : PlatformDependent0.allocateUninitializedArray(size);
}{code}
Where size is equivalent with the message size awaiting in the JMS queue
(around 1GB)
Which of course lead to a OutOfMemoryError
{code:java}
/:: Spring Boot :: (v2.7.18)
2024-02-28 19:00:54.857 INFO 36343 --- [ main]
org.example.MySpringBootApplication : Starting MySpringBootApplication
using Java 17.0.5 on atca-052347 with PID 36343
(/Users/at00191173/dev/src/playground/helloJms/target/classes started by
at00191173 in /Users/at00191173/dev/src/playground/helloJms)2024-02-28
19:00:54.859 INFO 36343 --- [ main]
org.example.MySpringBootApplication : No active profile set, falling back
to 1 default profile: "default"2024-02-28 19:00:55.518 WARN 36343 --- [
main] io.undertow.websockets.jsr : UT026010: Buffer pool was
not set on WebSocketDeploymentInfo, the default pool will be used2024-02-28
19:00:55.528 INFO 36343 --- [ main] io.undertow.servlet
: Initializing Spring embedded WebApplicationContext2024-02-28
19:00:55.528 INFO 36343 --- [ main]
w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext:
initialization completed in 645 ms2024-02-28 19:00:56.079 DEBUG 36343 --- [
main] i.netty.util.internal.PlatformDependent : sun.misc.Unsafe:
available2024-02-28 19:00:56.080 DEBUG 36343 --- [ main]
i.netty.util.internal.PlatformDependent : -Dio.netty.tmpdir:
/var/folders/1j/85cl6ywn0tbg9tg4z_kbnhsr0000gr/T (java.io.tmpdir)2024-02-28
19:00:56.080 DEBUG 36343 --- [ main]
i.netty.util.internal.PlatformDependent : -Dio.netty.bitMode: 64
(sun.arch.data.model)2024-02-28 19:00:56.080 DEBUG 36343 --- [ main]
i.netty.util.internal.PlatformDependent : Platform: MacOS2024-02-28
19:00:56.080 DEBUG 36343 --- [ main]
i.netty.util.internal.PlatformDependent : -Dio.netty.maxDirectMemory: -1
bytes2024-02-28 19:00:56.080 DEBUG 36343 --- [ main]
i.netty.util.internal.PlatformDependent :
-Dio.netty.uninitializedArrayAllocationThreshold: -12024-02-28 19:00:56.081
DEBUG 36343 --- [ main] i.netty.util.internal.PlatformDependent :
-Dio.netty.noPreferDirect: false2024-02-28 19:00:56.181 INFO 36343 --- [
main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 2 endpoint(s)
beneath base path '/actuator'2024-02-28 19:00:56.227 INFO 36343 --- [
main] io.undertow : starting server: Undertow -
2.2.28.Final2024-02-28 19:00:56.231 INFO 36343 --- [ main] org.xnio
: XNIO version 3.8.7.Final2024-02-28
19:00:56.236 INFO 36343 --- [ main] org.xnio.nio
: XNIO NIO Implementation Version 3.8.7.Final2024-02-28 19:00:56.256
INFO 36343 --- [ main] org.jboss.threads :
JBoss Threads version 3.1.0.Final2024-02-28 19:00:56.276 INFO 36343 --- [
main] o.s.b.w.e.undertow.UndertowWebServer : Undertow started on
port(s) 8080 (http)2024-02-28 19:00:56.398 DEBUG 36343 --- [ main]
o.a.camel.component.jms.JmsEndpoint : Using destinationName: testjms_q on
listenerContainer:
org.apache.camel.component.jms.DefaultJmsMessageListenerContainer@68454ed52024-02-28
19:00:56.400 INFO 36343 --- [ main]
o.a.c.impl.engine.AbstractCamelContext : Apache Camel 3.22.0 (MyCamel) is
starting2024-02-28 19:00:56.404 INFO 36343 --- [ main]
o.a.c.i.e.DefaultStreamCachingStrategy : StreamCaching in use with spool
directory: /Users/at00191173/temp and rules: [Spool > 9K body size]2024-02-28
19:00:56.408 TRACE 36343 --- [ main]
o.a.camel.component.jms.JmsConsumer : Starting listener container
org.apache.camel.component.jms.DefaultJmsMessageListenerContainer@68454ed5 on
destination testjms_q2024-02-28 19:00:56.481 DEBUG 36343 --- [ main]
i.netty.util.internal.PlatformDependent :
org.jctools-core.MpscChunkedArrayQueue: available2024-02-28 19:00:56.585 DEBUG
36343 --- [ main] c.c.j.DefaultJmsMessageListenerContainer :
Established shared JMS Connection2024-02-28 19:00:56.586 DEBUG 36343 --- [
main] c.c.j.DefaultJmsMessageListenerContainer : Resumed paused task:
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker@5bf523e32024-02-28
19:00:56.586 DEBUG 36343 --- [ main]
o.a.camel.component.jms.JmsConsumer : Started listener container
org.apache.camel.component.jms.DefaultJmsMessageListenerContainer@68454ed5 on
destination testjms_q2024-02-28 19:00:56.586 TRACE 36343 --- [umer[testjms_q]]
c.c.j.DefaultJmsMessageListenerContainer : runningAllowed() -> true2024-02-28
19:00:56.587 INFO 36343 --- [ main]
o.a.c.impl.engine.AbstractCamelContext : Routes startup (started:1)2024-02-28
19:00:56.587 INFO 36343 --- [ main]
o.a.c.impl.engine.AbstractCamelContext : Started route1
(jms://testjms_q)2024-02-28 19:00:56.587 INFO 36343 --- [ main]
o.a.c.impl.engine.AbstractCamelContext : Apache Camel 3.22.0 (MyCamel)
started in 296ms (build:18ms init:92ms start:186ms)2024-02-28 19:00:56.592
INFO 36343 --- [ main] org.example.MySpringBootApplication :
Started MySpringBootApplication in 1.906 seconds (JVM running for
2.423)2024-02-28 19:00:57.061 INFO 36343 --- [(3)-10.84.6.176]
io.undertow.servlet : Initializing Spring
DispatcherServlet 'dispatcherServlet'2024-02-28 19:00:57.062 INFO 36343 ---
[(3)-10.84.6.176] o.s.web.servlet.DispatcherServlet : Initializing
Servlet 'dispatcherServlet'2024-02-28 19:00:57.063 INFO 36343 ---
[(3)-10.84.6.176] o.s.web.servlet.DispatcherServlet : Completed
initialization in 1 ms2024-02-28 19:01:01.682 WARN 36343 --- [umer[testjms_q]]
c.c.j.DefaultJmsMessageListenerContainer : Setup of JMS message listener
invoker failed for destination 'testjms_q' - trying to recover. Cause: Java
heap space
java.lang.OutOfMemoryError: Java heap space at
io.netty.util.internal.PlatformDependent.allocateUninitializedArray(PlatformDependent.java:323)
~[netty-common-4.1.101.Final.jar:4.1.101.Final] at
io.netty.buffer.UnpooledUnsafeHeapByteBuf.allocateArray(UnpooledUnsafeHeapByteBuf.java:39)
~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeHeapByteBuf.allocateArray(UnpooledByteBufAllocator.java:144)
~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
io.netty.buffer.UnpooledHeapByteBuf.<init>(UnpooledHeapByteBuf.java:59)
~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
io.netty.buffer.UnpooledUnsafeHeapByteBuf.<init>(UnpooledUnsafeHeapByteBuf.java:34)
~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeHeapByteBuf.<init>(UnpooledByteBufAllocator.java:139)
~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
io.netty.buffer.UnpooledByteBufAllocator.newHeapBuffer(UnpooledByteBufAllocator.java:82)
~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:169)
~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:160)
~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
io.netty.buffer.Unpooled.buffer(Unpooled.java:119)
~[netty-buffer-4.1.101.Final.jar:4.1.101.Final] at
org.apache.activemq.artemis.core.message.impl.CoreMessage.initBuffer(CoreMessage.java:150)
~[artemis-core-client-2.19.1.jar:2.19.1] at
org.apache.activemq.artemis.core.client.impl.ClientLargeMessageImpl.checkBuffer(ClientLargeMessageImpl.java:153)
~[artemis-core-client-2.19.1.jar:2.19.1] at
org.apache.activemq.artemis.core.client.impl.ClientLargeMessageImpl.checkCompletion(ClientLargeMessageImpl.java:82)
~[artemis-core-client-2.19.1.jar:2.19.1] at
org.apache.activemq.artemis.jms.client.ActiveMQMessage.doBeforeReceive(ActiveMQMessage.java:799)
~[artemis-jms-client-2.19.1.jar:2.19.1] at
org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.getMessage(ActiveMQMessageConsumer.java:230)
~[artemis-jms-client-2.19.1.jar:2.19.1] at
org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:134)
~[artemis-jms-client-2.19.1.jar:2.19.1] at
org.springframework.jms.support.destination.JmsDestinationAccessor.receiveFromConsumer(JmsDestinationAccessor.java:132)
~[spring-jms-5.3.31.jar:5.3.31] at
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:431)
~[spring-jms-5.3.31.jar:5.3.31] at
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:316)
~[spring-jms-5.3.31.jar:5.3.31] at
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:270)
~[spring-jms-5.3.31.jar:5.3.31] at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1237)
~[spring-jms-5.3.31.jar:5.3.31] at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1227)
~[spring-jms-5.3.31.jar:5.3.31] at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1120)
~[spring-jms-5.3.31.jar:5.3.31] at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
~[na:na] at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
~[na:na] at java.base/java.lang.Thread.run(Thread.java:833) ~[na:na]{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
