I second what Matt wrote, especially the point that you're running 7-year-old code that's had lots of bug/security fixes since then.
I hadn't caught that this memory leak was in the client code, but the same philosophy applies: if you're leaking threads in a JVM (whether broker or client), they should be visible in a thread dump taken against that JVM, which will let you confirm what the memory analyzer is telling you. And it's worth checking how many of them there are; if your application only has one of these threads, it's hard to argue that that's a leak or that it's the root cause of whatever problem caused you to pull out the memory analyzer in the first place, whereas dozens or hundreds is a clear indication of a problem. Also, do you understand why your XA transactions are being rolled back? If not, it's probably worth investigating that as well, since transaction rollback isn't the expected steady-state behavior in a message-driven system. Tim On Mon, Nov 22, 2021, 8:26 AM Matt Pavlovich <mattr...@gmail.com> wrote: > Has the application been stable for a long period of time, or is this a > new setup? Any changes to the traffic pattern? ie.. increase in number of > connections, total throughput, etc. > > This line has me wondering if there is some unclean exception handling > leading to a leak: > > >> > weblogic.deployment.jms.WrappedXAResource_org_apache_activemq_TransactionContext.recover(I)[Ljavax/transaction/xa/Xid > >> weblogic.transaction.internal.XAResourceDescriptor.getXidsToBeRolledBack > > > If the app code, or the Weblogic platform is not handling exception errors > and this transaction recover properly, it could be leaking these resources. > Leaking resources can present as an OOM. > > Keep in mind, 5.9.1 was released April of 2014, so that is quite dated as > well. > > Hope this helps! > > -Matt Pavlovich > > > On Nov 22, 2021, at 9:01 AM, Mr. Spock <mgaspere...@gmail.com> wrote: > > > > Hi! We're using this lib: > > activemq-all-5.9.1.jar > > > > On Sun, Nov 21, 2021 at 9:54 PM Matt Pavlovich <mattr...@gmail.com> > wrote: > > > >> Hello- > >> > >> What version of AcrtiveMQ client libraries caused this? > >> > >> -Matt Pavlovich > >> > >>> On Nov 20, 2021, at 4:52 PM, Mr. Spock <mgaspere...@gmail.com> wrote: > >>> > >>> Hi all! > >>> I'm using Weblogic 11g to connect to a external ActiveMQ broker. > >>> I'm experiencing some issues on my service (OOM), and did some heapdump > >>> analyze. > >>> Using MAT leak report, it points to this thread. > >>> Did anybody saw something like this? > >>> Is a know issue? It's a perfectly normal thread? > >>> [ACTIVE] ExecuteThread: '255' for queue: 'weblogic.kernel.Default > >>> (self-tuning)' > >>> at sun.misc.Unsafe.park(ZJ)V (Native Method) > >>> at java.util.concurrent.locks.LockSupport.park(Ljava/lang/Object;)V > >>> (LockSupport.java:186) > >>> at > >>> > >> > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await()V > >>> (AbstractQueuedSynchronizer.java:2043) > >>> at java.util.concurrent.ArrayBlockingQueue.take()Ljava/lang/Object; > >>> (ArrayBlockingQueue.java:374) > >>> at > >>> > >> > org.apache.activemq.transport.FutureResponse.getResult()Lorg/apache/activemq/command/Response; > >>> (FutureResponse.java:40) > >>> at > >>> > >> > org.apache.activemq.transport.ResponseCorrelator.request(Ljava/lang/Object;)Ljava/lang/Object; > >>> (ResponseCorrelator.java:87) > >>> at > >>> > >> > org.apache.activemq.ActiveMQConnection.syncSendPacket(Lorg/apache/activemq/command/Command;)Lorg/apache/activemq/command/Response; > >>> (ActiveMQConnection.java:1397) > >>> at > >>> > >> > org.apache.activemq.TransactionContext.recover(I)[Ljavax/transaction/xa/Xid; > >>> (TransactionContext.java:651) > >>> at > >>> > >> > weblogic.deployment.jms.WrappedXAResource_org_apache_activemq_TransactionContext.recover(I)[Ljavax/transaction/xa/Xid; > >>> (Unknown Source) > >>> at > >>> > >> > weblogic.transaction.internal.XAServerResourceInfo.recover(Lweblogic/transaction/internal/CoordinatorDescriptor;)[Ljavax/transaction/xa/Xid; > >>> (XAServerResourceInfo.java:1038) > >>> at > >>> > >> > weblogic.transaction.internal.ServerSCInfo.recover(Ljava/lang/String;Lweblogic/transaction/internal/CoordinatorDescriptor;Lweblogic/transaction/internal/ResourceDescriptor;)[Ljavax/transaction/xa/Xid; > >>> (ServerSCInfo.java:660) > >>> at > >>> > >> > weblogic.transaction.internal.XAResourceDescriptor.getXidsToBeRolledBack(Lweblogic/transaction/internal/CoordinatorDescriptor;[Lweblogic/transaction/internal/ServerSCInfo;)[Ljavax/transaction/xa/Xid; > >>> (XAResourceDescriptor.java:1211) > >>> at > >>> > >> > weblogic.transaction.internal.XAResourceDescriptor.recover(Lweblogic/transaction/internal/CoordinatorDescriptor;)V > >>> (XAResourceDescriptor.java:1147) > >>> at > >>> > >> > weblogic.transaction.internal.XAResourceDescriptor.access$100(Lweblogic/transaction/internal/XAResourceDescriptor;Lweblogic/transaction/internal/CoordinatorDescriptor;)V > >>> (XAResourceDescriptor.java:60) > >>> at weblogic.transaction.internal.XAResourceDescriptor$2.run()V > >>> (XAResourceDescriptor.java:853) > >>> at weblogic.work.ExecuteThread.execute(Ljava/lang/Runnable;)V > >>> (ExecuteThread.java:263) > >>> at weblogic.work.ExecuteThread.run()V (ExecuteThread.java:221) > >>> > >>> > >>> Thanks!! > >> > >> > >
