Hello Gorav- Unfortunately, attachments are dropped by the mailing list system, so we won’t be able to see them.
Have you had any luck in querying the application-side thread dump? I suspect there is an application-side configuration or coding bug. Thanks, Matt > On Aug 3, 2023, at 5:57 AM, Gorav Singh -Extern > <gorav.singh-ext...@deutschebahn.com> wrote: > > Hi, > > Could anyone please have any idea/suggestion about this issue ? > > Mit freundlichen Grüßen > > Gorav Singh > > externe Fachkraft von der Firma Coforge Limited > im Auftrag der DB Systel GmbH > SEZ Developer Unit > Plot No. TZ-2 & 2A, Sector Tech Zone, > Greater Noida, UP 201308 > E-Mail: gorav.singh-ext...@deutschebahn.com > > -----Original Message----- > From: Gorav Singh -Extern > Sent: Tuesday, August 1, 2023 4:15 PM > To: users@activemq.apache.org > Subject: RE: Messages got stuck in ActiveMQ 5.18.2 until restart of server > > Hi Matt/Everyone, > > I hope that you are doing great ! > We upgraded our ActiveMQ version to 5.18.2 but still this issue is coming. I > have attached the dependency tree of consumer app and as you suggested in > trailing email point 1, we have activemq-client dependency as well under > activemq-broker:jar. Should I try to exclude this jar in pom.xml ? > Actually we upgraded this consumer application from JDK11 to JDK17 and this > issue was somehow coming with JDK11 as well so might be something else but > not activemq-client issue (just a guess). > Could anyone please suggest any solution for this issue ? > Thanks a lot in Advance ! > > Mit freundlichen Grüßen > > Gorav Singh > > externe Fachkraft von der Firma Coforge Limited im Auftrag der DB Systel GmbH > SEZ Developer Unit Plot No. TZ-2 & 2A, Sector Tech Zone, Greater Noida, UP > 201308 > E-Mail: gorav.singh-ext...@deutschebahn.com > > -----Original Message----- > From: Matt Pavlovich <mattr...@gmail.com> > Sent: Friday, June 30, 2023 10:38 PM > To: users@activemq.apache.org > Subject: Re: Messages got stuck in ActiveMQ 5.18.0 until restart of server > > [You don't often get email from mattr...@gmail.com. Learn why this is > important at https://aka.ms/LearnAboutSenderIdentification ] > > Hi Gorav- > > 1. Review dependency tree of the consumer app. Make sure there are only > jakarta versions of activemq-client-jakarta and NOT the activemq-client > dependency jars. I suspect you have both jars included transitively since you > have mentioned including activemq-pool. > > 2. The activemq-client-jakarta jar in 5.18.0 literally _can't_ work, since > some packaging was missing. That is what led to the v5.18.1 release. > > 3. A thread dump from the Consumer app when the scenario occurs would be the > best starting point to solve for why the consumer code is hanging onto > messages. > > 4. Prefetch comes into play, but is I wouldn't change it until you solve for > the core issue. > > Thanks, > Matt Pavlovich > >> On Jun 30, 2023, at 10:34 AM, Gorav Singh -Extern >> <gorav.singh-ext...@deutschebahn.com> wrote: >> >> Hi Matt, >> >> Thanks for your reply ! Sure, I will use 5.18.1 version but I am not getting >> any exception like timeout for hung consumer in the consumer app code. >> Is prefetch limit also play role here ? it is default as of now. Should we >> try to reduce this limit as well ? >> Any suggestion/help is much appreciated. >> Thanks a lot in Advance ! >> >> Mit freundlichen Grüßen >> >> Gorav Singh >> >> externe Fachkraft von der Firma Coforge Limited im Auftrag der DB >> Systel GmbH SEZ Developer Unit Plot No. TZ-2 & 2A, Sector Tech Zone, >> Greater Noida, UP 201308 >> E-Mail: gorav.singh-ext...@deutschebahn.com >> >> -----Original Message----- >> From: Matt Pavlovich <mattr...@gmail.com> >> Sent: Friday, June 30, 2023 8:10 PM >> To: users@activemq.apache.org >> Subject: Re: Messages got stuck in ActiveMQ 5.18.0 until restart of >> server >> >> [You don't often get email from mattr...@gmail.com. Learn why this is >> important at https://aka.ms/LearnAboutSenderIdentification ] >> >> Hi Gorav- >> >> For sure, you'll need to use the activemq-client-jakarta 5.18.1 jar. The >> v5.18.0 activemq-client-jakarta had a minor packaging bug that caused it not >> to work at all, so I'm not sure how that client would be working in a happy >> path. Also, the PooledConnectionFactory does not have jakarta support yet >> (jakarta support is coming in v5.19.x), so there may be a couple of >> structural issues going on within the app to start. >> >> The scenario you are describing aligns most closely to an app coding bug >> where the application consumer is hung, and not acknowledging the messages. >> The broker dispatched messages to that consumer, so it is not passing them >> to another consumer. Restarting the POD causes the Consumer to go away, and >> that allows the broker to dispatch those messages to another consumer. >> >> Look for the queue metric 'InFlightCount'. If that value is about the number >> of messages not dequeuing, then most likely the issue is in the consumer app >> code. >> >> Thanks, >> Matt Pavlovich >> >>> On Jun 30, 2023, at 2:02 AM, Gorav Singh -Extern >>> <gorav.singh-ext...@deutschebahn.com> wrote: >>> >>> Hi Team, >>> >>> We are using activemq-client-jakarta 5.18.0, activemq-broker 5.18.0 with >>> spring boot 3.0 and jdk 17. We are consuming messages using @JmsListener >>> initially having 1-5 concurrency. >>> We are consuming mostly 2500-3000 messages from the queue as the part of >>> some campaigns. >>> >>> >>> Now it works fine most of the days but once or twice in a week, we found >>> that some of the messages got stuck in the queue which are not consumed but >>> messages which pushed later got consumed successfully. >>> We have to restart the pod every time in such cases to receive stuck >>> messages. >>> >>> We tried some solution like not to use PooledConnectionFactory for >>> consumers, also we changed concurrency setting to 5-5 so that consumer >>> could not be killed but nothing worked. >>> >>> Could you please help me to find out the solution or is it a known >>> ActiveMQ issue ? (please note that in JMS configuration, we are using >>> ActiveMQConnectionFactory, DefaultJmsListenerContainerFactory, with >>> JmsTemplate) >>> >>> >>> Mit freundlichen Grüßen >>> >>> Gorav Singh >>> >>> externe Fachkraft von der Firma Coforge Limited im Auftrag der DB >>> Systel GmbH SEZ Developer Unit Plot No. TZ-2 & 2A, Sector Tech Zone, >>> Greater Noida, UP 201308 >>> E-Mail: >>> gorav.singh-ext...@deutschebahn.com<mailto:gorav.singh-extern@deutsch >>> e >>> bahn.com> >>> >>> >>> ________________________________ >>> >>> Pflichtangaben >>> anzeigen<https://www.deutschebahn.com/pflichtangaben/20230607> >>> >>> Nähere Informationen zur Datenverarbeitung im DB-Konzern finden Sie >>> hier: https://www.deutschebahn.com/de/konzern/datenschutz >> >> >> ________________________________ >> >> Pflichtangaben >> anzeigen<https://www.deutschebahn.com/pflichtangaben/20230607> >> >> Nähere Informationen zur Datenverarbeitung im DB-Konzern finden Sie >> hier: https://www.deutschebahn.com/de/konzern/datenschutz > > > ________________________________ > > Pflichtangaben anzeigen<https://www.deutschebahn.com/pflichtangaben/20230717> > > Nähere Informationen zur Datenverarbeitung im DB-Konzern finden Sie hier: > https://www.deutschebahn.com/de/konzern/datenschutz
