HI, Thanks for your reply.
We started to see the queue size growing in production recently. Following are some of the things we have tried so far We initially were on AMQ version 5.15.10 and JDBC driver version 4.2. We upgraded both AMQ as well as JDBC driver to the latest (5.16.3 and mssql-jdbc-9.4 respectively). We also tried KahaDB persistence instead of JDBC. We also counted the actual number of messages that we produce from our side. As I mentioned in the initial report, that in one case we measured, the actual numbers that we produced were around 60,000 whereas the queue size was 10 times more than that. We also came across this issue https://issues.apache.org/jira/browse/AMQ-7425 which is similar to what we experience. Though upgrading to the latest didn't help, our issues are limited to only those queues that we use BrowseMessages against. This is the part I am investigating right now. Regards, Bala On 2021/10/05 05:39:38, JB Onofré <j...@nanthrax.net> wrote: > Can you elaborate a bit your analyze and test case ? > > Thanks > Regards > JB > > > Le 5 oct. 2021 à 07:20, Balamurugan Seenivasan <tosb...@gmail.com> a écrit : > > > > Hi, > > > > Thanks for your reply. > > > > This issue exists for us even if we start out with a clean environment > > (with a brand new database). > > We even noticed this issue with KahaDB too. > > This is consistently reproducible for us. > > > > Regards, > > Bala > > > >> On 2021/10/01 17:56:07, Matt Pavlovich <mattr...@gmail.com> wrote: > >> Hi Balamurugan- > >> > >> Is the broker restarted at any time in this sequence of events? > >> > >> > >> Keep in mind— > >> > >> Pending Count can be higher than Enqueue Count when the broker is > >> restarted and messages remained on the queue. > >> > >> a. Enqueue and Dequeue counts reset are for that broker’s uptime. > >> > >> b. Pending Count (aka queueSize) is how many messages are currently in the > >> store. > >> > >> -Matt Pavlovich > >> > >>>> On Oct 1, 2021, at 7:29 AM, Tim Bain <tb...@alumni.duke.edu> wrote: > >>> > >>> The fact that pending messages > enqueued messages looks suspicious, so my > >>> expectation is that this will turn out to be a bug in the statistics code. > >>> But the information I requested may help us to confirm that. > >>> > >>> Tim > >>> > >>>> On Fri, Oct 1, 2021, 6:23 AM Jean-Baptiste Onofré <j...@nanthrax.net> > >>>> wrote: > >>> > >>>> Hi, > >>>> > >>>> enqueued and dequeued only increase. > >>>> > >>>> The main indicator to check is the pending. > >>>> > >>>> If pending count is increasing, it means that the queue size grow > >>>> infinitely. enqueued is the number of produced messages (not necessary > >>>> still in the queue). So, which value are you referencing ? > >>>> BrowsingMessages can "lock" the messages, blocking the consumers > >>>> (depending the way you do it). > >>>> > >>>> Regards > >>>> JB > >>>> > >>>> On 29/09/2021 09:23, Balamurugan Seenivasan wrote: > >>>>> Hi, > >>>>> > >>>>> We notice an issue recently in production where the messages in queue > >>>> size grows disproportionate to the number of messages actually produced. > >>>>> Queue size grows close to 3 million in about 3 days while the actual > >>>> messages produced are way less than the actual queue size. > >>>>> > >>>>> Here is one snapshot of how the queue looks like when actual messages > >>>> produced are only around 60000. > >>>>> 794712 (pending) 8(consumers) 679793(enqueued) 507419(dequeued) > >>>>> > >>>>> We use browseMessages extensively (from around 50 threads) using > >>>> selectors like 'JMSCorrelationID LIKE <data>%'. > >>>>> We notice this issue only with those queues we use browseMessages > >>>> against. > >>>>> Not sure if there is a link to usage of browseMessages. > >>>>> > >>>>> AMQ version: 5.16.3 (windows) > >>>>> Persistence: > >>>>> Yes (Microsoft SQL Server 2019 on Linux (Ubuntu 16.04.6 LTS) <X64>, > >>>> mssql-jdbc-9.4) > >>>>> config: > >>>>> <persistenceAdapter> > >>>>> <jdbcPersistenceAdapter > >>>> dataDirectory="activemq-data" dataSource="#mssql-ds" > >>>> lockKeepAlivePeriod="5000"> > >>>>> > >>>> <adapter><transact-jdbc-adapter/></adapter> > >>>>> <locker> > >>>>> <lease-database-locker > >>>> lockAcquireSleepInterval="10000"/> > >>>>> </locker> > >>>>> </jdbcPersistenceAdapter> > >>>>> </persistenceAdapter> > >>>>> > >>>>> Appreciate any help/pointers narrowing down the cause. > >>>>> > >>>> > >> > >> > >
