Hi Ritesh- I have never experienced a queue purge taking 2.5 to 6.5 minutes. That is a very long time relative to modern computing standards to complete that task. I suspect the issue is your kahadb + NFS (AWS EFS) is not tuned optimally for that computing environment — rate limited IOPS on cloud-based storage.
If you do identify an ActiveMQ coding problem that could be optimized, I’d be happy to review it for adoption. Thanks, Matt Pavlovich > On Jul 1, 2024, at 5:26 PM, Ritesh Adval > <ritesh.ad...@automationanywhere.com.INVALID> wrote: > > Hi Everyone, > > Pinging again on below query to check if anyone has any suggestions on the > use case mentioned below ? > > Thx > > > -----Original Message----- > From: Ritesh Adval <ritesh.ad...@automationanywhere.com.INVALID> > Sent: Wednesday, June 5, 2024 10:38 AM > To: users@activemq.apache.org > Subject: Best practices on queue purging using activemq classic with kahadb > > [EXTERNAL] > > Hi folks, > > I have some questions regarding best practices for purging queues when using > kahadb with activemq. we are using aws efs for kahadb storage with soft mount > and these mount settings: > > type nfs4 > (rw,sync,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,soft,proto=tcp,timeo=20,retrans=2,sec=sys,clientaddr=10.12.95.205,lookupcache=none,local_lock=none,addr=10.12.116.20) > > Initially we used a single kahadb configured for all destinations in the > activemq. If there are multiple queues which gets backed up and queue size > keep increasing then we want to purge those queues but we are finding that > purging queues are very slow when done via activemq cli or jmx. When we used > jdbc storage and need to purge large queues we could just run sql statements > to delete queue data from the activemq messages table and which is fast but > that's not the case with kahadb as the cli is slow and deleting data for a > specific queue from file system is not possible unless we use multi kahadb > and kahadb per destination settings (more on this below). > > For example one queue with 10k message is taking 9 seconds to purge via cli. > But same queue with 20k messages is taking 2.5 minute and with 50k messages > its taking 6.5 minutes to purge !!! > > > So we then tried to use multi kahadb ( kahadb per destination) so that in > case we need to purge we could just wipe out a specific destination directory > to overcome slowness of purging the queues via cli. Do people use multi > kahadb for this purpose ? we are also seeing better throughput when using > multi kahadb but also facing some issue related to aws efs IOPS getting > consumed a lot which is expected as there many file read and writes due to > multi kahadb. > > Does anyone have any better ideas or suggestion in terms of how to tackle the > queue purging which is faster when using kahadb ? > > Thx > Ritesh > > > > > ** This mail has been sent from an external source. Please make sure you > trust the sender before responding, clicking links or opening attachments. ** > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@activemq.apache.org > For additional commands, e-mail: users-h...@activemq.apache.org > For further information, visit: https://activemq.apache.org/contact > > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@activemq.apache.org For additional commands, e-mail: users-h...@activemq.apache.org For further information, visit: https://activemq.apache.org/contact
