Just switched my repo to public. https://github.com/riccardomodanese/address-test
Riccardo From: Justin Bertram <[email protected]> Date: Friday, 19 September 2025 at 22:00 To: [email protected] <[email protected]> Subject: Re: How to programmatically delete addresses in Artemis I was able to reproduce the issue with your sample project so thanks for that. I added -XX:+HeapDumpOnOutOfMemoryError and analyzed the heap dumps generated once the broker stopped. I found 2 address-related data structures with a leak, and I opened ARTEMIS-5672 [1] to track the problem. I have a preliminary fix that resolves the problem in your project, but I'm continuing to test to confirm that it doesn't cause any regressions. In the future I would ask that projects like yours be public as it makes logistics much simpler and provides more benefit to the community. This is an open source source community, after all. Justin [1] https://issues.apache.org/jira/browse/ARTEMIS-5672 On Fri, Sep 19, 2025 at 5:21 AM Modanese, Riccardo <[email protected]> wrote: > There is no specific reason to keep it in a private repo. I just prefer > this way. > Could we continue in a git issue in my repo? You should have rights to > open it. > > Riccardo > > From: Justin Bertram <[email protected]> > Date: Wednesday, 17 September 2025 at 17:27 > To: [email protected] <[email protected]> > Subject: Re: How to programmatically delete addresses in Artemis > I received the invitation, and I've cloned the Git repo. > > However, it's not clear to me how you want to proceed here since the repo > is private. Do you want to continue discussing it here or somewhere else? > Continuing here will be a bit awkward since nobody else on the list will be > able to follow along or assist in any direct way. Continuing elsewhere begs > the question of where. Is there a specific reason you made the repo > private? > > > Justin > > On Wed, Sep 17, 2025 at 9:27 AM Modanese, Riccardo > <[email protected]> wrote: > > > I uploaded it to a private repo in my github account. I send you an > > invitation. > > > > Please confirm me if you received the invitation. > > > > Thank you > > > > Riccardo > > > > From: Justin Bertram <[email protected]> > > Date: Wednesday, 17 September 2025 at 16:13 > > To: [email protected] <[email protected]> > > Subject: Re: How to programmatically delete addresses in Artemis > > I recommend you upload it to a Git repo somewhere (e.g. GitHub, GitLab, > > BitBucket, Codeberg, etc.). > > > > > > Justin > > > > On Wed, Sep 17, 2025 at 2:49 AM Modanese, Riccardo > > <[email protected]> wrote: > > > > > Hi Justin, I have built a small project to reproduce the issue. > > > > > > There is a readme file, in the root directory, with the instructions to > > > build and run the test. > > > > > > How can you provide the code? (it’s composed by few maven projects) > > > > > > Riccardo > > > > > > From: Modanese, Riccardo <[email protected]> > > > Date: Monday, 15 September 2025 at 09:33 > > > To: [email protected] <[email protected]> > > > Subject: Re: How to programmatically delete addresses in Artemis > > > I’m still working to isolate the code. > > > > > > About the auto deletion, we have different subscriptions made MQTT and > > > AMQP clients (we had a custom plugin with ActiveMQ classic also). > > > > > > > > > Not clear why but, with the same address-settings configuration, if I > > > disable our plugin the addresses are not deleted. > > > > > > If I remember correctly, the addresses are removed only once the MQTT > > > devices are disconnected. We did several attempts to find the way to > get > > > them deleted but, when we got it, we had also durable subscriptions > from > > > AMQP clients deleted once AMQP clients were disconnected. > > > > > > > > > > > > About possible memory leak, I can estimate a couple of hundred Mb for 1 > > > million deleted addresses. > > > > > > > > > > > > Hope to be back with a test case. > > > > > > > > > > > > Riccardo > > > > > > From: Justin Bertram <[email protected]> > > > Date: Sunday, 14 September 2025 at 06:14 > > > To: [email protected] <[email protected]> > > > Subject: Re: How to programmatically delete addresses in Artemis > > > If there was some kind of memory leak during the process of removing an > > > address I would expect there to be lots of reports from users because > > it's > > > a pretty common operation during the life of a broker. This is why I'm > > keen > > > on getting clear details and hard facts. > > > > > > Regarding your MQTT use-case, it's not clear to me why the broker can't > > > handle it out-of-the-box. Why specifically do you need a plugin? > > > > > > There's a few things to note regarding the methods you've mentioned > here. > > > First, the clearAddressCache method on ActiveMQServer is already > invoked > > > when the address is removed via the methods we've discussed previously. > > > Second, message counters [1] are not enabled by default so > > > invoking resetAllMessageCounterHistories or resetAllMessageCounters > > > shouldn't be necessary unless you've specifically enabled message > > counters > > > in your configuration. Have you, in fact, enabled message counters in > > your > > > configuration? > > > > > > I look forward to your test-case. > > > > > > > > > Justin > > > > > > [1] > > > > > > > > > https://activemq.apache.org/components/artemis/documentation/latest/management.html#message-counters > > > > > > On Fri, Sep 12, 2025 at 10:56 AM Modanese, Riccardo > > > <[email protected]> wrote: > > > > > > > About > > > > “The answer to the question I posed in my first response (and > repeated > > in > > > > my > > > > second) still isn't clear to me. You originally said that it, > "...looks > > > > like there are addresses related objects not removed from the broker > > > > memory..." Can you elaborate on exactly what those "related objects" > > > are?” > > > > > > > > I mean, since the Heap memory grows constantly, I suppose there are > > > > objects not deallocated that I suppose are related to the address > > and/or > > > > the delete operation. But I have no real infos, just my thoughts. > > > > > > > > Just to better describe our use case, our clients are sending > messages > > to > > > > topics used only one time, so every message is published to a new > topic > > > > (like prefix/client-id/timestamp) (MQTT qos ) > > > > If we force the publish to use only one fixed topic per client (like > > > > prefix/client-id/topic), the memory doesn’t grow. > > > > > > > > I’m working to extract a use case to test. > > > > > > > > I did a lot of attempts to isolate the issue but unfortunately > without > > > any > > > > progress. > > > > I tuned various GC’s parameters, disabled the broker persistence, the > > > > address-related metrics export, forced the address cache clean and > > > message > > > > count after each address deletion iteration like > > > > server.clearAddressCache(); > > > > server.getActiveMQServerControl().resetAllMessageCounterHistories(); > > > > server.getActiveMQServerControl().resetAllMessageCounters(); > > > > > > > > but, even if the memory usage is lower and it takes more time to > reach > > > the > > > > Out Of Memory, the OOM is reached anyway. > > > > > > > > I connected a profiler, and from my understanding of the object > > > allocation > > > > flame graph, the memory allocated by the > > > > server.getActiveMQServerControl().deleteAddress(address, true); > > > > calls grows. > > > > > > > > Unfortunately, looks like there are only a lot of String[] and byte[] > > > > objects instantiated. So not too much helpful to investigate the > > possible > > > > issue. > > > > > > > > I hope to be able to isolate soon a use case to reproduce my test. > > > > > > > > Riccardo > > > > > > > > From: Justin Bertram <[email protected]> > > > > Date: Wednesday, 10 September 2025 at 19:33 > > > > To: [email protected] <[email protected]> > > > > Subject: Re: How to programmatically delete addresses in Artemis > > > > Are you running the same load in both test-cases? If so, even if the > > > > addresses weren't being deleted properly in the 2nd test I wouldn't > > > expect > > > > the broker to use _more_ memory than the first test. At the very > worst > > it > > > > should just use the _same_ amount of memory as the first test where > the > > > > addresses are not deleted, right?. Am I missing something? > > > > > > > > In any case, have you gathered heap dumps and analyzed them to see > what > > > the > > > > difference is between the two test-cases? If so, what did you find? > > > > > > > > The proper method to use is, in fact, > > > > o.a.a.a.c.s.ActiveMQServer#removeAddressInfo(SimpleString, > > SecurityAuth, > > > > boolean). > > > > > > > > The deleteAddress method on the ActiveMQServerControl is essentially > > > just a > > > > wrapper around the aforementioned removeAddressInfo method which is > > > exposed > > > > for remote management. > > > > > > > > The unregisterAddress method on the ManagementService simply removes > > the > > > > management & metrics resources corresponding to the address, but it > > > doesn't > > > > delete the address itself. > > > > > > > > The removeAddressInfo on the PostOffice does a lot of the heavy > lifting > > > > related to removing the address, but it doesn't delete the address > > > binding > > > > from storage or remove any pages stored on disk. > > > > > > > > Ultimately, removeAddressInfo on ActiveMQServer will > > > > invoke removeAddressInfo on the PostOffice which, in turn, will > invoke > > > > unregisterAddress method on the ManagementService so there's no need > to > > > > call these yourself. It's no surprise that invoking all these methods > > in > > > > various orders might generate an exception at some point. > > > > > > > > The answer to the question I posed in my first response (and repeated > > in > > > my > > > > second) still isn't clear to me. You originally said that it, > "...looks > > > > like there are addresses related objects not removed from the broker > > > > memory..." Can you elaborate on exactly what those "related objects" > > are? > > > > > > > > If you have a test-case you can share that would be helpful as well. > > > > > > > > > > > > Justin > > > > > > > > On Wed, Sep 10, 2025 at 11:33 AM Modanese, Riccardo > > > > <[email protected]> wrote: > > > > > > > > > To satisfy our specific logic we have our own address deletion > > plugin. > > > > > In any case we have the section [1] you mentioned correctly set in > > our > > > > > broker.xml. > > > > > > > > > > I try to explain myself in another way. > > > > > We saw a possible memory leak in our tests, so we created 2 test > > cases > > > to > > > > > try to reproduce this issue. > > > > > In the first one our plugin doesn’t delete addresses (logic tells > the > > > > > addresses are not to be deleted). The broker heap memory remains > more > > > or > > > > > less constant. > > > > > In the second scenario there are a lot of random addresses that are > > > > > deleted by our plugin. In this scenario the broker heap memory > grows > > > > > constantly, and the broker goes Out Of Memory. I added a task that > > > > > periodically prints the memory reported by the jvm but we also use > > the > > > > > Artemis core metrics to verify the memory growth. > > > > > > > > > > I was investigating available methods to delete addresses, and I > > found > > > 4 > > > > > different way that look to me similar. > > > > > Have no idea about which is the difference, I was looking for > > > > > documentation, but I didn’t find it. > > > > > > > > > > server.removeAddressInfo(simpleAddress, null, true); > > > > > server.getActiveMQServerControl().deleteAddress(address, true); > > > > > server.getManagementService().unregisterAddress(simpleAddress); > > > > > server.getPostOffice().removeAddressInfo(simpleAddress, true); > > > > > > > > > > > > > > > I played with these methods, and I found a call sequence order that > > > > throws > > > > > exception (something like not existing address exception). So I’m > > > wonder > > > > if > > > > > our delete is calling the wrong method. > > > > > > > > > > Riccardo > > > > > > > > > > > > > > > > > > > > From: Justin Bertram <[email protected]> > > > > > Date: Wednesday, 10 September 2025 at 16:49 > > > > > To: [email protected] <[email protected]> > > > > > Subject: Re: How to programmatically delete addresses in Artemis > > > > > If an MQTT client sends messages to an address and that address > never > > > has > > > > > any queue created on it then it will not be automatically deleted > by > > > > > default since the broker will not consider that address to be > "used." > > > > That > > > > > may be what is happening in your MQTT "case A." If so, check out > > > > > the auto-delete-addresses-skip-usage-check [1] address setting. > > > > > > > > > > I'm not sure what you mean by "no memory eating" and "memory > eating" > > in > > > > > this context. Every AddressInfo object consumes memory simply due > to > > > its > > > > > existence. > > > > > > > > > > As far as I can tell, you're deleting the address in the proper > way. > > > > > However, you said that it, "...looks like there are addresses > related > > > > > objects not removed from the broker memory..." and it's not clear > > what > > > > > those "related objects" are. Can you elaborate on this? > > > > > > > > > > > > > > > Justin > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > https://activemq.apache.org/components/artemis/documentation/latest/address-settings.html#address-settings > > > > > > > > > > On Wed, Sep 10, 2025 at 2:37 AM Modanese, Riccardo > > > > > <[email protected]> wrote: > > > > > > > > > > > Hi Justin, > > > > > > it’ just a thought from my findings. > > > > > > Same system setup. > > > > > > Case A) MQTT clients publishing on topics that create addresses > > that > > > > are > > > > > > not deleted => no memory eating on broker side > > > > > > Case B) MQTT clients publishing on different topics (every > message > > is > > > > > > published on a new different topic). These addresses are deleted > => > > > > > memory > > > > > > eating on broker side > > > > > > From my checks the address looks to be deleted correctly (Artemis > > > core > > > > > > metrics reports a constant address count). > > > > > > > > > > > > So I’m wonder if I’m doing address deletion in the right way or > I’m > > > > > > forgetting something. > > > > > > > > > > > > Thanks for your support, > > > > > > Riccardo > > > > > > > > > > > > From: Justin Bertram <[email protected]> > > > > > > Date: Tuesday, 9 September 2025 at 18:39 > > > > > > To: [email protected] <[email protected]> > > > > > > Subject: Re: How to programmatically delete addresses in Artemis > > > > > > Can you elaborate on the "related objects" here? What exactly > isn't > > > > being > > > > > > removed? > > > > > > > > > > > > > > > > > > Justin > > > > > > > > > > > > On Tue, Sep 9, 2025 at 11:17 AM Modanese, Riccardo > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > Hi guys, I have to delete programmatically addresses based on > > some > > > > > logic > > > > > > > in Artemis broker. > > > > > > > > > > > > > > I tried with > > > > > > > SimpleString address = new SimpleString(someStringAddress); > > > > > > > ... > > > > > > > ActiveMQServer server = <current_instance>; > > > > > > > ... > > > > > > > server.removeAddressInfo(simpleAddress, null, true); > > > > > > > > > > > > > > but looks like there are addresses related objects not removed > > from > > > > the > > > > > > > broker memory (there is a memory leak) > > > > > > > > > > > > > > Thanks in advance for your support. > > > > > > > > > > > > > > Regards, > > > > > > > Riccardo Modanese > > > > > > > > > > > > > > > > > > > > > > > > > > > >
