I see your broker config is explicitly configuring a 'multicast prefix'. I didnt realise from your comments this was some explicit config. It isnt something I have tried before and so I dont know what the expected behaviour is, but the actual behaviour you describe seems like a bug to me.
On Thu, 10 Jun 2021 at 13:51, La Fleur, Sebastiaan <sebastiaan.lafl...@ns.nl.invalid> wrote: > > Good afternoon Robbie and Clebert, > > Thank you both for your ideas on this matter! Based on your replies I have > investigated if changing the ClientID between the commands has impact on the > situation (it did not) and I decided to document the issue more detailed for > easy reproduction. > > Regarding the auto-delete I do not expect that the auto-delete is involved. > If the auto-delete is involved I would expect it would work on both the ALL > and NEW commands but instead the issue I described only happens with the NEW > command. If I am missing something here please let me know because that might > be the crucial piece of info that is lacking in my understanding! > > Regarding the deletion as the subscription is overwritten: Perhaps Artemis > does indeed delete the old queue and adds the new one because it believes it > is a new durable subscription for a different topic! In the test descriptions > below I now use different ClientID's. > Now the topic subscriptions are independent and should not change from my > point of view as a user. > > At the end of this mail is a copy of the minimal broker.xml that I used. > What I will show are 2 points: 1) The ALL command does not renew/overwrite > the existing, durable subscription queue. Pending messages are received. and > 2) The NEW does renew/overwrite the existing, durable subscription queue. > Pending messages are lost. > Both situations appear to not influence each other as the order in which the > 2 tests are tried do not matter. > > Startup every test: > 0. Configure the 'someUser' user with the amq role and password 'somePassword' > 1. Start Artemis 2.17.0 with the broker.xml from the link. > 2. Change directory to the Artemis binary. > 3. Login and open the management console at > http://localhost:8161/console/artemis/artemisQueues?nid=root-org.apache.activemq.artemis-ARTEMIS > in a browser (the Queues page) > 4. Run this command in a separate terminal: ./artemis producer --user > someUser --password somePassword --url amqp://localhost:5672 --sleep 1000 > --protocol amqp --destination topic://foobar --verbose > > Cleanup every test: > 11. Close terminals > 12. Close management console. > 13. Quit Artemis. > 14. Remove persistent files e.g. docker container & volume. > > Test ALL: > 5. Run: ./artemis consumer --durable --protocol amqp --user someUser > --password somePassword --clientID artemisconsoleALL --destination > topic://foobar --url amqp://localhost:5672 --verbose > 6. Note: > - Messages are now being received in the terminal. > - In the management console click reset and note that a queue is created > with the name 'artemisconsoleALL.Consumer foobar, thread=0'. > - The ID of the queue. > - The queue is multicast. > - The queue is durable. > 7. Quit the command with CTRL+C and wait a couple of seconds. > 8. Note that the messages on the queue is increasing in the management > console. > 9. Run the command again to connect the receiver. > 10. Note that ALL pending messages are received. Note that the queue id in > the management console did NOT change. Note that the queue in the management > console now has 0 pending messages. > > Test NEW: > 5. Run: ./artemis consumer --durable --protocol amqp --user someUser > --password somePassword --clientID artemisconsoleNEW --destination > topic://multicast:foobar --url amqp://localhost:5672 --verbose > 6. Note: > - Messages are now being received in the terminal. > - In the management console click reset and note that a queue is created > with the name 'artemisconsoleNEW.Consumer multicast:foobar, thread=0'. > - The ID of the queue. > - The queue is multicast. > - The queue is durable. > 7. Quit the command with CTRL+C and wait a couple of seconds. > 8. Note that the messages on the queue is increasing in the management > console. > 9. Run the command again to connect the receiver. > 10. Note that NO pending messages are received. Note that the queue id in the > management console did change. Note that the queue in the management console > now has 0 pending messages. > > Conclusion: For 2 independent durable subscriptions adding the multicast: > prefix will remove the existing durable subscription and replace it with a > new one while not adding the multicast: prefix will only the receiver to > reuse the existing durable subscription. > > Hopefully this leads to a better understanding of the issue I am > investigating. Curious if anyone has any other insights. Or could I have > discovered a bug? Thank you in advance! > > > Regards, > > Sebastiaan la Fleur > > > > Minimal broker.xml: > <?xml version='1.0'?> > > <configuration xmlns="urn:activemq" > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xmlns:xi="http://www.w3.org/2001/XInclude"; > xsi:schemaLocation="urn:activemq > /schema/artemis-configuration.xsd"> > > <core xmlns="urn:activemq:core" > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xsi:schemaLocation="urn:activemq:core "> > > <name>ARTEMIS</name> > > <acceptors> > <acceptor > name="amqp">tcp://0.0.0.0:5672?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=AMQP;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpMinLargeMessageSize=102400;amqpDuplicateDetection=true;anycastPrefix=anycast:;multicastPrefix=multicast:;sslEnabled=false</acceptor> > </acceptors> > > <security-settings> > <security-setting match="foobar"> > <!-- * = own (sender+receiver) appname --> > <permission type="createNonDurableQueue" roles="amq"/> > <permission type="deleteNonDurableQueue" roles="amq"/> > <permission type="createDurableQueue" roles="amq"/> > <permission type="deleteDurableQueue" roles="amq"/> > <permission type="createAddress" roles="amq"/> > <permission type="deleteAddress" roles="amq"/> > <permission type="browse" roles="amq"/> > <permission type="consume" roles="amq"/> > <permission type="send" roles="amq"/> > </security-setting> > </security-settings> > > > <address-settings> > <address-setting match="foobar"> > <!-- * = sender appname --> > <max-size-bytes>100000000</max-size-bytes> > <page-size-bytes>10000000</page-size-bytes> > <address-full-policy>BLOCK</address-full-policy> > <auto-create-queues>true</auto-create-queues> > <auto-create-addresses>true</auto-create-addresses> > <auto-create-jms-queues>false</auto-create-jms-queues> > <auto-create-jms-topics>true</auto-create-jms-topics> > <min-expiry-delay>60000</min-expiry-delay> > <max-expiry-delay>604800000</max-expiry-delay> > <default-queue-routing-type>MULTICAST</default-queue-routing-type> > > <default-address-routing-type>MULTICAST</default-address-routing-type> > </address-setting> > </address-settings> > > <addresses> > > <address name="foobar"> > <multicast> > </multicast> > </address> > </addresses> > </core> > </configuration> > > > -----Original Message----- > From: Robbie Gemmell <robbie.gemm...@gmail.com> > Sent: dinsdag 8 juni 2021 16:49 > To: users@activemq.apache.org > Subject: Re: Understanding of Artemis multicastPrefix in AMQP address > prevents the transfer of historical and pending messages of queue > > I haven't tried this, and what i'm about to say may not fully explain all the > behaviour you describe, though I expected its related to the overall > situation. > > The 'durable' option will then be causing it to create a durable topic > subscriber, on your given destination. As there is no setting passing a > subscription name, it is presumably creating its own stable name in some way. > It will thus presumably be using the same subscription name when operating > for both commands, and this seems likely to be the key point. > > JMS only allows to have 1 durable subscription with a given subscription name > per ClientID. If you reuse the subscription name for a for a different > destination with the same ClientID then you effectively create a new > replacement subscription with the name at that point. This is done > server-side. > > "If an unshared durable subscription already exists with the same name and > client identifier but a different topic, message selector or noLocal value > has been specified, and there is no consumer already active (i.e. not closed) > on the durable subscription then this is equivalent to unsubscribing > (deleting) the old one and creating a new one." > https://docs.oracle.com/javaee/7/api/javax/jms/Session.html#createDurableConsumer-javax.jms.Topic-java.lang.String- > > In short, you need to use a different Client ID (leverages the AMQP > container-id to pass it), or different subscription names (leverages the AMQP > link name to pass it) to avoid two distinct unshared durable subscriptions > from interfering with each other. > > On Fri, 4 Jun 2021 at 12:22, La Fleur, Sebastiaan > <sebastiaan.lafl...@ns.nl.invalid> wrote: > > > > Good afternoon all! > > > > Artemis version: 2.17.0 > > > > This week I ran into a behavior of Artemis that I did not expect nor fully > > understand so I hope that with this e-mail someone is able to point me to > > the relevant documentation. > > > > TL;DR: > > Say we have a durable queue configured connected as multicast to the > > address (so it is a topic) and we have a sender who sends a durable message > > to the queue every 1 second with a different body. > > Then we can connect a receiver and depending on if the multicast address > > prefix is added, all pending and new messages are received or only new > > messages are received. > > > > Details: > > Queue configuration in broker.xml: > > <address-setting match="foobar"> > > <max-size-bytes>100000000</max-size-bytes> > > <page-size-bytes>10000000</page-size-bytes> > > <address-full-policy>BLOCK</address-full-policy> > > <auto-create-queues>true</auto-create-queues> > > <auto-create-addresses>false</auto-create-addresses> > > <auto-create-jms-queues>false</auto-create-jms-queues> > > <auto-create-jms-topics>false</auto-create-jms-topics> > > <min-expiry-delay>60000</min-expiry-delay> > > <max-expiry-delay>604800000</max-expiry-delay> > > > > <default-address-routing-type>MULTICAST</default-address-routing-type> > > </address-setting> > > > > <address name="foobar"> > > <multicast> > > </multicast> > > </address> > > > > I have been able to reproduce this using the Artemis CLI client: > > > > To produce $ ./artemis producer --user someUser --password somePW > > --url amqp://localhost:5672 --sleep 1000 --protocol amqp --destination > > topic://foobar --verbose > > > > To consume ALL $ ./artemis consumer --durable --protocol amqp --user > > someUser --password somePW --clientID artemisconsole --destination > > topic://foobar --url amqp://localhost:5672 --verbose > > > > To consume NEW only $ ./artemis consumer --durable --protocol amqp > > --user someUser --password somePW --clientID artemisconsole > > --destination topic://multicast:foobar --url amqp://localhost:5672 > > --verbose > > > > The producer command runs in the background and is continuously sending > > messages. Now if I run the ALL consume command, the queue is created and I > > will receive messages from that point on. If I quit the command with > > (CTRL+C), wait a couple of seconds and run the ALL command again, I will > > receive all queued, pending messages and any new messages. > > > > Now if I run the NEW consume command, another queue is created and I will > > receive messages from that point on. If I quit the command with (CTRL+C), > > wait a couple of seconds and run the NEW command again, I will NOT received > > the queued, pending messages and will ONLY receive the messages that are > > new from that point on. All pending, queued messages are removed and the > > queue gets a new ID according to queue attributes in the management web ui. > > > > The only difference between the 2 receiver commands is the addition of the > > multicast: prefix in the Artemis/AMQP address. > > > > Now if I look in the web GUI, I do see that the queue attribute ID > > stays the same whenever the ALL command (re)connects while the queue > > attribute ID changes whenever the NEW command (re)connects. An example > > of where I found this id: https://imgur.com/a/pmJPnkd > > > > I have discovered this behavior with a programmatic Python Proton client > > but the steps written in this e-mail uses the artemis binary to make it > > easily reproducible and show the behavior probably originates from the > > broker. > > > > Curious to hear if this is expected behavior! Could someone point me to the > > relevant documentation? > > > > > > Thank you in advance! > > > > Sincere regards, > > > > Sebastiaan la Fleur > > > > > > > > > > > > > > ________________________________ > > > > Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor > > (gebruik door) de geadresseerde. De e-mail kan persoonlijke of > > vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, > > verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en > > eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u > > niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene > > die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail > > (en eventuele bijlagen) te vernietigen. > > > > Informatie vennootschap<http://www.ns.nl/emaildisclaimer> > > ________________________________ > > Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor > (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke > informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of > verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan > derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde > geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond > hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) > te vernietigen. > > Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
