Hello,
Is there a way to configure the local broker to connect to the primary remote
broker and then only attempt to connect to the backup remote broker if the
primary is unavailable and avoid the problem I described originally?
uri="static:(failover:(ssl://primary-amq-broker:61617?verifyHostName=false,ssl://backup-amq-broker:61617?verifyHostName=false)?initialReconnectDelay=100&randomize=false)"
This is the configuration I have for this setup, and it works as I expected
except for the "duplicate" message problem that occurs when the local broker
reconnects to the primary broker, or when if fails over to the backup and then
back to the primary broker. If I remove the 'failover' as suggested:
uri="static:(ssl://primary-amq-broker?verifyHostName=false,ssl://
backup-amq-broker:61617?verifyHostName=false)?initialReconnectDelay=100"
Then the local broker will continue to log error messages attempting to connect
to the backup which is offline in normal operation. If I force a failover by
stopping the primary remote broker and then starting the backup, the connection
succeeds, and operation continues as desired.
However, I would like to avoid the errors in the log on the local broker.
Is there a configuration that I can use to accomplish the desired behavior?
Thanks,
Greg
-----Original Message-----
From: Greg Rabil
Sent: Tuesday, December 31, 2024 12:44 PM
To: users@activemq.apache.org
Subject: RE: Reconnected producers not handled correctly
Hi Matt,
I was indeed using the 'failover' URI in the networkConnector of the
activemq.xml on the local AMQ broker. As you suggested, I changed it to just
use the 'static' URI, and this appears to resolve the problem.
However, now I am curious how to configure the local AMQ broker to support
"failover" to a second remote broker in the event that the primary remote
broker is unavailable.
When there is only a single remote broker, I would configure the network
connector as follows:
uri="static:(failover:(ssl://remote-amq-broker:61617?verifyHostName=false)?initialReconnectDelay=100&randomize=false)"
As you suggested, I changed this to the following, which resolved my reconnect
issue:
uri="static:(ssl://remote-amq-broker:61617?verifyHostName=false)?initialReconnectDelay=100
For a high-availability setup, I would use the following URI:
uri="static:(failover:(ssl://primary-amq-broker:61617?verifyHostName=false,ssl://backup-amq-broker:61617?verifyHostName=false)?initialReconnectDelay=100&randomize=false)"
In the normal operation, the primary-amq-broker is running, and the
backup-amq-broker is not started. When a disaster recovery event is initiated,
the primary broker is stopped or otherwise disconnected from the network and
the backup broker is started. This would allow the local broker to "failover"
to the backup as needed. When the DR event is over, the backup broker is
stopped, and the primary is brought back online, allowing the local broker to
"fail-back" to the primary. Without the 'failover' URI, I've configured it
this way:
uri="static:(ssl://primary-amq-broker?verifyHostName=false,ssl://
backup-amq-broker:61617?verifyHostName=false)?initialReconnectDelay=100"
My understanding is that this configuration will cause a network connector to
be created for each broker in the list. Indeed, this causes errors in the
local broker's log because the connection to backup-amq-broker cannot be made
in the normal operation. It appears that it just keeps trying to connect, so
this may work for my scenario (I haven't tested it yet), but I'm wondering if
there is a better way to accomplish this goal without introducing the problem I
had originally reported.
Thanks,
Greg
-----Original Message-----
From: Matt Pavlovich <mattr...@gmail.com>
Sent: Tuesday, December 31, 2024 11:43 AM
To: users@activemq.apache.org
Subject: Re: Reconnected producers not handled correctly
Hi Greg-
A review of the activemq configuration files and logs would be needed to get a
full picture.
From your description, it sounds like you are using a ‘failover’ uri for the
brokerA to brokerB communication. That is a common misconfiguration. Network
Connectors automatically reconnect, so you should not use a failover: uri.
For a static store-and-forward configuration such as yours, you probably just
need the static:// uri in the network connector setting. I recommend correcting
that configuration first, and then seeing if the issue persists.
Thanks,
Matt
> On Dec 31, 2024, at 9:29 AM, Greg Rabil <greg.ra...@cygnalabs.com> wrote:
>
> Hello ActiveMQ users,
>
> I am using ActiveMQ 5.15.16 on a Linux platform. My setup is such that I
> have an application which produces messages for two separate queues. That
> application sends the messages to a local AMQ broker, and the local AMQ
> broker has a network connection to a remote AMQ broker using the failover
> protocol. The consumer of both queues is on the remote broker.
>
> In the initial setup, all works as expected. The producers send messages to
> the two queues “through” the local AMQ broker to the remote AMQ broker where
> the messages are consumed. However, if I restart the remote AMQ broker, then
> problems will occur. Specifically, when the producers start sending messages
> after the brokers are reconnected, the sequence IDs are no longer associated
> with the two producers, but as if there is only one producer.
>
> The problem is that the producer for queue A sends two messages, with
> sequence IDs 1 and 2. Then the producer for queue B sends a message with
> sequence ID=1, but the remote broker suppresses the delivery of this message,
> complaining that it is a duplicate because ID 1<=2 (last sequence ID from
> producer for queue A). It seems like the reconnect is not handled correctly.
> I can see this message in the debug log of the remote broker when the
> producer for queue a sends the message with sequence ID=1 after the reconnect:
>
> 2024-12-26 10:40:04,888 | DEBUG | last stored sequence id set: -1 |
> org.apache.activemq.broker.ProducerBrokerExchange | ActiveMQ Transport:
> ssl:///10.10.10.11:46160
>
> Immediately after this message, I see it send message with sequence ID=1 to
> queue A where it is consumed. Shortly thereafter, queue A’s producer sends
> message with sequence ID=2, and that is properly sent and consumed. Then
> when the producer for queue B sends a message (sequence ID=1), it is
> suppressed:
>
> 2024-12-26 10:40:50,076 | DEBUG | suppressing duplicated message send
> [ID:test-43384-1735226828739-1:1:2:1:1] with producerSequenceId [1] <= last
> stored: 2 | org.apache.activemq.broker.ProducerBrokerExchange | ActiveMQ
> Transport: ssl:///10.10.10.11:46160
>
>
> I have tried using auditNetworkProducers="true" on the transportConnector on
> the remote AMQ broker, but that did not make a difference.
>
> I’m not sure if there is some other configuration that is required, or if
> this is a bug. Can someone help me to fix this problem?
>
> Thanks,
> Greg
>
> --
> A. Gregory Rabil
>
> Sr. Software Architect DDI
> cygnalabs.com <http://cygnalabs.com/>
>
> <https://www.facebook.com/cygnalabs>
> <https://www.linkedin.com/company/cygnalabs/>
> <https://twitter.com/cygna_labs>
> This electronic message contains information from Cygna Labs Corp., which may
> be privileged or confidential. The information is intended for use only by
> the individual(s) or entity named above. If you are not the intended
> recipient, please notify the sender immediately by e-mail and be aware that
> any disclosure, copying, distribution or use of the contents of this e-mail
> is strictly prohibited. Cygna Labs monitors its e-mail system and may record
> your e-mails. Cygna Labs Corp, 1111 Lincoln Rd., Suite 760, Miami Beach, FL
> 33139
