Thanks Justin! I will make that change and give it a try. Paul
From: Justin Bertram <jbert...@apache.org> Date: Monday, January 15, 2024 at 10:56 PM To: users@activemq.apache.org <users@activemq.apache.org> Subject: Re: Trouble with Replication HA Master/Slave config performing failback In the replication case you still need a PV in K8s, but I don't believe you need to put the whole journal on the PV (and suffer the performance penalty). You should just need to point the node-manager-lock-directory in broker.xml to the PV. Justin On Mon, Jan 15, 2024 at 4:35 PM Lino Pereira <lino.pere...@redpointglobal.com.invalid> wrote: Hi Justin, Based on your reply in the provided link, are you saying that even for the replication HA case, you still need an external FileShare PV, in K8s, so that the nodeID can be persisted between restarts. In this use case, is the performance of this FileShare less of a concern than it is for the shared-store use case? Thanks, Lino [rg]<https://www.redpointglobal.com/> Lino Pereira C++ Developer, Redpoint Global Inc. 34 Washington Street, Suite 205 Wellesley Hills, MA 02481 lino.pere...@redpointglobal.com<mailto:lino.pere...@redpointglobal.com> -----Original Message----- From: Justin Bertram <jbert...@apache.org<mailto:jbert...@apache.org>> Sent: Friday, January 12, 2024 4:53 PM To: users@activemq.apache.org<mailto:users@activemq.apache.org> Subject: Re: Trouble with Replication HA Master/Slave config performing failback *** [Caution] This email is from an external source. Please use caution responding, opening attachments or clicking embedded links. *** I sent that email on December 15. You can find it on the web-based mailing list interface [1]. Justin [1] https://lists.apache.org/thread/5bv74br0rx5nxgk816tvbq53y51vch1l<https://lists.apache.org/thread/5bv74br0rx5nxgk816tvbq53y51vch1l> On Fri, Jan 12, 2024 at 4:45 PM Shields, Paul Michael <paul.shie...@hpe.com<mailto:paul.shie...@hpe.com>> wrote: > Hi Justin, > > Forgive me but I don’t remember a journal persistence issue discussion. I > did an email search too and did not see that discussion either. Would you > please refresh me on the discussion? I may have inadvertently deleted that? > > I do see data being replicated to my production slave/standby instance on > initial startup of the two broker instances. Failover works as expected. > But when the primary instance starts backup I do not see the active slave > replicate to the starting primary and I do not see the active slave restart > and go to the backup server state. > > What is the communication sequence when the primary starts up after it > quit with the active backup broker? In other words, how is the active > backup broker notified to start replication to the newly started primary > broker? > > I ran the replicated-failback example again with TRACE level logging > turned on and I do see the attemptFailback=false message in the > target/server1/log/artemis.log file at startup, so it must be changed later > or am I not understanding the purpose of this flag? > > 2024-01-12 14:40:18,341 DEBUG > [org.apache.activemq.artemis.core.server.cluster.BackupManager] ****** > BackupManager connecting to DiscoveryBackupConnector > [group=DiscoveryGroupConfiguration{name='dg-group1', refreshTimeout=5000, > discoveryInitialWaitTimeout=10000}] > > 2024-01-12 14:40:18,341 DEBUG > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > Starting Backup Server > > 2024-01-12 14:40:18,341 INFO [org.apache.activemq.artemis.core.server] > AMQ221109: Apache ActiveMQ Artemis Backup Server version 2.28.0 [null] > started, waiting live to fail before it gets active > > 2024-01-12 14:40:18,341 TRACE > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > Setting server state as started > > 2024-01-12 14:40:18,341 TRACE > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > looking up the node through nodeLocator.locateNode() > > 2024-01-12 14:40:18,341 DEBUG > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > Connecting towards a possible live, connection > information=Pair[a=TransportConfiguration(name=netty-connector, > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > ?port=61616&host=localhost, b=null], > nodeID=ab94924c-9f4e-11ee-95ee-569984593a04 > > 2024-01-12 14:40:18,341 DEBUG > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > attemptFailback=false, nodeID=ab94924c-9f4e-11ee-95ee-569984593a04 > > 2024-01-12 14:40:18,341 TRACE > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > Calling > clusterController.connectToNodeInReplicatedCluster(TransportConfiguration(name=netty-connector, > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > ?port=61616&host=localhost) > > From my production slave broker log > > > 024-01-11 18:24:14,082 DEBUG > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > Starting Backup Server > > 2024-01-11 18:24:14,083 INFO [org.apache.activemq.artemis.core.server] > AMQ221109: Apache ActiveMQ Artemis Backup Server version 2.28.0 [null] > started, waiting live to fail before it gets active > > 2024-01-11 18:24:14,083 TRACE > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > Setting server state as started > > 2024-01-11 18:24:14,083 TRACE > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > looking up the node through nodeLocator.locateNode() > > 2024-01-11 18:24:14,083 DEBUG > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > Connecting towards a possible live, connection > information=Pair[a=TransportConfiguration(name=artemis, > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local, > b=null], nodeID=86d380d3-b0ae-11ee-9cff-7abbf21b3434 > > 2024-01-11 18:24:14,083 DEBUG > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > attemptFailback=false, nodeID=86d380d3-b0ae-11ee-9cff-7abbf21b3434 > > 2024-01-11 18:24:14,083 TRACE > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > Calling > clusterController.connectToNodeInReplicatedCluster(TransportConfiguration(name=artemis, > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local) > > Thanks, > Paul > From: Justin Bertram <jbert...@apache.org<mailto:jbert...@apache.org>> > Date: Friday, January 12, 2024 at 3:38 PM > To: users@activemq.apache.org<mailto:users@activemq.apache.org> > <users@activemq.apache.org<mailto:users@activemq.apache.org>> > Subject: Re: Trouble with Replication HA Master/Slave config performing > failback > > Is it possible to use jgroups instead of UDP for the cluster connection > in the example? > > Yes. Check out examples/features/clustered/clustered-jgroups for an example > configuration. Keep in mind that UDP, JGroups, etc. is only used for > initial discovery. Once brokers are discovered then normal TCP connections > are created. > > Before I continue looking through your logs, etc. can you confirm that you > resolved the journal persistence issue I described in my previous email? If > that's not resolved there's no chance for failback to occur. > > > Justin > > > > On Thu, Jan 11, 2024 at 3:09 PM Shields, Paul Michael < > paul.shie...@hpe.com<mailto:paul.shie...@hpe.com>> > wrote: > > > Hi Justin, > > > > I tried the example and it works. I turned up logging to get a success > > path through the code to maybe see where things get de-railed. Is it > > possible to use jgroups instead of UDP for the cluster connection in the > > example? That would give me a better success signature. > > > > My standby instance has this in the broker.xml which is the same as in > the > > example: > > > > <ha-policy> > > > > <replication> > > > > <slave> > > > > <allow-failback>true</allow-failback> > > > > </slave> > > > > </replication> > > > > </ha-policy> > > > > I am seeing this series of log messages that suggests that failback is > not > > enabled. Is the attemptFailback=false the default at this point in > startup > > and it is changed later or it there something gone wrong at this point? > > > > 2024-01-11 18:24:14,066 TRACE > > [org.apache.activemq.artemis.core.client.impl.Topology] Topology@7e0a913c > [owner=ServerLocatorImpl > > [initialConnectors=[TransportConfiguration(name=artemis, > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local], > > > discoveryGroupConfiguration=DiscoveryGroupConfiguration{name='my-discovery-group', > > refreshTimeout=10000, discoveryInitialWaitTimeout=10000}]] informing > > QuorumManager(server=null) about node up = > > 86d380d3-b0ae-11ee-9cff-7abbf21b3434 connector = > > Pair[a=TransportConfiguration(name=artemis, > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local, > > b=null] > > > > 2024-01-11 18:24:14,067 TRACE > > [org.apache.activemq.artemis.core.client.impl.Topology] Topology@7e0a913c > [owner=ServerLocatorImpl > > [initialConnectors=[TransportConfiguration(name=artemis, > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local], > > > discoveryGroupConfiguration=DiscoveryGroupConfiguration{name='my-discovery-group', > > refreshTimeout=10000, discoveryInitialWaitTimeout=10000}]] informing > > > org.apache.activemq.artemis.core.server.impl.AnyLiveNodeLocatorForReplication@43d39048 > > <mailto: > > > org.apache.activemq.artemis.core.server.impl.AnyLiveNodeLocatorForReplication@43d39048 > > > > about node up = 86d380d3-b0ae-11ee-9cff-7abbf21b3434 connector = > > Pair[a=TransportConfiguration(name=artemis, > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local, > > b=null] > > > > 2024-01-11 18:24:14,067 DEBUG > > [org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl] > > ClientSessionFactoryImpl received backup update for live/backup pair = > > TransportConfiguration(name=artemis, > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local > > / null but it didn't belong to TransportConfiguration(name=artemis, > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local > > > > 2024-01-11 18:24:14,067 TRACE > > > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > > Cluster Connected > > > > 2024-01-11 18:24:14,068 DEBUG > > > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > > Starting backup manager > > > > 2024-01-11 18:24:14,069 DEBUG > > [org.apache.activemq.artemis.core.server.cluster.BackupManager] deploy > > backup config ClusterConnectionConfiguration{name='my-cluster', > address='', > > connectorName='artemis', clientFailureCheckPeriod=30000, > > connectionTTL=60000, retryInterval=1000, retryIntervalMultiplier=2.0, > > maxRetryInterval=32000, initialConnectAttempts=20, reconnectAttempts=-1, > > callTimeout=30000, callFailoverTimeout=-1, duplicateDetection=true, > > messageLoadBalancingType=ON_DEMAND, compositeMembers=null, > > staticConnectors=[], discoveryGroupName='my-discovery-group', maxHops=1, > > confirmationWindowSize=10485760, allowDirectConnectionsOnly=false, > > minLargeMessageSize=102400, clusterNotificationInterval=1000, > > clusterNotificationAttempts=2} > > > > 2024-01-11 18:24:14,081 DEBUG > > [org.apache.activemq.artemis.core.server.cluster.BackupManager] ****** > > BackupManager connecting to DiscoveryBackupConnector > > [group=DiscoveryGroupConfiguration{name='my-discovery-group', > > refreshTimeout=10000, discoveryInitialWaitTimeout=10000}] > > > > 2024-01-11 18:24:14,082 DEBUG > > > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > > Starting Backup Server > > > > 2024-01-11 18:24:14,083 INFO [org.apache.activemq.artemis.core.server] > > AMQ221109: Apache ActiveMQ Artemis Backup Server version 2.28.0 [null] > > started, waiting live to fail before it gets active > > > > 2024-01-11 18:24:14,083 TRACE > > > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > > Setting server state as started > > > > 2024-01-11 18:24:14,083 TRACE > > > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > > looking up the node through nodeLocator.locateNode() > > > > 2024-01-11 18:24:14,083 DEBUG > > > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > > Connecting towards a possible live, connection > > information=Pair[a=TransportConfiguration(name=artemis, > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local, > > b=null], nodeID=86d380d3-b0ae-11ee-9cff-7abbf21b3434 > > > > 2024-01-11 18:24:14,083 DEBUG > > > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > > attemptFailback=false, nodeID=86d380d3-b0ae-11ee-9cff-7abbf21b3434 > > > > 2024-01-11 18:24:14,083 TRACE > > > [org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation] > > Calling > > > clusterController.connectToNodeInReplicatedCluster(TransportConfiguration(name=artemis, > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local) > > > > 2024-01-11 18:24:14,083 TRACE > > [org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl] > > getConnectionWithRetry::0 with retryInterval = 1000 multiplier = 2.0 > > > > java.lang.Exception:trace<ftp://java.lang.Exception/trace> > > > > at > > > org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fClientSessionFactoryImpl.java%3a846&c=E,1,CCRYD9asOwUO9WfztF7iYeTstIMcTzC1PzToDMj0EbrmX2UCrDAtORKi3KkbIeNCyngS9b8yN3Espjh69EqwU7zCBnH8r4y0QR9JRY_jmA,,&typo=1<https://linkprotect.cudasvc.com/url?a=https://ClientSessionFactoryImpl.java:846&c=E,1,CCRYD9asOwUO9WfztF7iYeTstIMcTzC1PzToDMj0EbrmX2UCrDAtORKi3KkbIeNCyngS9b8yN3Espjh69EqwU7zCBnH8r4y0QR9JRY_jmA,,&typo=1>) > > > > ~[artemis-core-client-2.28.0.jar:2.28.0<ftp://artemis-core-client-2.28.0.jar/2.28.0>] > > > > at > > > org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.connect(https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fClientSessionFactoryImpl.java%3a252&c=E,1,UWnIwEtYN-Os30noUm80TEjKQiOmsXl1jXnQy4rkVg_Mm8pEjR42vAnpDjuGLxaY-t6mzoLNPTHhGijPkTpytvNjie3IC5OpNppGu6mvWofV&typo=1<https://linkprotect.cudasvc.com/url?a=https://ClientSessionFactoryImpl.java:252&c=E,1,UWnIwEtYN-Os30noUm80TEjKQiOmsXl1jXnQy4rkVg_Mm8pEjR42vAnpDjuGLxaY-t6mzoLNPTHhGijPkTpytvNjie3IC5OpNppGu6mvWofV&typo=1>) > > > > ~[artemis-core-client-2.28.0.jar:2.28.0<ftp://artemis-core-client-2.28.0.jar/2.28.0>] > > > > at > > > org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fServerLocatorImpl.java%3a610&c=E,1,vriKlmcWPN4P5DVpudFT4BD6wYmRsznKQY-aJUx5dUDeuqbQ0B2VpgXUS0rVx_3I71s9Y8t9XsgeIR1ZeyEBM2jnHdASl70n9RRWcouCee6yazcDqAIo&typo=1<https://linkprotect.cudasvc.com/url?a=https://ServerLocatorImpl.java:610&c=E,1,vriKlmcWPN4P5DVpudFT4BD6wYmRsznKQY-aJUx5dUDeuqbQ0B2VpgXUS0rVx_3I71s9Y8t9XsgeIR1ZeyEBM2jnHdASl70n9RRWcouCee6yazcDqAIo&typo=1>) > > > > ~[artemis-core-client-2.28.0.jar:2.28.0<ftp://artemis-core-client-2.28.0.jar/2.28.0>] > > > > at > > > org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fServerLocatorImpl.java%3a628&c=E,1,yZA0ipuWpmPIBY8F3Y6JTV4_GbJo7sPd9oUJykEzBTWOClB24oHhFNGyNJVyU4q2vKiCq8EI6pqavdcUQ4YeYkQJJ-sdYdrA-9ozNVsiWN0,&typo=1<https://linkprotect.cudasvc.com/url?a=https://ServerLocatorImpl.java:628&c=E,1,yZA0ipuWpmPIBY8F3Y6JTV4_GbJo7sPd9oUJykEzBTWOClB24oHhFNGyNJVyU4q2vKiCq8EI6pqavdcUQ4YeYkQJJ-sdYdrA-9ozNVsiWN0,&typo=1>) > > > > ~[artemis-core-client-2.28.0.jar:2.28.0<ftp://artemis-core-client-2.28.0.jar/2.28.0>] > > > > at > > > org.apache.activemq.artemis.core.server.cluster.ClusterController.connectToNodeInReplicatedCluster(https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fClusterController.java%3a313&c=E,1,hQdcc7oc1tjMYHgfz96Agje1LexFtj3XPS7thuEqXZ0nb-a9oo4CebgIK5y6J6mpQZgBEit92vAGgbN0AxU514WfP1gx_8TZqGr4ScoNNW-rkzGKcDQPSTY,&typo=1<https://linkprotect.cudasvc.com/url?a=https://ClusterController.java:313&c=E,1,hQdcc7oc1tjMYHgfz96Agje1LexFtj3XPS7thuEqXZ0nb-a9oo4CebgIK5y6J6mpQZgBEit92vAGgbN0AxU514WfP1gx_8TZqGr4ScoNNW-rkzGKcDQPSTY,&typo=1>) > > > > ~[artemis-server-2.28.0.jar:2.28.0<ftp://artemis-server-2.28.0.jar/2.28.0>] > > > > at > > > org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation.tryConnectToNodeInReplicatedCluster(https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fSharedNothingBackupActivation.java%3a348&c=E,1,i8Ok1EuiD_R938NmaeR8WDOMKGg9vgyUWMRtpW_pJ3eYEKZHcAwICLV2rEh1XaYhPfrlVpNgyzxMr9ycx65H-0UWqEi5Wg3fgsndcGZyuIdTsNgpyvvXD76mefY,&typo=1<https://linkprotect.cudasvc.com/url?a=https://SharedNothingBackupActivation.java:348&c=E,1,i8Ok1EuiD_R938NmaeR8WDOMKGg9vgyUWMRtpW_pJ3eYEKZHcAwICLV2rEh1XaYhPfrlVpNgyzxMr9ycx65H-0UWqEi5Wg3fgsndcGZyuIdTsNgpyvvXD76mefY,&typo=1>) > > > > ~[artemis-server-2.28.0.jar:2.28.0<ftp://artemis-server-2.28.0.jar/2.28.0>] > > > > at > > > https://linkprotect.cudasvc.com/url?a=https%3a%2f%2forg.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation.run&c=E,1,tQLZyyhJ9jYhtxt5fWofp5RfJOuoNSMT9BrpaC1Kfq-XAO8pGRZaU_gpz9u2opMkMpr-C2SmX2n_Y96jt8rsp5g4EdC8qXoj8i8mjSB2EyonliwDOUod-Q,,&typo=1<https://linkprotect.cudasvc.com/url?a=https://org.apache.activemq.artemis.core.server.impl.SharedNothingBackupActivation.run&c=E,1,tQLZyyhJ9jYhtxt5fWofp5RfJOuoNSMT9BrpaC1Kfq-XAO8pGRZaU_gpz9u2opMkMpr-C2SmX2n_Y96jt8rsp5g4EdC8qXoj8i8mjSB2EyonliwDOUod-Q,,&typo=1>(https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fSharedNothingBackupActivation.java%3a199&c=E,1,3aR-ZwQ0L12WO6RcuG85ekB-kFU_DBCFf8XP3VkQwsiRzaKP0Dr7yyr1D_SIRByeDOfjSXZS-sk01qGGFSbY4 > > rdD44bwCYRLdceWEwpKomO4blh1UmHfapKb&typo=1<https://linkprotect.cudasvc.com/url?a=https://SharedNothingBackupActivation.java:199&c=E,1,3aR-ZwQ0L12WO6RcuG85ekB-kFU_DBCFf8XP3VkQwsiRzaKP0Dr7yyr1D_SIRByeDOfjSXZS-sk01qGGFSbY4rdD44bwCYRLdceWEwpKomO4blh1UmHfapKb&typo=1>) > > > > ~[artemis-server-2.28.0.jar:2.28.0<ftp://artemis-server-2.28.0.jar/2.28.0>] > > > > at > > > org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fActivationThread.run&c=E,1,E_Lo507fBrshPuy7zCANtA8BFPJ8Dqdfs6ai5o8MUPXHFZrxHGvfWmNiMihyA9Z34jHm93WN2awSavXmoqRLEsNKcLsV91wPT-p6mX28_Zg3s0Z0F3mZYkDdHQdK&typo=1<https://linkprotect.cudasvc.com/url?a=https://ActivationThread.run&c=E,1,E_Lo507fBrshPuy7zCANtA8BFPJ8Dqdfs6ai5o8MUPXHFZrxHGvfWmNiMihyA9Z34jHm93WN2awSavXmoqRLEsNKcLsV91wPT-p6mX28_Zg3s0Z0F3mZYkDdHQdK&typo=1>(https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fActiveMQServerImpl.java%3a4473&c=E,1,K6a3M2nqN4-XbzaIFsZcqt65q4f8RCxlytHn-3pKsz5nMyS3M4B8RT0M4eOPJv9HotCE27zfFlucw9ZliuGmoQXbV5t7kRaRNEJjwCwwSF-HHrjzwlECJfssqEo,&typo=1<https://linkprotect.cudasvc.com/url?a=https://ActiveMQServerImpl.java:4473&c=E,1,K6a3M2nqN4-XbzaIFsZcqt65q4f8RCxlytHn-3pKsz5nMyS3M4B8RT0M4eOPJv9HotCE27zfFlucw9ZliuGmoQXbV5t7kRaRNEJjwCwwSF-HHrjzwlECJfssqEo,&typo=1>) > > > > ~[artemis-server-2.28.0.jar:2.28.0<ftp://artemis-server-2.28.0.jar/2.28.0>] > > > > 2024-01-11 18:24:14,084 DEBUG > > [org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl] > > Trying reconnection attempt 0/0 > > > > 2024-01-11 18:24:14,084 DEBUG > > [org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl] > > Trying to connect with > > > connectorFactory=org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory@66996f18 > > > <mailto:connectorFactory<mailto:connectorFactory> > > > =org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory@66996f18 > > > > and currentConnectorConfig: TransportConfiguration(name=artemis, > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local > > > > 2024-01-11 18:24:14,084 DEBUG > > [org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector] > > Connector NettyConnector > > [host=cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local, > > port=61616, httpEnabled=false, httpUpgradeEnabled=false, > useServlet=false, > > servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true] > > using native epoll > > > > 2024-01-11 18:24:14,084 DEBUG > > [org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector] > > Started EPOLL Netty Connector version 4.1.86.Final to > > cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:61616<ftp://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local/61616> > > > > 2024-01-11 18:24:14,084 DEBUG > > [org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector] > > Remote destination: > > cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local/ > > 10.44.0.25:61616<ftp://10.44.0.25/61616> > > > > > > Thanks, > > Paul > > > > From: Justin Bertram <jbert...@apache.org<mailto:jbert...@apache.org>> > > Date: Wednesday, December 13, 2023 at 3:23 PM > > To: users@activemq.apache.org<mailto:users@activemq.apache.org> > > <users@activemq.apache.org<mailto:users@activemq.apache.org>> > > Subject: Re: Trouble with Replication HA Master/Slave config performing > > failback > > ActiveMQ Artemis ships with an example in the > > examples/features/ha/replicated-failback directory that demonstrates how > > this works. I ran this example and it worked fine. I also started up the > > brokers manually and triggered failover and failback. Again, everything > > worked fine. My only guess is that there's something configured wrong or > > perhaps there's an environmental issue that's preventing failback from > > working properly for you. If you could provide clear steps on how to > > reproduce this issue I could investigate further. > > > > > > Justin > > > > On Fri, Dec 8, 2023 at 3:45 PM Shields, Paul Michael < > paul.shie...@hpe.com<mailto:paul.shie...@hpe.com> > > > > > wrote: > > > > > Hi, > > > > > > I am having an issue with failback to the master broker. I start the > > > master broker then the slave broker. I see the replication from the > > master > > > to the slave happen. I stop the master/active broker and a failover > > happens > > > to the slave as expected. When the master is started a failback to it > is > > > expected but does not happen. The following is my ha-policy config. > > Did I > > > miss something in the ha-policy configuration? We are using version > > 2.28.0 > > > and java17. > > > > > > Thanks, > > > Paul > > > > > > > > > Master broker.xml config > > > > > > <ha-policy> > > > > > > <replication> > > > > > > <master> > > > > > > <check-for-live-server>true</check-for-live-server> > > > > > > </master> > > > > > > </replication> > > > > > > </ha-policy> > > > > > > Slave broker.xml config > > > > > > <ha-policy> > > > > > > <replication> > > > > > > <slave> > > > > > > <allow-failback>true</allow-failback> > > > > > > </slave> > > > > > > </replication> > > > > > > </ha-policy> > > > > > > The following are the logs from the master and slave brokers > > > HA “master” broker log after restart > > > 2023-12-08 20:42:57,509 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221020: Started EPOLL Acceptor at > > > cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:1883<ftp://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local/1883> > for > > > protocols [MQTT] > > > 2023-12-08 20:42:57,534 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221020: Started EPOLL Acceptor at > > > cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:61616<ftp://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local/61616> > for > > > protocols [CORE] > > > 2023-12-08 20:42:57,534 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221007: Server is now live > > > 2023-12-08 20:42:57,534 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221001: Apache ActiveMQ Artemis Message Broker version 2.28.0 > > > [amq-broker, nodeID=59d00b93-960a-11ee-b571-7ab0027aea5f] > > > 2023-12-08 20:42:57,539 INFO [org.apache.activemq.artemis] AMQ241003: > > > Starting embedded web server > > > 2023-12-08 20:42:57,625 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 15, result: NO_BINDINGS > > > 2023-12-08 20:42:57,627 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841007: > created > > > queue: > > > > > > QueueImpl[name=$.artemis.internal.sf.my-cluster.9759f092-9609-11ee-9862-e26c98ea5364, > > > postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=amq-broker], > > > temp=false]@15525ed3 > > > 2023-12-08 20:42:57,657 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: > created > > > connection: RemotingConnectionImpl [ID=47edf16f, clientID=null, > > > nodeID=59d00b93-960a-11ee-b571-7ab0027aea5f, > > > > > > transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@556a5ff2 > > > > [ID=47edf16f<mailto:transportConnection<mailto:transportConnection> > > > > > > =org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@556a5ff2 > > [ID=47edf16f>, > > > local= /10.44.0.18:61616<http://10.44.0.18:61616>, > > > remote=/10.44.0.18:34508<http://10.44.0.18:34508>]] > > > 2023-12-08 20:42:57,720 INFO > > > [org.apache.activemq.hawtio.branding.PluginContextListener] Initialized > > > activemq-branding plugin > > > 2023-12-08 20:42:57,737 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221027: Bridge ClusterConnectionBridge@1bb2e6ff > > > > > > [name=$.artemis.internal.sf.my-cluster.9759f092-9609-11ee-9862-e26c98ea5364, > > > > > > queue=QueueImpl[name=$.artemis.internal.sf.my-cluster.9759f092-9609-11ee-9862-e26c98ea5364, > > > postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=amq-broker], > > > temp=false]@15525ed3 targetConnector=ServerLocatorImpl > > > (identity=(Cluster-connection-bridge::ClusterConnectionBridge@1bb2e6ff > > > > > > [name=$.artemis.internal.sf.my-cluster.9759f092-9609-11ee-9862-e26c98ea5364, > > > > > > queue=QueueImpl[name=$.artemis.internal.sf.my-cluster.9759f092-9609-11ee-9862-e26c98ea5364, > > > postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=amq-broker], > > > temp=false]@15525ed3 targetConnector=ServerLocatorImpl > > > [initialConnectors=[TransportConfiguration(name=artemis, > > > > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > > > > ?port=61616&host=cray-dvs-mqtt-ss-1-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local], > > > discoveryGroupConfiguration=null]]::ClusterConnectionImpl@1409513883 > > [nodeUUID=59d00b93-960a-11ee-b571-7ab0027aea5f, > > > connector=TransportConfiguration(name=artemis, > > > > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local, > > > address=, server=ActiveMQServerImpl::name=amq-broker])) > > > [initialConnectors=[TransportConfiguration(name=artemis, > > > > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > > > > ?port=61616&host=cray-dvs-mqtt-ss-1-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local], > > > discoveryGroupConfiguration=null]] is connected > > > 2023-12-08 20:42:57,755 INFO > > > [org.apache.activemq.hawtio.plugin.PluginContextListener] Initialized > > > artemis-plugin plugin > > > 2023-12-08 20:42:57,766 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 18, result: NO_BINDINGS > > > 2023-12-08 20:42:57,767 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 20, result: NO_BINDINGS > > > 2023-12-08 20:42:57,968 INFO [io.hawt.HawtioContextListener] > > Initialising > > > hawtio services > > > 2023-12-08 20:42:57,970 INFO [io.hawt.system.ConfigManager] > > Configuration > > > will be discovered via system properties > > > 2023-12-08 20:42:57,971 INFO [io.hawt.jmx.JmxTreeWatcher] Welcome to > > > Hawtio 2.15.0 > > > 2023-12-08 20:42:57,975 INFO > > > [io.hawt.web.auth.AuthenticationConfiguration] Starting hawtio > > > authentication filter, JAAS realm: "activemq" authorized role(s): > "admin" > > > role principal classes: > > > "org.apache.activemq.artemis.spi.core.security.jaas.RolePrincipal" > > > 2023-12-08 20:42:57,979 INFO [io.hawt.web.auth.LoginRedirectFilter] > > > Hawtio loginRedirectFilter is using 1800 sec. HttpSession timeout > > > 2023-12-08 20:42:57,986 INFO [io.hawt.web.proxy.ProxyServlet] Proxy > > > servlet is disabled > > > 2023-12-08 20:42:57,989 INFO > > > [io.hawt.web.servlets.JolokiaConfiguredAgentServlet] Jolokia overridden > > > property: [key=policyLocation, > > > value=file:/home/jboss/amq-broker/etc/jolokia-access.xml] > > > 2023-12-08 20:42:58,058 INFO [org.apache.activemq.artemis] AMQ241001: > > > HTTP Server started at > > > > > > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161> > < > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161> > > > > < > > > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161> > < > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161> > > > > > > > > 2023-12-08 20:42:58,058 INFO [org.apache.activemq.artemis] AMQ241002: > > > Artemis Jolokia REST API available at > > > > > > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console/jolokia<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console/jolokia> > < > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console/jolokia<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console/jolokia> > > > > < > > > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console/jolokia<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console/jolokia> > < > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console/jolokia<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console/jolokia> > > > > > > > > 2023-12-08 > > > < > > > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console/jolokia2023-12-08<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console/jolokia2023-12-08> > > < > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console/jolokia2023-12-08<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console/jolokia2023-12-08> > > > > > > > > > 20:42:58,058 INFO [org.apache.activemq.artemis] AMQ241004: Artemis > > Console > > > available at > > > > > > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console> > < > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console> > > > > < > > > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console> > < > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console> > > > > > > > > 2023-12-08 > > > < > > > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console2023-12-08<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console2023-12-08> > < > http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console2023-12-08<http://cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:8161/console2023-12-08> > > > > > > > > 20:42:58,455 INFO > [org.apache.activemq.artemis.core.server.plugin.impl] > > > AMQ841000: created connection: RemotingConnectionImpl [ID=f85ec51b, > > > clientID=null, nodeID=59d00b93-960a-11ee-b571-7ab0027aea5f, > > > > > > transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@1cb2285d > > > > [ID=f85ec51b<mailto:transportConnection<mailto:transportConnection> > > > > > > =org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@1cb2285d > > [ID=f85ec51b>, > > > local= /10.44.0.18:61616<http://10.44.0.18:61616>, > > > remote=/127.0.0.6:49865<http://127.0.0.6:49865>]] > > > 2023-12-08 20:42:59,524 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: > created > > > connection: RemotingConnectionImpl [ID=7cef23a2, clientID=null, > > > nodeID=59d00b93-960a-11ee-b571-7ab0027aea5f, > > > > > > transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@5f45858 > > > > [ID=7cef23a2<mailto:transportConnection<mailto:transportConnection> > > > > > > =org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@5f45858 > > [ID=7cef23a2>, > > > local= /10.44.0.18:61616<http://10.44.0.18:61616>, > > > remote=/127.0.0.6:59691<http://127.0.0.6:59691>]] > > > 2023-12-08 20:42:59,535 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 22, result: NO_BINDINGS > > > 2023-12-08 20:42:59,536 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: > created > > > session name: 5fbf845a-960a-11ee-8014-12a7ed0622c7, session > connectionID: > > > 7cef23a2 > > > 2023-12-08 20:42:59,549 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 24, result: NO_BINDINGS > > > 2023-12-08 20:42:59,550 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: > created > > > session name: 5fc2919b-960a-11ee-8014-12a7ed0622c7, session > connectionID: > > > 7cef23a2 > > > 2023-12-08 20:42:59,583 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 26, result: OK > > > 2023-12-08 20:42:59,584 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841007: > created > > > queue: > > > > > > QueueImpl[name=notif.5fc32ddc-960a-11ee-8014-12a7ed0622c7.ActiveMQServerImpl_name=amq-broker, > > > postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=amq-broker], > > > temp=true]@5f46053b > > > 2023-12-08 20:42:59,593 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841005: > created > > > consumer with ID: 0, with session name: > > 5fc2919b-960a-11ee-8014-12a7ed0622c7 > > > 2023-12-08 20:42:59,594 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 28, result: OK > > > 2023-12-08 20:42:59,616 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841009: sent > > > message with ID: 29, result: OK, transaction: UNAVAILABLE > > > 2023-12-08 20:42:59,615 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841014: > > > acknowledged message: > > > > > > Reference[31]:NON-RELIABLE:CoreMessage[messageID=31,durable=false,userID=null,priority=0, > > > timestamp=0,expiration=0, durable=false, > > > > > > address=notif.5fc32ddc-960a-11ee-8014-12a7ed0622c7.ActiveMQServerImpl_name=amq-broker,size=248,properties=TypedProperties[AMQRESET_QUEUE_DATA=true]]@967407085, > > > > with transaction: null > > > 2023-12-08 20:42:59,621 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841012: > > delivered > > > message with message ID: 31, to consumer on address: > > > activemq.notifications, queue: > > > > > > notif.5fc32ddc-960a-11ee-8014-12a7ed0622c7.ActiveMQServerImpl_name=amq-broker, > > > consumer sessionID: 5fc2919b-960a-11ee-8014-12a7ed0622c7, consumerID: 0 > > > 2023-12-08 20:42:59,621 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841014: > > > acknowledged message: > > > > > > Reference[32]:NON-RELIABLE:CoreMessage[messageID=32,durable=false,userID=null,priority=0, > > > timestamp=Fri Dec 08 20:42:59 UTC 2023,expiration=0, durable=false, > > > > > > address=notif.5fc32ddc-960a-11ee-8014-12a7ed0622c7.ActiveMQServerImpl_name=amq-broker,size=684,properties=TypedProperties[AMQRoutingName=ExpiryQueue,AMQDistance=0,AMQAddress=ExpiryQueue,AMQNotifType=BINDING_ADDED,AMQBinding_ID=7,AMQFilterString=NULL-value,AMQNotifTimestamp=1702068179613,AMQClusterName=ExpiryQueue59d00b93-960a-11ee-b571-7ab0027aea5f]]@1373537114, > > > > with transaction: null > > > 2023-12-08 20:42:59,622 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841012: > > delivered > > > message with message ID: 32, to consumer on address: > > > activemq.notifications, queue: > > > > > > notif.5fc32ddc-960a-11ee-8014-12a7ed0622c7.ActiveMQServerImpl_name=amq-broker, > > > consumer sessionID: 5fc2919b-960a-11ee-8014-12a7ed0622c7, consumerID: 0 > > > 2023-12-08 20:42:59,622 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841014: > > > acknowledged message: > > > > > > Reference[35]:NON-RELIABLE:CoreMessage[messageID=35,durable=false,userID=null,priority=0, > > > timestamp=Fri Dec 08 20:42:59 UTC 2023,expiration=0, durable=false, > > > > > > address=notif.5fc32ddc-960a-11ee-8014-12a7ed0622c7.ActiveMQServerImpl_name=amq-broker,size=636,properties=TypedProperties[AMQRoutingName=DLQ,AMQDistance=0,AMQAddress=DLQ,AMQNotifType=BINDING_ADDED,AMQBinding_ID=3,AMQFilterString=NULL-value,AMQNotifTimestamp=1702068179614,AMQClusterName=DLQ59d00b93-960a-11ee-b571-7ab0027aea5f]]@958347658, > > > > with transaction: null > > > 2023-12-08 20:42:59,622 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841012: > > delivered > > > message with message ID: 35, to consumer on address: > > > activemq.notifications, queue: > > > > > > notif.5fc32ddc-960a-11ee-8014-12a7ed0622c7.ActiveMQServerImpl_name=amq-broker, > > > consumer sessionID: 5fc2919b-960a-11ee-8014-12a7ed0622c7, consumerID: 0 > > > 2023-12-08 20:42:59,622 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841014: > > > acknowledged message: > > > > > > Reference[39]:NON-RELIABLE:CoreMessage[messageID=39,durable=false,userID=null,priority=0, > > > timestamp=0,expiration=0, durable=false, > > > > > > address=notif.5fc32ddc-960a-11ee-8014-12a7ed0622c7.ActiveMQServerImpl_name=amq-broker,size=266,properties=TypedProperties[AMQRESET_QUEUE_DATA_COMPLETE=true]]@333704293, > > > > with transaction: null > > > 2023-12-08 20:42:59,622 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841012: > > delivered > > > message with message ID: 39, to consumer on address: > > > activemq.notifications, queue: > > > > > > notif.5fc32ddc-960a-11ee-8014-12a7ed0622c7.ActiveMQServerImpl_name=amq-broker, > > > consumer sessionID: 5fc2919b-960a-11ee-8014-12a7ed0622c7, consumerID: 0 > > > 2023-12-08 20:43:08,447 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: > created > > > connection: RemotingConnectionImpl [ID=1d1d79ce, clientID=null, > > > nodeID=59d00b93-960a-11ee-b571-7ab0027aea5f, > > > > > > transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@51aa6256 > > > > [ID=1d1d79ce<mailto:transportConnection<mailto:transportConnection> > > > > > > =org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@51aa6256 > > [ID=1d1d79ce>, > > > local= /10.44.0.18:61616<http://10.44.0.18:61616>, > > > remote=/127.0.0.6:60787<http://127.0.0.6:60787>]] > > > 2023-12-08 20:43:08,464 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: > created > > > connection: RemotingConnectionImpl [ID=8f88c6d5, clientID=null, > > > nodeID=59d00b93-960a-11ee-b571-7ab0027aea5f, > > > > > > transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@14a6e5e7 > > > > [ID=8f88c6d5<mailto:transportConnection<mailto:transportConnection> > > > > > > =org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@14a6e5e7 > > [ID=8f88c6d5>, > > > local= /10.44.0.18:61616<http://10.44.0.18:61616>, > > > remote=/127.0.0.6:40065<http://127.0.0.6:40065>]] > > > > > > > > > HA “slave” broker log. > > > > > > 2023-12-08 20:37:58,374 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221109: Apache ActiveMQ Artemis Backup Server version 2.28.0 [null] > > > started, waiting live to fail before it gets active > > > > > > 2023-12-08 20:37:59,020 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221024: Backup server ActiveMQServerImpl::name=amq-broker is > > > synchronized with live server, > > nodeID=9759f092-9609-11ee-9862-e26c98ea5364. > > > > > > 2023-12-08 20:38:00,032 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221031: backup announced > > > 2023-12-08 20:42:27,415 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221066: Initiating quorum vote: LiveFailoverQuorumVote > > > 2023-12-08 20:42:27,416 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221084: Requested 0 quorum votes > > > 2023-12-08 20:42:27,416 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221083: ignoring quorum vote as max cluster size is 1. > > > 2023-12-08 20:42:27,416 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221071: Failing over based on quorum vote results. > > > 2023-12-08 20:42:27,431 WARN [org.apache.activemq.artemis.core.client] > > > AMQ212037: Connection failure to > > > cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local/ > > > 10.44.0.20:61616<ftp://10.44.0.20/61616> has been detected: AMQ219015: > > > The connection was > > > disconnected because of server shutdown [code=DISCONNECTED] > > > 2023-12-08 20:42:27,431 WARN [org.apache.activemq.artemis.core.client] > > > AMQ212037: Connection failure to > > > cray-dvs-mqtt-ss-0.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local/ > > > 10.44.0.20:61616<ftp://10.44.0.20/61616> has been detected: AMQ219015: > > > The connection was > > > disconnected because of server shutdown [code=DISCONNECTED] > > > 2023-12-08 20:42:27,434 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221037: ActiveMQServerImpl::name=amq-broker to become 'live' > > > 2023-12-08 20:42:27,574 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221080: Deploying address DLQ supporting [ANYCAST] > > > 2023-12-08 20:42:27,575 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221003: Deploying ANYCAST queue DLQ on address DLQ > > > 2023-12-08 20:42:27,578 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221080: Deploying address ExpiryQueue supporting [ANYCAST] > > > 2023-12-08 20:42:27,578 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221003: Deploying ANYCAST queue ExpiryQueue on address ExpiryQueue > > > 2023-12-08 20:42:27,611 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221007: Server is now live > > > 2023-12-08 20:42:27,699 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221020: Started EPOLL Acceptor at > > > cray-dvs-mqtt-ss-1.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:1883<ftp://cray-dvs-mqtt-ss-1.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local/1883> > for > > > protocols [MQTT] > > > 2023-12-08 20:42:27,717 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221020: Started EPOLL Acceptor at > > > cray-dvs-mqtt-ss-1.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local:61616<ftp://cray-dvs-mqtt-ss-1.cray-dvs-mqtt-hdls-svc.dvs.svc.cluster.local/61616> > for > > > protocols [CORE] > > > 2023-12-08 20:42:51,946 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: > created > > > connection: RemotingConnectionImpl [ID=df357d0e, clientID=null, > > > nodeID=9759f092-9609-11ee-9862-e26c98ea5364, > > > > > > transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@4cbed077 > > > > [ID=df357d0e<mailto:transportConnection<mailto:transportConnection> > > > > > > =org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@4cbed077 > > [ID=df357d0e>, > > > local= /10.40.0.106:61616<http://10.40.0.106:61616>, > > > remote=/127.0.0.6:56167<http://127.0.0.6:56167>]] > > > 2023-12-08 20:42:56,977 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841001: > > destroyed > > > connection: RemotingConnectionImpl [ID=df357d0e, clientID=null, > > > nodeID=9759f092-9609-11ee-9862-e26c98ea5364, > > > > > > transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@4cbed077 > > > > [ID=df357d0e<mailto:transportConnection<mailto:transportConnection> > > > > > > =org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@4cbed077 > > [ID=df357d0e>, > > > local= /10.40.0.106:61616<http://10.40.0.106:61616>, > > > remote=/127.0.0.6:56167<http://127.0.0.6:56167>]] > > > 2023-12-08 20:42:57,443 WARN [org.apache.activemq.artemis.core.client] > > > AMQ212004: Failed to connect to server. > > > 2023-12-08 20:42:57,445 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 26, result: NO_BINDINGS > > > 2023-12-08 20:42:57,447 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841007: > created > > > queue: > > > > > > QueueImpl[name=$.artemis.internal.sf.my-cluster.59d00b93-960a-11ee-b571-7ab0027aea5f, > > > postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=amq-broker], > > > temp=false]@737e64a6 > > > 2023-12-08 20:42:57,512 WARN [org.apache.activemq.artemis.core.server] > > > AMQ222186: unable to authorise cluster control: AMQ219016: Connection > > > failure detected. Unblocking a blocking call that will never get a > > response > > > 2023-12-08 20:42:57,513 WARN [org.apache.activemq.artemis.core.server] > > > AMQ224091: Bridge ClusterConnectionBridge@2a063f52 > > > > > > [name=$.artemis.internal.sf.my-cluster.59d00b93-960a-11ee-b571-7ab0027aea5f, > > > > > > queue=QueueImpl[name=$.artemis.internal.sf.my-cluster.59d00b93-960a-11ee-b571-7ab0027aea5f, > > > postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=amq-broker], > > > temp=false]@737e64a6 targetConnector=ServerLocatorImpl > > > (identity=(Cluster-connection-bridge::ClusterConnectionBridge@2a063f52 > > > > > > [name=$.artemis.internal.sf.my-cluster.59d00b93-960a-11ee-b571-7ab0027aea5f, > > > > > > queue=QueueImpl[name=$.artemis.internal.sf.my-cluster.59d00b93-960a-11ee-b571-7ab0027aea5f, > > > postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=amq-broker], > > > temp=false]@737e64a6 targetConnector=ServerLocatorImpl > > > [initialConnectors=[TransportConfiguration(name=artemis, > > > > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local], > > > discoveryGroupConfiguration=null]]::ClusterConnectionImpl@139110008 > > [nodeUUID=9759f092-9609-11ee-9862-e26c98ea5364, > > > connector=TransportConfiguration(name=artemis, > > > > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > > > > ?port=61616&host=cray-dvs-mqtt-ss-1-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local, > > > address=, server=ActiveMQServerImpl::name=amq-broker])) > > > [initialConnectors=[TransportConfiguration(name=artemis, > > > > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local], > > > discoveryGroupConfiguration=null]] is unable to connect to destination. > > > Retrying > > > 2023-12-08 20:42:57,629 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: > created > > > connection: RemotingConnectionImpl [ID=483170f6, clientID=null, > > > nodeID=9759f092-9609-11ee-9862-e26c98ea5364, > > > > > > transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@3df0e251 > > > > [ID=483170f6<mailto:transportConnection<mailto:transportConnection> > > > > > > =org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@3df0e251 > > [ID=483170f6>, > > > local= /10.40.0.106:61616<http://10.40.0.106:61616>, > > > remote=/127.0.0.6:55911<http://127.0.0.6:55911>]] > > > 2023-12-08 20:42:57,630 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: > created > > > connection: RemotingConnectionImpl [ID=4514a2b3, clientID=null, > > > nodeID=9759f092-9609-11ee-9862-e26c98ea5364, > > > > > > transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@f9b983b > > > > [ID=4514a2b3<mailto:transportConnection<mailto:transportConnection> > > > > > > =org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@f9b983b > > [ID=4514a2b3>, > > > local= /10.40.0.106:61616<http://10.40.0.106:61616>, > > > remote=/127.0.0.6:60199<http://127.0.0.6:60199>]] > > > 2023-12-08 20:42:57,648 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: > created > > > connection: RemotingConnectionImpl [ID=fcde4d78, clientID=null, > > > nodeID=9759f092-9609-11ee-9862-e26c98ea5364, > > > > > > transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@16ef79c8 > > > > [ID=fcde4d78<mailto:transportConnection<mailto:transportConnection> > > > > > > =org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@16ef79c8 > > [ID=fcde4d78>, > > > local= /10.40.0.106:61616<http://10.40.0.106:61616>, > > > remote=/127.0.0.6:53221<http://127.0.0.6:53221>]] > > > 2023-12-08 20:42:57,663 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 29, result: NO_BINDINGS > > > 2023-12-08 20:42:57,663 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: > created > > > session name: 5ea190dc-960a-11ee-b571-7ab0027aea5f, session > connectionID: > > > fcde4d78 > > > 2023-12-08 20:42:57,679 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 31, result: NO_BINDINGS > > > 2023-12-08 20:42:57,679 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: > created > > > session name: 5ea4ec3d-960a-11ee-b571-7ab0027aea5f, session > connectionID: > > > fcde4d78 > > > 2023-12-08 20:42:57,708 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 33, result: OK > > > 2023-12-08 20:42:57,709 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841007: > created > > > queue: > > > > > > QueueImpl[name=notif.5ea5af8e-960a-11ee-b571-7ab0027aea5f.ActiveMQServerImpl_name=amq-broker, > > > postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=amq-broker], > > > temp=true]@3645037f > > > 2023-12-08 20:42:57,719 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841005: > created > > > consumer with ID: 0, with session name: > > 5ea4ec3d-960a-11ee-b571-7ab0027aea5f > > > 2023-12-08 20:42:57,720 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 35, result: OK > > > 2023-12-08 20:42:57,755 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841009: sent > > > message with ID: 36, result: OK, transaction: UNAVAILABLE > > > 2023-12-08 20:42:57,753 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841014: > > > acknowledged message: > > > > > > Reference[38]:NON-RELIABLE:CoreMessage[messageID=38,durable=false,userID=null,priority=0, > > > timestamp=0,expiration=0, durable=false, > > > > > > address=notif.5ea5af8e-960a-11ee-b571-7ab0027aea5f.ActiveMQServerImpl_name=amq-broker,size=248,properties=TypedProperties[AMQRESET_QUEUE_DATA=true]]@2125985563, > > > > with transaction: null > > > 2023-12-08 20:42:57,760 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841012: > > delivered > > > message with message ID: 38, to consumer on address: > > > activemq.notifications, queue: > > > > > > notif.5ea5af8e-960a-11ee-b571-7ab0027aea5f.ActiveMQServerImpl_name=amq-broker, > > > consumer sessionID: 5ea4ec3d-960a-11ee-b571-7ab0027aea5f, consumerID: 0 > > > 2023-12-08 20:42:57,761 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841014: > > > acknowledged message: > > > > > > Reference[41]:NON-RELIABLE:CoreMessage[messageID=41,durable=false,userID=null,priority=0, > > > timestamp=Fri Dec 08 20:42:57 UTC 2023,expiration=0, durable=false, > > > > > > address=notif.5ea5af8e-960a-11ee-b571-7ab0027aea5f.ActiveMQServerImpl_name=amq-broker,size=636,properties=TypedProperties[AMQRoutingName=DLQ,AMQDistance=0,AMQAddress=DLQ,AMQNotifType=BINDING_ADDED,AMQBinding_ID=3,AMQFilterString=NULL-value,AMQNotifTimestamp=1702068177753,AMQClusterName=DLQ9759f092-9609-11ee-9862-e26c98ea5364]]@243253411, > > > > with transaction: null > > > 2023-12-08 20:42:57,761 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841012: > > delivered > > > message with message ID: 41, to consumer on address: > > > activemq.notifications, queue: > > > > > > notif.5ea5af8e-960a-11ee-b571-7ab0027aea5f.ActiveMQServerImpl_name=amq-broker, > > > consumer sessionID: 5ea4ec3d-960a-11ee-b571-7ab0027aea5f, consumerID: 0 > > > 2023-12-08 20:42:57,762 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841014: > > > acknowledged message: > > > > > > Reference[43]:NON-RELIABLE:CoreMessage[messageID=43,durable=false,userID=null,priority=0, > > > timestamp=Fri Dec 08 20:42:57 UTC 2023,expiration=0, durable=false, > > > > > > address=notif.5ea5af8e-960a-11ee-b571-7ab0027aea5f.ActiveMQServerImpl_name=amq-broker,size=684,properties=TypedProperties[AMQRoutingName=ExpiryQueue,AMQDistance=0,AMQAddress=ExpiryQueue,AMQNotifType=BINDING_ADDED,AMQBinding_ID=7,AMQFilterString=NULL-value,AMQNotifTimestamp=1702068177753,AMQClusterName=ExpiryQueue9759f092-9609-11ee-9862-e26c98ea5364]]@1544502860, > > > > with transaction: null > > > 2023-12-08 20:42:57,762 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841012: > > delivered > > > message with message ID: 43, to consumer on address: > > > activemq.notifications, queue: > > > > > > notif.5ea5af8e-960a-11ee-b571-7ab0027aea5f.ActiveMQServerImpl_name=amq-broker, > > > consumer sessionID: 5ea4ec3d-960a-11ee-b571-7ab0027aea5f, consumerID: 0 > > > 2023-12-08 20:42:57,762 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841014: > > > acknowledged message: > > > > > > Reference[44]:NON-RELIABLE:CoreMessage[messageID=44,durable=false,userID=null,priority=0, > > > timestamp=0,expiration=0, durable=false, > > > > > > address=notif.5ea5af8e-960a-11ee-b571-7ab0027aea5f.ActiveMQServerImpl_name=amq-broker,size=266,properties=TypedProperties[AMQRESET_QUEUE_DATA_COMPLETE=true]]@1828250555, > > > > with transaction: null > > > 2023-12-08 20:42:57,762 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841012: > > delivered > > > message with message ID: 44, to consumer on address: > > > activemq.notifications, queue: > > > > > > notif.5ea5af8e-960a-11ee-b571-7ab0027aea5f.ActiveMQServerImpl_name=amq-broker, > > > consumer sessionID: 5ea4ec3d-960a-11ee-b571-7ab0027aea5f, consumerID: 0 > > > 2023-12-08 20:42:59,607 INFO [org.apache.activemq.artemis.core.server] > > > AMQ221027: Bridge ClusterConnectionBridge@2a063f52 > > > > > > [name=$.artemis.internal.sf.my-cluster.59d00b93-960a-11ee-b571-7ab0027aea5f, > > > > > > queue=QueueImpl[name=$.artemis.internal.sf.my-cluster.59d00b93-960a-11ee-b571-7ab0027aea5f, > > > postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=amq-broker], > > > temp=false]@737e64a6 targetConnector=ServerLocatorImpl > > > (identity=(Cluster-connection-bridge::ClusterConnectionBridge@2a063f52 > > > > > > [name=$.artemis.internal.sf.my-cluster.59d00b93-960a-11ee-b571-7ab0027aea5f, > > > > > > queue=QueueImpl[name=$.artemis.internal.sf.my-cluster.59d00b93-960a-11ee-b571-7ab0027aea5f, > > > postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=amq-broker], > > > temp=false]@737e64a6 targetConnector=ServerLocatorImpl > > > [initialConnectors=[TransportConfiguration(name=artemis, > > > > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local], > > > discoveryGroupConfiguration=null]]::ClusterConnectionImpl@139110008 > > [nodeUUID=9759f092-9609-11ee-9862-e26c98ea5364, > > > connector=TransportConfiguration(name=artemis, > > > > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > > > > ?port=61616&host=cray-dvs-mqtt-ss-1-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local, > > > address=, server=ActiveMQServerImpl::name=amq-broker])) > > > [initialConnectors=[TransportConfiguration(name=artemis, > > > > > > factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) > > > > > > ?port=61616&host=cray-dvs-mqtt-ss-0-cray-dvs-mqtt-hdls-svc-dvs-svc-cluster-local], > > > discoveryGroupConfiguration=null]] is connected > > > 2023-12-08 20:42:59,625 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 46, result: OK > > > 2023-12-08 20:42:59,625 INFO > > > [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010: routed > > > message with ID: 48, result: OK > > > 2023-12-08 20:43:07,437 WARN [org.apache.activemq.artemis.core.server] > > > AMQ222186: unable to authorise cluster control: AMQ219016: Connection > > > failure detected. Unblocking a blocking call that will never get a > > response > > > > > > > > > > > > PLEASE NOTE: This e-mail from Redpoint Global Inc. (“Redpoint”) is confidential and is intended solely for the use of the individual(s) to whom it is addressed. If you believe you received this e-mail in error, please notify the sender immediately, delete the e-mail from your computer and do not copy, print or disclose it to anyone else. If you properly received this e-mail as a customer, partner or vendor of Redpoint, you should maintain its contents in confidence subject to the terms and conditions of your agreement(s) with Redpoint.
