Hey Justing, thanks for the help. We are using version 2.31.2, which is pretty recent, if not the latest.
With regards to the log traces, other AMQ threads show similar behavior, and our system isn’t doing much until AMQ-client stabilizes. I am running on a k3d cluster and the failover is triggered by me deleting the dm-activemq-live pod, which k8s then spins back up. Also, our slave config does NOT specify “allow-failback=true”, so the spin up of the new dm-activemq-live instance does not trigger a fail-back, which is intentional to eliminate failover-failback chaos in such a short time. I am trying to setup a simpler test case by using one of the examples, artemis/2.31.2/examples/features/ha/replicated-failback-static, however, I am getting this NPE on line 112: Caused by: java.lang.NullPointerException: Cannot invoke "javax.jms.TextMessage.getText()" because "message0" is null The NPE seems to indicate that, you cannot call consumer.receive() again even if NOT acknowledged, which seems to contradict the comments on the code. I have run into this same behavior in another example as well, artemis/2.31.2/examples/features/ha/replicated-failback. Can you please confirm the expected behavior of the example code in the 2.31.2 version? I can then modify the code accordingly. Or please point me to a working sample code. Thanks, Lino [rg] <https://www.redpointglobal.com/> Lino Pereira C++ Developer, Redpoint Global Inc. 34 Washington Street, Suite 205 Wellesley Hills, MA 02481 lino.pere...@redpointglobal.com<mailto:lino.pere...@redpointglobal.com> From: Justin Bertram <jbert...@apache.org> Sent: Monday, January 8, 2024 9:20 PM To: users@activemq.apache.org Subject: Re: Looking for HA/replication boilerplate broker.xml and advice (continued) *** [Caution] This email is from an external source. Please use caution responding, opening attachments or clicking embedded links. *** > Now, the backup process does start processing msgs right away... What evidence do you have of that? > ...the ActiveMQ-client needs 2 “Trying reconnection attempts”, 1-minute > apart, before it stabilizes after the failover and is once again usable. The log snippets you provided show a reconnection attempt back to dm-activemq-live-svc:61616 (i.e. the broker which failed) and then to dm-activemq-backup-svc:61617 both of which fail almost immediately. Then right after that another connection attempt is made to dm-activemq-live-svc:61616 which appears to take a minute to fail (i.e. from 20:03:44.874 to 20:04:44.933). Are there any other logs during this minute? Did you happen to get any thread dumps during this time? If so, what did they show? If not, could you? What was happening with dm-activemq-live-svc during this time? Perhaps there was something weird going on with the network related to whatever you were doing with that machine to trigger failover. You might consider simplifying your test-case and then adding complexity bit by bit to see what triggers the difference in behavior. For example, run your test with the client and both brokers on the *same machine*. If that works then move just the client to a remote machine. If that works then test the client just reconnecting to the primary broker upon restart (i.e. no failover). Then finally put all three entities on separate machines and test failover. Hopefully such an exercise will give you a better understanding of where the problem might lie. Lastly, please indicate what version of ActiveMQ Artemis you're using. Justin On Mon, Jan 8, 2024 at 7:43 PM Lino Pereira <lino.pere...@redpointglobal.com.invalid<mailto:lino.pere...@redpointglobal.com.invalid>> wrote: Hi Justin, I’m working with John Lilley on getting an ActiveMQ HA configuration operational. Our current HA config is replication with master and slave instances and with static connections, and it seems to be working, except that the ActiveMQ-client is taking over 1 minute to reconfigure/reconnect itself after a failover from the live to the backup instance. Now, the backup process does start processing msgs right away, but the ActiveMQ-client needs 2 “Trying reconnection attempts”, 1-minute apart, before it stabilizes after the failover and is once again usable. I have included the relevant log traces from the Thread-0 ActiveMQ-client showing the Connection failure detection all the way through when the client is usable again, over 1 minute later. I am hoping that, based on the trace, you can find something that, perhaps, we should be doing differently in the client side? Here is our Client connection url: (tcp://dm-activemq-live-svc: 61616,tcp://dm-activemq-backup-svc:61617)?ha=true&reconnectAttempts=100&initialConnectAttempts=100 Should we also be including the corresponding connector names? In an earlier email with John Lilley you recommended 2 brokers using “shared storage” instead of “replication”, which I will try next, however, since this issue seems to be on the client-side, I want to make sure our client-side is setup correctly. Thanks in advance for your help. Lino Thread-0 log traces: 2024-01-08T20:03:43.852 [Thread-0 (ActiveMQ-client-global-threads)] client.connectionFailureDetected:293 [] WARN - AMQ212037: Connection failure to dm-activemq-live-svc/10.43.197.248:61616<http://10.43.197.248:61616> has been detected: AMQ219015: The connection was disconnected because of server shutdown [code=DISCONNECTED] 2024-01-08T20:03:43.852 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.failoverOrReconnect:587 [] DEBUG - Failure captured on connectionID=a8a4a77c, performing failover or reconnection now ActiveMQDisconnectedException[errorType=DISCONNECTED message=AMQ219015: The connection was disconnected because of server shutdown] --- 2024-01-08T20:03:43.857 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.getConnectionWithRetry:956 [] DEBUG - Trying reconnection attempt 0/200 2024-01-08T20:03:43.857 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.createTransportConnection:1328 [] DEBUG - Trying to connect with connectorFactory=org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory@68dd9fb0<mailto:connectorFactory=org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory@68dd9fb0> and currentConnectorConfig: TransportConfiguration(name=null, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=dm-activemq-live-svc&reconnectAttempts=200&ha=true&initialConnectAttempts=200 2024-01-08T20:03:43.857 [Thread-0 (ActiveMQ-client-global-threads)] NettyConnector.start:553 [] DEBUG - Connector NettyConnector [host=dm-activemq-live-svc, port=61616, httpEnabled=false, httpUpgradeEnabled=false, useServlet=false, servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true] using nio 2024-01-08T20:03:43.857 [Thread-0 (ActiveMQ-client-global-threads)] NettyConnector.start:772 [] DEBUG - Started NIO Netty Connector version 4.1.101.Final to dm-activemq-live-svc:61616 2024-01-08T20:03:43.857 [Thread-0 (ActiveMQ-client-global-threads)] NettyConnector.createConnection:877 [] DEBUG - Remote destination: dm-activemq-live-svc/10.43.197.248:61616<http://10.43.197.248:61616> 2024-01-08T20:03:43.864 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.openTransportConnection:1215 [] DEBUG - Connector towards NettyConnector [host=dm-activemq-live-svc, port=61616, httpEnabled=false, httpUpgradeEnabled=false, useServlet=false, servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true] failed 2024-01-08T20:03:43.866 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.createTransportConnection:1328 [] DEBUG - Trying to connect with connectorFactory=org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory@68dd9fb0<mailto:connectorFactory=org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory@68dd9fb0> and backupConnectorConfig: TransportConfiguration(name=netty-connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61617&host=dm-activemq-backup-svc 2024-01-08T20:03:43.866 [Thread-0 (ActiveMQ-client-global-threads)] NettyConnector.start:553 [] DEBUG - Connector NettyConnector [host=dm-activemq-backup-svc, port=61617, httpEnabled=false, httpUpgradeEnabled=false, useServlet=false, servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true] using nio 2024-01-08T20:03:43.866 [Thread-0 (ActiveMQ-client-global-threads)] NettyConnector.start:772 [] DEBUG - Started NIO Netty Connector version 4.1.101.Final to dm-activemq-backup-svc:61617 2024-01-08T20:03:43.866 [Thread-0 (ActiveMQ-client-global-threads)] NettyConnector.createConnection:877 [] DEBUG - Remote destination: dm-activemq-backup-svc/10.43.11.76:61617<http://10.43.11.76:61617> 2024-01-08T20:03:43.872 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.openTransportConnection:1215 [] DEBUG - Connector towards NettyConnector [host=dm-activemq-backup-svc, port=61617, httpEnabled=false, httpUpgradeEnabled=false, useServlet=false, servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true] failed 2024-01-08T20:03:43.873 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.createTransportConnection:1297 [] DEBUG - no connection been made, returning null --- 2024-01-08T20:03:44.873 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.getConnectionWithRetry:956 [] DEBUG - Trying reconnection attempt 1/200 2024-01-08T20:03:44.873 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.createTransportConnection:1328 [] DEBUG - Trying to connect with connectorFactory=org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory@68dd9fb0<mailto:connectorFactory=org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory@68dd9fb0> and currentConnectorConfig: TransportConfiguration(name=null, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=dm-activemq-live-svc&reconnectAttempts=200&ha=true&initialConnectAttempts=200 2024-01-08T20:03:44.873 [Thread-0 (ActiveMQ-client-global-threads)] NettyConnector.start:553 [] DEBUG - Connector NettyConnector [host=dm-activemq-live-svc, port=61616, httpEnabled=false, httpUpgradeEnabled=false, useServlet=false, servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true] using nio 2024-01-08T20:03:44.873 [Thread-0 (ActiveMQ-client-global-threads)] NettyConnector.start:772 [] DEBUG - Started NIO Netty Connector version 4.1.101.Final to dm-activemq-live-svc:61616 2024-01-08T20:03:44.873 [Thread-0 (ActiveMQ-client-global-threads)] NettyConnector.createConnection:877 [] DEBUG - Remote destination: dm-activemq-live-svc/10.43.197.248:61616<http://10.43.197.248:61616> 2024-01-08T20:03:44.874 [Thread-8 (ActiveMQ-client-netty-threads)] NettyConnector.initChannel:762 [] DEBUG - Added ActiveMQClientChannelHandler to Channel with id = ced14fbc 2024-01-08T20:04:44.933 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.openTransportConnection:1215 [] DEBUG - Connector towards NettyConnector [host=dm-activemq-live-svc, port=61616, httpEnabled=false, httpUpgradeEnabled=false, useServlet=false, servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true] failed 2024-01-08T20:04:44.933 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.createTransportConnection:1328 [] DEBUG - Trying to connect with connectorFactory=org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory@68dd9fb0<mailto:connectorFactory=org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory@68dd9fb0> and backupConnectorConfig: TransportConfiguration(name=netty-connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61617&host=dm-activemq-backup-svc 2024-01-08T20:04:44.933 [Thread-0 (ActiveMQ-client-global-threads)] NettyConnector.start:553 [] DEBUG - Connector NettyConnector [host=dm-activemq-backup-svc, port=61617, httpEnabled=false, httpUpgradeEnabled=false, useServlet=false, servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true] using nio 2024-01-08T20:04:44.933 [Thread-0 (ActiveMQ-client-global-threads)] NettyConnector.start:772 [] DEBUG - Started NIO Netty Connector version 4.1.101.Final to dm-activemq-backup-svc:61617 2024-01-08T20:04:44.933 [Thread-0 (ActiveMQ-client-global-threads)] NettyConnector.createConnection:877 [] DEBUG - Remote destination: dm-activemq-backup-svc/10.43.11.76:61617<http://10.43.11.76:61617> 2024-01-08T20:04:44.936 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.createTransportConnection:1336 [] DEBUG - Connected with the backupConnectorConfig=TransportConfiguration(name=netty-connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61617&host=dm-activemq-backup-svc 2024-01-08T20:04:44.966 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.getConnectionWithRetry:964 [] DEBUG - Reconnection successful 2024-01-08T20:04:44.969 [Thread-0 (ActiveMQ-client-global-threads)] ActiveMQSessionContext.reattachOnNewConnection:888 [] DEBUG - Couldn't reattach session 11, performing as a failover operation now and recreating objects 2024-01-08T20:04:45.002 [Thread-0 (ActiveMQ-client-global-threads)] ActiveMQSessionContext.reattachOnNewConnection:888 [] DEBUG - Couldn't reattach session 10, performing as a failover operation now and recreating objects 2024-01-08T20:04:45.019 [Thread-0 (ActiveMQ-client-global-threads)] AbstractProducerCreditsImpl.reset:123 [] DEBUG - reset credits on address activemq.management<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2factivemq.management&c=E,1,hqiYs6vnJzfq-X42G9W9ohJtYgn9LRPR-82RHxMyPnKUD5xBwCbo8W6b-PegfmAGwvEENlDO0k2IkoGFFegGxR-H6J72UHHlpEFbGtJzhCF3sucQ&typo=1&ancr_add=1> 2024-01-08T20:04:45.019 [Thread-0 (ActiveMQ-client-global-threads)] AbstractProducerCreditsImpl.checkCredits:178 [] DEBUG - Requesting 65536 credits on address activemq.management<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2factivemq.management&c=E,1,KX9atZvl9XxpGPWobW9hXsmaLrVekfw2221zvD0PJ9Ik3fEQVyHKcZZCe7MQRciamxD-T-WDEk5Xa7_cAt0PSHwVW3jG7zzhq8XH9XIRC8EXmg,,&typo=1&ancr_add=1>, needed = 65536, arriving = 0 2024-01-08T20:04:45.019 [Thread-0 (ActiveMQ-client-global-threads)] AbstractProducerCreditsImpl.requestCredits:187 [] DEBUG - Request 65536 credits on address activemq.management<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2factivemq.management&c=E,1,ZOCwWoQv34cYLuw9hZvJzBHfSrX6oQeQ_3-SetFMJa1hRC6lXgiKsAElOhfZ_vfnBhRGRJPZjSOqHIbwWO2ltSYYY7ElQjyItp0uDo79_jaDkw,,&typo=1&ancr_add=1> 2024-01-08T20:04:45.020 [Thread-0 (ActiveMQ-client-global-threads)] ClientSessionFactoryImpl.failoverOrReconnect:587 [] DEBUG - Failure captured on connectionID=a8a4a77c, performing failover or reconnection now ActiveMQNotConnectedException[errorType=NOT_CONNECTED message=AMQ219006: Channel disconnected] at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.connectionDestroyed(ClientSessionFactoryImpl.java:410<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fClientSessionFactoryImpl.java%3a410&c=E,1,obbNTaFqn6mglgyZXy1S-0nBLNVtEYlkZBwAEKSGDGN0RuqlR_-7fIQPW4ELNqVFJdenAcKh65C3yQ2b_Lef7W01Tst6ZTrmI66KfHLae572jXv-HF3O&typo=1&ancr_add=1>) at org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector.close(NettyConnector.java:835<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fNettyConnector.java%3a835&c=E,1,ytVVTXzfq-J1NXl2ct7T_47F5EbfZNQTXP8dlqw720EiCe32c6CLwlpk0-v4MzUD61FErq0nkRTalFpcTtIN8TDeBvz70rzb__TyeIDm48ZKfFU28qQ,&typo=1&ancr_add=1>) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:666<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fClientSessionFactoryImpl.java%3a666&c=E,1,KQYUuEgOaR6V8a_ZC_xBpKZmDneB4LTL1cQzn2Gqtla8ntHJ1fgRs9KelLgvbMd9zvBTg8maXUerYZYQZa-DlfK7fon0JZ0O74PNJ1SOCHqIPN4fZQ6Xv731OGOD&typo=1&ancr_add=1>) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:566<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fClientSessionFactoryImpl.java%3a566&c=E,1,OSVh6NSORxz7qZ6eIuUDgWskoYfyXT4NNdjJYGMbH7HeWEUiymys2DnptjYKdmQfqjfHmI-egRZ31yWwd9MKzEd-gbbsuTUsqn_46niKaw,,&typo=1&ancr_add=1>) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed(ClientSessionFactoryImpl.java:1407<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fClientSessionFactoryImpl.java%3a1407&c=E,1,DM2boEBGg44zzcd0m37ITkOnfDrkqLE8ZSFGJpEI4AGo2GyOI2yKX08AEkKHxw_UnDMF1vccYEvESFDXjXBBjjIGO5TEsMLWZHbJF3AOIEQ73uc,&typo=1&ancr_add=1>) at org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners(AbstractRemotingConnection.java:98<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fAbstractRemotingConnection.java%3a98&c=E,1,wipgGeA7UVZIy1uuqvuxJEx4BA8TvYfSYmO617IgrvnOo-z7sn1vUjXFqIEU9M1HmedBVoHxB_LpvuEZ9p_biioPM8V-V9eS8i8oST46GvBhZZIc&typo=1&ancr_add=1>) at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2forg.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail&c=E,1,0FNTjscT_d2CNIid0J8d_x7C7Peb6_neE-a_MZeKqkM1o_sPGujvpkXuiLv6kms9wSFyYqSux86C1MX2R8WKM8IZBsjKt7sCs03U2lRYlVbw2eTDg0ihSEV1nA,,&typo=1&ancr_add=1>(RemotingConnectionImpl.java:212<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fRemotingConnectionImpl.java%3a212&c=E,1,3yKPqxpIi9n43QJSCQiysKGlEYttZ5Dmuirc7jUAWNz6orLfIUCYwdDEJ3eU0QiHywsNAJdWqrvnx5rZK6-8TTJb6o8tr1WSvSL7omLF&typo=1&ancr_add=1>) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$CloseRunnable.run<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fCloseRunnable.run&c=E,1,6B4qLrj7q9DzDb0Trp9xgst7cOBhbD4hcEcGyPFul5E_mh3_oGmColB9YQkQTB1gJ6x5Mcm0VwvBLddpDpfFkcZwlzb4YNR9Ym5fcDclFW97pIrQ9oG4&typo=1&ancr_add=1>(ClientSessionFactoryImpl.java:1172<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fClientSessionFactoryImpl.java%3a1172&c=E,1,KqR93A_gEw1FpU7__PsYjSkOC5LH0zFTSKu5K9YlCdVAz3h_c9EkBndO7aliE4RSbK_qslEZp0b-YzJfCOxmB22jAfplsYxguMl1qjC0rWpNjfr9_fh4RjNL3AOK&typo=1&ancr_add=1>) at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fOrderedExecutor.java%3a57&c=E,1,mhChlKkihYjDw_Kk5SL1d_4eb7zhBlSzri_nwjOUsrs6MAvEfDAAjia07yTMUU6CQqc0WbDS2Fn8HJJI0UwJh1N7SKiz9dsP1gJLmgY66huLhXZE9c-3ECGXyIWQ&typo=1&ancr_add=1>) at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fOrderedExecutor.java%3a32&c=E,1,nn8IDmg6XQJLLXzH7kunXDNZa_3A7fUlnHdRCLoha7gQWJO1QeY32duGFAQMYaFHsfIbHbv142mei8HfCqRwrpuUfikYxLkFoXaD3pORZb9roBJs0H8,&typo=1&ancr_add=1>) at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fProcessorBase.java%3a68&c=E,1,DPLEvt-MPmmzv78LEkeXtYHM2q37m12TxX-nqjuIJurQdeXydsSakco9t_O7jgBbQ6y115xxdDb8IonFZbM86zgRDLcvUmeOSJuDmZybD_6JaTX0zpbQ&typo=1&ancr_add=1>) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fThreadPoolExecutor.java%3a1136&c=E,1,1vNFV-mDaLK92lJo0Mq6jvkCDzuwrSHT4lp1nB6KTzOCSO6comH5xNpEcvtZ_9pc0cQCD2kgDfB5xdxg0tAX2dF5oG5DHY6kxWXOBLnuAwiz1BkEFhw,&typo=1&ancr_add=1>) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fWorker.run&c=E,1,aLydz0SGsclotqCro2upcTg44B_Ij7ckjApLu4B5JUwZKKdMo5z3RyUR4ikHFY8L3t_yB87Zp5atGRwoAPA1L9Mc9c681UjFrG16n-udVjeHtxQNxx8V-Nug&typo=1&ancr_add=1>(ThreadPoolExecutor.java:635<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fThreadPoolExecutor.java%3a635&c=E,1,1mZX-u-XHEZ_e4Ut7YETRvWm7U_qkC-q9QaPeHah4K4yNl5foyM1VSExCtYiisknkelDj6_K2MUHGc65Kd-6-JG_TXLby0OPWodJVjX3BtKaDNBUzusgpzT6ajL7&typo=1&ancr_add=1>) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2f1.run&c=E,1,-nbEPDxI7J-DFvbJfVVt2Cs1Vi_hN5bw-8vQF3SlM1nthw3VbJLWmS1h73cupUxqBoHCImTzDqI1m5gutx3ZriLpLpU7IqutHCqZuT6CTVWACPtj&typo=1&ancr_add=1>(ActiveMQThreadFactory.java:118<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fActiveMQThreadFactory.java%3a118&c=E,1,-lUcdhDzaHWdAWLCc0IxWCFkKtKHxoEd64GWhzXAbeuG_Zm9mvn624BY6uS8MgQn-s2TMF8PbEPYVbOOy6BJuI8ByeiuAF9Y8f2I5wib&typo=1&ancr_add=1>) [rg]<https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fwww.redpointglobal.com%2f&c=E,1,T2UV-1dji1RODRcU4Ff4j_IzfTH2n47BatzyLZcEOU8VROFr3GS8re7CjDcnnHCSaPEtR-35J3z30dnYcz5K66rVsMVxgDiyh0FvEDKnN67naHjYbn0NpzAc&typo=1> Lino Pereira C++ Developer, Redpoint Global Inc. 34 Washington Street, Suite 205 Wellesley Hills, MA 02481 lino.pere...@redpointglobal.com<mailto:lino.pere...@redpointglobal.com> PLEASE NOTE: This e-mail from Redpoint Global Inc. (“Redpoint”) is confidential and is intended solely for the use of the individual(s) to whom it is addressed. If you believe you received this e-mail in error, please notify the sender immediately, delete the e-mail from your computer and do not copy, print or disclose it to anyone else. If you properly received this e-mail as a customer, partner or vendor of Redpoint, you should maintain its contents in confidence subject to the terms and conditions of your agreement(s) with Redpoint. PLEASE NOTE: This e-mail from Redpoint Global Inc. (“Redpoint”) is confidential and is intended solely for the use of the individual(s) to whom it is addressed. If you believe you received this e-mail in error, please notify the sender immediately, delete the e-mail from your computer and do not copy, print or disclose it to anyone else. If you properly received this e-mail as a customer, partner or vendor of Redpoint, you should maintain its contents in confidence subject to the terms and conditions of your agreement(s) with Redpoint.