Hello John, " I don’t expect anyone to know the answer, but I’m hoping someone can tell me what additional diagnostics might be enabled in the client that would show what the client is waiting on? " I think the suggestions of
*Justin Bertram* - Wednesday, February 21, 2024 9:56:28 PM GMT+1 are a very good answer to your question. *Distinti Saluti / *Kind Regards M.G. Il giorno ven 23 feb 2024 alle ore 14:39 John Lilley <[email protected]> ha scritto: > Vilius, > > > > Thanks for the reply! > > > > We have not seen this outside of K8S, but the question is why… > > We are hard-killing the active broker (shell into the node and do a “sudo > kill -9”) > > > > And I can see in our logs that the backup broker senses the demise of the > primary broker and takes over within two seconds: > > > > 2024-02-16 22:42:56,523 INFO [org.apache.activemq.artemis.core.server] > AMQ221007: Server is now live > > > > AND it immediately processes other messages , which we log in the broker > pod using a plugin: > > 2024-02-16 22:42:58,074 INFO > [net.redpoint.rpdm.artemis_logger.RpdmArtemisLogger] SEND: … > > 2024-02-16 22:42:58,254 INFO > [net.redpoint.rpdm.artemis_logger.RpdmArtemisLogger] DELIVER:… > > > > To be clear, ALMOST all of the failover succeeds within a couple seconds, > and a lot of messages are being processed. But there seems to be one > message caught in the act of sending just as the failover happens, and it > takes 40 seconds to report an issue: > > > > 2024-02-16T22:43:35.988 [http-nio-9910-exec-9] JmsProducerPool.send_:376 > [9v2zwvclclrc] WARN - Error sending message, will retry > javax.jms.JMSException: AMQ219014: Timed out after waiting 10000 ms for > response when sending packet 71 > > > > Our own application-level logging show that we attempted to send the > message just at the time of failover: > > 2024-02-16T22:42:55.981 [http-nio-9910-exec-9] > JmsRpcClientChannel.prepareCall:84 [9v2zwvclclrc] INFO - REQUEST OUT: { > "header": {"version":1,"type":"get_task_status_request","id":"9v2zwvclclrc" > > > > We know this is the same message because we tag them all with unique IDs, > in this case “9v2zwvclclrc”, and stuff that into a thread context for > Log4j2 so it’s tracked. > > > > So our question is, what happened during the 40 seconds? Clearly the > broker went down, and the backup sensed that and took over, and everyone > quickly switched to the backup. Except for this one message. This is > consistent behavior under load. > > > > I don’t expect anyone to know the answer, but I’m hoping someone can tell > me what additional diagnostics might be enabled in the client that would > show what the client is waiting on? > > > > Thanks > > john > > > > > > > [image: rg] > <https://urldefense.com/v3/__https://www.redpointglobal.com/__;!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3H50EbXg$> > > John Lilley > > Data Management Chief Architect, Redpoint Global Inc. > > 34 Washington Street, Suite 205 Wellesley Hills, MA 02481 > > *M: *+1 7209385761 <+1%207209385761> | [email protected] > > *From:* Vilius Šumskas <[email protected]> > *Sent:* Wednesday, February 21, 2024 2:44 PM > *To:* [email protected] > *Subject:* RE: HA failover: Nothing we try reduces client recovery below > one minute > > > > **** [Caution] This email is from an external source. Please use caution > responding, opening attachments or clicking embedded links. **** > > > > Hi, > > > > are you able to reproduce your issue without Kubernetes layer on Artemis > instances? I’m not sure how exactly do you kill a pod, but 40 second > timeout very much looks like default pod grace timeout + 10 seconds. > > > > -- > > Vilius > > > > *From:* John Lilley <[email protected]> > *Sent:* Wednesday, February 21, 2024 8:45 PM > *To:* [email protected] > *Cc:* Lino Pereira <[email protected]> > *Subject:* HA failover: Nothing we try reduces client recovery below one > minute > > > > Greetings! > > > > We are having a devil of a time trying to reduce delay during a failover > event. We’ve set our URL to > > > (tcp://dm-activemq-live-svc:61616,tcp://dm-activemq-backup-svc:61617)?ha=true&reconnectAttempts=200&initialConnectAttempts=200&clientFailureCheckPeriod=10000&connectionTTL=10000&callTimeout=10000 > > > > But nothing seems to reduce the time it takes for a sender to detect the > issue and failover. With or without these parameters, it takes a full > minute to recover after failover. Are there other parameters to adjust? > Something in the broker? Is there some internal retry loop and timer that > waits longer that we can influence? > > > > We are testing failover by killing one of the AMQ pods. This almost > always succeeds without issue, except occasionally it doesn’t. > > > > The backup broker log shows that it assumes control very quickly > > 2024-02-16 22:42:56,523 INFO [org.apache.activemq.artemis.core.server] > AMQ221007: Server is now live > > 2024-02-16 22:42:56,533 INFO [org.apache.activemq.artemis.core.server] > AMQ221020: Started EPOLL Acceptor at 0.0.0.0:61617 for protocols > [CORE,MQTT,AMQP,STOMP,HORNETQ,OPENWIRE] > > 2024-02-16 22:42:58,074 INFO > [net.redpoint.rpdm.artemis_logger.RpdmArtemisLogger] SEND: HEADER= > {"version":1,"type":"void","id":"jagbjffeu3of","api":"hpa_metrics","method":"get_expected_pod_counts","authorization":"Q979k6lzOu9KmqMA89GbvmZnIAMNpJJP/TEAAa7Yjpo="}, > BODY={"message_type":"void"} > > 2024-02-16 22:42:58,254 INFO > [net.redpoint.rpdm.artemis_logger.RpdmArtemisLogger] DELIVER: HEADER= > {"version":1,"type":"void","id":"jagbjffeu3of","api":"hpa_metrics","method":"get_expected_pod_counts","authorization":"Q979k6lzOu9KmqMA89GbvmZnIAMNpJJP/TEAAa7Yjpo="}, > BODY={"message_type":"void"} > > > > In our app logs, we show that we are attempting to send a message to the > AMQ broker at the time of failover: > > 2024-02-16T22:42:55.981 [http-nio-9910-exec-9] > JmsRpcClientChannel.prepareCall:84 [9v2zwvclclrc] INFO - REQUEST OUT: { > "header": > {"version":1,"type":"get_task_status_request","id":"9v2zwvclclrc","api":"test_harness","method":"get_task_status","instance":"combined","authorization":"***REDACTED***"}, > "body":{"id":"d1123865-ac47-4238-ab02-5f2324a43264","progress_start_index":0,"message_type":"get_task_status_request"} > } > > > > But *then it takes 40 seconds for the “10000ms” timeout to happen.* > > 2024-02-16T22:43:35.988 [http-nio-9910-exec-9] JmsProducerPool.send_:376 > [9v2zwvclclrc] WARN - Error sending message, will retry > javax.jms.JMSException: AMQ219014: Timed out after waiting 10000 ms for > response when sending packet 71 > > > > Our problem is really that this 40-second delay seems to push everything > back so that recovery takes over a minute. Once we get into the one-minute > mark, we start hitting several timeouts, like our own RPC timeout setting > and the nginx ingress controller for K8S. How can we know why this takes > so long? We’ve set every timeout we can find for the AMQ client to 10 > seconds. Is there some other setting we need to adjust? > > > > > > Meanwhile the primary broker pod has returned, and the backup decides it > is not needed: > > 2024-02-16 22:43:16,115 INFO [org.apache.activemq.artemis.core.server] > AMQ221002: Apache ActiveMQ Artemis Message Broker version 2.31.2 > [10952195-b6ec-11ee-9c87-aa03cb64206a] stopped, uptime 16 minutes > > 2024-02-16 22:43:16,115 INFO [org.apache.activemq.artemis.core.server] > AMQ221039: Restarting as Replicating backup server after live restart > > > > But our app is waiting for its response on the reply-to queue, and gets > this error nearly a minute later: > > 2024-02-16T22:43:58.033 [Thread-6] JmsStaticConnectionPool.onException:78 > [] ERROR - Receive error occurred. javax.jms.JMSException: > ActiveMQDisconnectedException[errorType=DISCONNECTED message=AMQ219015: The > connection was disconnected because of server shutdown] > > > > This is the stack trace at time of the first timeout: > > at > org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking( > ChannelImpl.java:550 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fChannelImpl.java*3a550&c=E,1,hb7aGgx-vHCuvCZitz5EiH3vZi_XC1uzDAFkEHTxXekOnlXOXMP4ibOn6V4KDEB1MD9djxVG0pCPgN9wOUF6dYSlvQjZDxnFdtoXzY5E9EG2FGcoIhbpnw,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1g6FXp6A$> > ) > > at > org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking( > ChannelImpl.java:452 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fChannelImpl.java*3a452&c=E,1,gF3UMbAQeYfSPT8cA2REDOjwMzwGDYxipRXgXQCDiWReJE7I74vBcqvEKJaemOXlOU1-G4hJB4qNbaM1K_LmdumKZ7DAKdaqBEqDSniSuD46bi8,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt20cJ_Apg$> > ) > > at > org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.sendFullMessage( > ActiveMQSessionContext.java:588 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fActiveMQSessionContext.java*3a588&c=E,1,9Q1r0SpyXwLIB-EMPBJw-aI7zdjFiqijbOY_h2LMwTn-B3FR_vV-GEiHQ8QWZkRub6NnY7OQvVJRoEEjiWottZjxkgKMpVcwv7wPDZiq6s8LgQ,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3h25NFSw$> > ) > > at > org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.sendRegularMessage( > ClientProducerImpl.java:305 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fClientProducerImpl.java*3a305&c=E,1,_M2sIEUuJWPDvj5jVjnOj7fletSz95vJyLou8Gve4mp0Wae3oDPYH9uOyeCWbMIwsiPMoIiEo_WDB9yfskeia55SN4MiEe3LZEGXVrtUFone1wdABZb6gOPC2m9w&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0SFxm2mw$> > ) > > at > org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.doSend( > ClientProducerImpl.java:277 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fClientProducerImpl.java*3a277&c=E,1,oQ1uB8jzdsj7gUjIdMAYjGGFhBP2jGP0m_2VkTenJuUFc4-qsEa92fBmhf4b0zrZRLdm-Ctl5FiTaRL7ALhu7_f_P5fL9RmiP1TcxNc7wZhHCcA,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2SJGiapg$> > ) > > at > org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.send( > ClientProducerImpl.java:147 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fClientProducerImpl.java*3a147&c=E,1,2JXRlcwBE1DgWg6Nk_YIcpOYWcBvJBBdvO075qubJv8IiA1-t8CMakQozpCrfoZ0t4l0nLmaIhS5d6py-XVc-yhJzsTx-N42mWmNZOT6yHDC2ljCuzo,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2KXjWXmg$> > ) > > at > org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.send( > ClientProducerImpl.java:129 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fClientProducerImpl.java*3a129&c=E,1,Jzd6hLneXOmyA5D7vxY21LhDyWMJrnaXXxf8o-gF4Erwuf8q-9gnnJY-qV8bRHP7bHdZZtiUFhv1qNRCQl-ysyzGr6_YjHKyBoOjv1rEjOYdhzI0zgkpOw,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0vpIbCDw$> > ) > > at > org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.doSendx( > ActiveMQMessageProducer.java:483 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fActiveMQMessageProducer.java*3a483&c=E,1,0yivDeiDyAHDMF-bwOucG30LADmXrdItj_pQJnK_SRY9vGr_DfKM_LL-LEoVVOZ6x34Zuzto8nyL_HNWC7xmqAe7ZEjcFKAUJSZ-Yjv_eFT-PJ-2pUKa&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt34eeiSRw$> > ) > > at > org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.send( > ActiveMQMessageProducer.java:221 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fActiveMQMessageProducer.java*3a221&c=E,1,NuJzonWY3dCHvQuGAvWh5R9wQe_JiLaD6RfhoD0drObSHoBwRrh5zqwLDiOv2F3MUqtErz3Q5t8JJj5iqy9DHLusThoeoJwww3tnwamhrMVFj1oZ7aSh5Q,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2iAw6dFw$> > ) > > at net.redpoint.ipc.jms.JmsProducerPool.send_( > JmsProducerPool.java:372 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fJmsProducerPool.java*3a372&c=E,1,fLwu6-XhE-35j29l5My769fvdrx1oKFsEumlcnxIJ-zl3lpqQHIknDa4dZS2DSKoRlr4XIkh18khm9yTNif1ZUgCfPW4T3JUc54rnFzytNc,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3YTy5riw$> > ) > > at net.redpoint.ipc.jms.JmsProducerPool.sendRequest( > JmsProducerPool.java:301 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fJmsProducerPool.java*3a301&c=E,1,L5JFmpnTDTQel8SbnG3VK4xEP8oY_gtJYyoVXO_F1DJnNe5rrOo4AySDLqTxMtiqc5Sv2j-a2dIG16jBZ-d4LNnUK_3MUhcD9diVP_2Do4_KTe9r&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0tq8gcfg$> > ) > > at net.redpoint.ipc.jms.JmsRpcClientChannel.sendRequest( > JmsRpcClientChannel.java:228 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fJmsRpcClientChannel.java*3a228&c=E,1,CZrfHT-US8E2FAm8Os73ubg9knljLI7YUiud8KYRU6cShYCwqNJ5cz2B2aC_G4nBOGULRt1AZed5AhfU9Qu2pubWVzKotlmqo6chQELiwyc4k4BjKvy0702Q&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3_4x6g6w$> > ) > > at net.redpoint.ipc.jms.JmsRpcClientChannel.invokeRaw( > JmsRpcClientChannel.java:202 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fJmsRpcClientChannel.java*3a202&c=E,1,gv6BduvhcXnCEEmt3MhN4xVBzok9tDgp18LcXG5zlG49w7C_x40T8KqCdYngblOJh4unBVqWeWqv_kwjWTHbFq4jos3V8aInP1_BWc6Q&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2s59WSyA$> > ) > > at net.redpoint.ipc.jms.JmsRpcClientChannel.call > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fnet.redpoint.ipc.jms.JmsRpcClientChannel.call&c=E,1,nR8Axi6nKaDe0idP2FZMm4-S0OB2WBHH0-vP-7T0n4y8bP2jnr8PyKQEkIh-GNbOulFVyo-JmkNEsFJq2kRASqXXzC1nK5ih-T8jawiX12EX18KZ&typo=1&ancr_add=1__;JSUl!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1_dQaRyQ$> > (JmsRpcClientChannel.java:101 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fJmsRpcClientChannel.java*3a101&c=E,1,uDLLFYMJiricH0b2CCzCbONAYihnNI1rkLL_9o3o5XX8VSwfL42BSuXV314BKJbKYXk3dquUadeP3jwSJj-tPB3mO2FYtaFj3fh_2x6yXBK-Qtar70K_&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0erUTSeg$> > ) > > at net.redpoint.ipc.clients.RpcClientBase._sync( > RpcClientBase.java:169 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fRpcClientBase.java*3a169&c=E,1,wyTB_YLwnilukCP4yaV-KnfiFMrJ_d219DnfTQjBnr0T8xdChqgpha2b3O0CcQKmua6RRShv7kQXt2cDL57kooPnGPR3-BRqa6PUUlyTB7iV2Lv0z7njICo,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0D2cXUuQ$> > ) > > at net.redpoint.ipc.clients.RpcClientBase._rpc( > RpcClientBase.java:237 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fRpcClientBase.java*3a237&c=E,1,LWdl8WVF2wTGJOIp7_iMum6NPwuAg1IuIskb-1ta9DuBVPvU2ZUDiHcWfbamo5braSd7dxBcn43KYr_foQZvO0TXasHJKFRnAzVHKKivXATR_x2T&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3URjlVSQ$> > ) > > at > net.redpoint.rpdm.ipc.clients.TestHarnessClient.getTaskStatus( > TestHarnessClient.java:229 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fTestHarnessClient.java*3a229&c=E,1,PgvRlByMcnxeUXiBba7IiePoAnF4IVs_7Lxnrd_8jq8Q-fIuuubZr_y7olwgeULE3XlvUsXDNp3_9NhOFKteaq-i16oXJtIJx_X7AEnPucP-fu4,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2AY4kxNA$> > ) > > at > net.redpoint.rpdm.ipc.web_service_gateway.TestHarnessHttpServer.lambda$getTaskStatus$25( > TestHarnessHttpServer.java:390 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fTestHarnessHttpServer.java*3a390&c=E,1,6IBjNSLNijOWzvN4rhSDmgiqTjpEHPembkvVBDdhfGLziM1KCtYXTrNIXtmV93l3zOoKEVXSeigVB8bRDpPEy6lhIMT7eXHSfG3-EgPqV8YVibSm3GK7URjjPw,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2GZtq9pA$> > ) > > at net.redpoint.ipc.SecurityControl.doAsNoThrow( > SecurityControl.java:272 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fSecurityControl.java*3a272&c=E,1,JEaIr0zNLFXmv2-s5Q2nqgJtrQ6Xe_7nI1J5cgFtpjXbGHFfuEDxpXCJql-hCKG2zwv67dnsNsq1bDiCfHaGOM_1-um8PTmJwWr5lly2wBvaCDnxCP7xLoW6&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2pX8cpDQ$> > ) > > at > net.redpoint.rpdm.ipc.web_service_gateway.TestHarnessHttpServer.lambda$getTaskStatus$26( > TestHarnessHttpServer.java:390 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fTestHarnessHttpServer.java*3a390&c=E,1,cr6EzskPARQs3atjS-qqk13h29wSx5_Ost_yPOnOsCvrReemnvm5SneMupe5u7j3QG-MQNRzjiHhaPSCz14D0aS4qcC8AqMjGPxxr5ldyf0,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0bMUq77g$> > ) > > at net.redpoint.ipc.InstanceControl.doAsNoThrow( > InstanceControl.java:84 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fInstanceControl.java*3a84&c=E,1,KEjKjtppW9ddjt2brqtRGzD7gf9hKSZaBiRK_pT4nfQlFKVX7I1MI6xmv7Ae7Haq7lU1mm5AEvHVdurLVtx75mtR8WhQ2DpUekgrYqPCFIV_&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1JIG_JIw$> > ) > > at > net.redpoint.rpdm.ipc.web_service_gateway.TestHarnessHttpServer.getTaskStatus( > TestHarnessHttpServer.java:390 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fTestHarnessHttpServer.java*3a390&c=E,1,xxxxXpDzVQJIHIDnITNGO8paqNClGREjDResRK_0fPJ4RlIbFS0gq33a6q6Fa_ytIrAq7LiutsX_07eAsBrSW1D8OSUlQaj0eJP6XFt691Pv&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2w7GGphQ$> > ) > > at > jdk.internal.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) > > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fDelegatingMethodAccessorImpl.java*3a43&c=E,1,dtPs-F3WSSDUIYKORhz5rbXvE7bGcxPI4Al1JWQIRY8xj66Q7Ack4YUCB8Nw7udnKiMx1n-rq-No4Nyj3whIzuGpiFBSboAGz7zyfyMa5g1qE_BQwIFkFd4m5g,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3mKGevRw$> > ) > > at java.base/java.lang.reflect.Method.invoke( > Method.java:568 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fMethod.java*3a568&c=E,1,_UdYZqtoNh8X1NEr2kGR0aTolwK_yJVCjNFBA-hXoLQT1e43ZfwtMHdJIUIGbzyINUpdy8FRnzLB2ii_7KLn6WR7Hxl6bLhJEUOb5-ewhU2IJkgKbQ,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3pH5PkTQ$> > ) > > at > org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0( > ResourceMethodInvocationHandlerFactory.java:52 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fResourceMethodInvocationHandlerFactory.java*3a52&c=E,1,_ANSTEmLreJLEQz9RwIa3Hq0m-XWtYh1mmCLvHtKh_Nl94vWQgtz8kzCx6Su-0NUwC_E5f9R0_G34M1fcvyw_tbwt2BabAq1ByTCwEGQyOUk&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3Bot_Jpw$> > ) > > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$ > 1.run > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2f1.run&c=E,1,MZVqn6IzVn_wW73NZf9lIBKoDm6cMaU1WgNNYImFDQDDEY5Z0iFVnAvgt7x5mIfTkUF09OXzo9CFw-vPVsul-Izp5qW2Yrck4_6eujcuN-u_sllrUoywmJU,&typo=1&ancr_add=1__;JSUl!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0SmMpshA$> > (AbstractJavaResourceMethodDispatcher.java:146 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fAbstractJavaResourceMethodDispatcher.java*3a146&c=E,1,kizlN5vIvVLLgA2heqHvJlSyoVlUscXya5obSG66VCdZ-DUNMo10tEZwrh5x-1BykW4pb2xn-vEOY43NbjHHVG1y_FwY-MMYotiVd428HZlx9w,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt16EunjTg$> > ) > > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke( > AbstractJavaResourceMethodDispatcher.java:189 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fAbstractJavaResourceMethodDispatcher.java*3a189&c=E,1,u_xqzYFUz2Mx9lP8JBBlRUcq32BdYTzMnt7peBRr2FpOwDFlvp4liz3EBNUbZq1ZLE9BfvsVlloDJ5TIxFdivqlPSdCZc1dzcixUXMHvni4,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt20wlRH3Q$> > ) > > at > org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch( > JavaResourceMethodDispatcherProvider.java:219 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fJavaResourceMethodDispatcherProvider.java*3a219&c=E,1,W8GCe6oN-l2zCyZhE5onAlq0kIKg8WCIQAKI7Ert2GHO2-GAaKAyzEWTkO68IsniGdhLmuMMq47ojjdzidPYom1g5MXCCxZ6w8ZtJ29AxhR0Pw,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt01pruuyg$> > ) > > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch( > AbstractJavaResourceMethodDispatcher.java:93 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fAbstractJavaResourceMethodDispatcher.java*3a93&c=E,1,vnLQ__mSY8cdDiqo3_dWQgOEaspcJJhuBzEQ4DMznsTMKXqZP1lx2dcwqn2aSOQqSY8YEfuBsU4WYUXtAQVpglh1dQ9kz-Gn-SLO_r4D9zMt&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3IIxYpxg$> > ) > > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke( > ResourceMethodInvoker.java:478 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fResourceMethodInvoker.java*3a478&c=E,1,vtRvnVh3vxShrzucCuAx-7fWKPkzGkUiH9DjAnxttNhMJhqObrBBWGcDx9l5cfjoZoQ6vR2TNiDaql1bs5uP0jK4Qo4Y9VbjuILqGjYTKFNaZsJYT9pV9Z_FqA,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1zJfVL4g$> > ) > > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.apply( > ResourceMethodInvoker.java:400 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fResourceMethodInvoker.java*3a400&c=E,1,QbIG4TxOeShedV6j671Uv5vLnReJwXRCbSGagUyectQ5EHwxr5ovmRa6WhLDZWqioys1UYyT-20IpKMg6KQ2o1XXUkcy2oGZeW4p_KmXBBE,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2Z-QVu8g$> > ) > > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.apply( > ResourceMethodInvoker.java:81 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fResourceMethodInvoker.java*3a81&c=E,1,ZVyXYuf1u2G26rwo3XVG3Z8_2or4DcafcB3zlO2SdB6wb6R4w11zwbOHrLj_ZPmhrx3O6_AnSfWzWJBGi3JDQzxGByxzRTNjKqm71joHVWUdQXigItfn9Fpw&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2zc6H2DA$> > ) > > at org.glassfish.jersey.server.ServerRuntime$1.run > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2f1.run&c=E,1,5VdU5X2PgA-c6MijT2k3BoWYlOm_QZvwG-1u3s1bvib1G3MgAOipXud40Vx8MwfBxJLGeYU4FhLDJ59Nv51dBb3lxaCq0-DairoRJsZt&typo=1&ancr_add=1__;JSUl!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3uMp13Og$> > (ServerRuntime.java:261 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fServerRuntime.java*3a261&c=E,1,ngWN53pTjMa1KEncK0_jy2Mi8gO-yrsz4XmCGEgfdXTCBWl7AyOAIgrwcmQjj3uDCpIWPfc05kbIwQFDlzeALGQA2c6p1LRwHvO9XSw13_JOQ_1tpN1NMthFbg,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3Irprhkw$> > ) > > at org.glassfish.jersey.internal.Errors$1.call > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2f1.call&c=E,1,gpcxIIgn7_ZegqcuKnXoAFvZ31ppjxUZnNewxEWON2yyskEr-VLMK-VKPnUhIvj-jg6fw-FemOr_TjszVBXOK8O8Z9sysAKjrd58HGYpgRy8HhVUKKTiGziR358,&typo=1&ancr_add=1__;JSUl!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1BpwtCqA$> > (Errors.java:248 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fErrors.java*3a248&c=E,1,-4lNjwYENaRsriy3ayEWHiwSRHlsRk0A2JKqmAIjx8UYUguXhgukOJmIcAe7K-6Ug4fPj2aSGnE6qK8XTSl9214Wgyrnumxj1RQZLtMncSxqqpZkM9ANRg7g&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2FA6Qtiw$> > ) > > at org.glassfish.jersey.internal.Errors$1.call > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2f1.call&c=E,1,13JGq-_iEwBcwV5mPfYx3V7i_FkPJ4YflLbmNmIIFHY0c-VPpXwqgbZEC59xZM7gjk84I1s10Wo1esMdauVsqwj6k90skMDADwp_MdPvwNupS2jKtlQ9Lw,,&typo=1&ancr_add=1__;JSUl!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt34m4sJiw$> > (Errors.java:244 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fErrors.java*3a244&c=E,1,DeHOvMPeAsDyExSsCn1tzVYIUwljHqkTLT2fVzPpTOvt2eUNGvdIvGN0zuwdIKxOq0vcLo9gG7Q1rklzEELzIllI78LKvuVzlAWC2mda--WK7Bd5RgbBybvv1w,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0ZA_QU3w$> > ) > > at org.glassfish.jersey.internal.Errors.process( > Errors.java:292 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fErrors.java*3a292&c=E,1,-2TodtPTpE03ZGg_ww8ftKGn_pNdZxKFDN9YlA6us-U8mFuIRhb8cyqlQ46PSABDEdtcVCgQQ6p9DWK7goIr6ihIifN518cQeV2vLZuaQ14I&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2zpyDq2g$> > ) > > at org.glassfish.jersey.internal.Errors.process( > Errors.java:274 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fErrors.java*3a274&c=E,1,c1aDRBx-Kkhe46k62MbX3RYaKICs6JicYQPPJa24Veuptu1ic45DvtPjKO3vn_lsZcG6cBgBB-DB-mG24cbdHPoUqPGioK3RpW5rE9QWCjRdl8yLPVgPCHgMxA,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2N07l4cA$> > ) > > at org.glassfish.jersey.internal.Errors.process( > Errors.java:244 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fErrors.java*3a244&c=E,1,w1FDKjTa9Six6ZVu9g3QJYnGQuflyNbum7nYq3utYVnzfWFOiu1z4auY76BL3tgpquZCZRb3cWcZ2UWireyX03yjfuZVK1yq5_8_qi1GwLstFZa1Q0dcxQ,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2Hw2nqzg$> > ) > > at > org.glassfish.jersey.process.internal.RequestScope.runInScope( > RequestScope.java:265 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fRequestScope.java*3a265&c=E,1,TgExyJ7UXeTRjHTUHoKEFcexeWKopISAsK-5WCliWSvgVasuQhW_xB_ZKivdXmCAR9HosV8xJIfkTyvZaKzQoxbnPgNoKkQwAWteG-9-AuEf4w1h60U,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2QswbnQw$> > ) > > at org.glassfish.jersey.server.ServerRuntime.process( > ServerRuntime.java:240 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fServerRuntime.java*3a240&c=E,1,I-aQ9e78vqVvBnz_Ul46LjEl5AxbyOb5qBkDMavndcJMjZBTPPCbrCTBUt-UYdXaoOZ4u29XzqDfxy2eTamMsdzEfg44HfbufG4C3oM5kbFn78SgQA,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3Ju84Unw$> > ) > > at org.glassfish.jersey.server.ApplicationHandler.handle( > ApplicationHandler.java:697 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fApplicationHandler.java*3a697&c=E,1,Jsego7u0OUnIQqzGf5sprW5iJZCmr2L2KDCQOBs43_3NIuQXU9RPUgPHy-bG-YKQL7wVaDTLxoHhUOB58RkM-ybq2AWCr5aWqmR6B6Ob&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0E3JvDwg$> > ) > > at org.glassfish.jersey.servlet.WebComponent.serviceImpl( > WebComponent.java:394 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fWebComponent.java*3a394&c=E,1,lTSTlVXOoWmj6glMwmyv2CE_mfCD0zE1J8-2LyhxmJBQqBSViWGsyg_vaBj2Vrjv3_7zkGzv2VPu8UIfg98HtikJNOYTpW62dTRkVdbMnC7pvnA5-B3C6Db6YQ,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3wrzSdjg$> > ) > > at org.glassfish.jersey.servlet.WebComponent.service( > WebComponent.java:346 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fWebComponent.java*3a346&c=E,1,KOLlb6JskDrWev0dVrCArHDSpfP5fNCXtw_oCiOYEJlHIJfINTLNNeKhy-H1ArSiQ-0ENNaD_Qh0Geb8X7wqrqwfpqBwLtEm4G98wam8F7O3&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1NH6JNvQ$> > ) > > at org.glassfish.jersey.servlet.ServletContainer.service( > ServletContainer.java:357 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fServletContainer.java*3a357&c=E,1,xE8kQtKULT9CbuOJSHjcAq8nmkXVmrwW0pi6-5d4-ohnPEEJFxn_EinkKIPz_1VrgyvWUsHXNZFDtupRmAsrQqhTRXENL7kSa5LIjuqNojj2cwdWi2oRKE17U9l1&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1pV22FEg$> > ) > > at org.glassfish.jersey.servlet.ServletContainer.service( > ServletContainer.java:311 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fServletContainer.java*3a311&c=E,1,CGR56oCDqMxWtXmFVVxBImiJVtbbAnuo8i75aV-g9KkYVlBBWtMCnF1kTybr_6JDC2vBezhqfLbIBUgYve0GY6zk_eFI22NTS969ZV3KkquxqBkl_Lk547k,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1w8YLu2g$> > ) > > at org.glassfish.jersey.servlet.ServletContainer.service( > ServletContainer.java:205 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fServletContainer.java*3a205&c=E,1,RHUk7TgmAHI59I04g3HWyoryTWNqfXzhI5EEKYB6gOr3gsQ78kYA1UJwYXbIx_4c8K7uM2fG6K1GtP3ejLMG3qYpXpr8gyK3hl736QY7FLSbJ7on3G0s9_4DBQ,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3iKuJlPQ$> > ) > > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter( > ApplicationFilterChain.java:205 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fApplicationFilterChain.java*3a205&c=E,1,h2cMVon6xGXZ_SOU3zyFb-JtijRHlxgdyfCJHNtTXWCsZudhQv2hsGK9Gq6da_M2mu0HnRL_qoXqRZzFaeGKW1zw9MTKaY8hl3QmPlydzgE,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt00e4waYA$> > ) > > at > org.apache.catalina.core.ApplicationFilterChain.doFilter( > ApplicationFilterChain.java:149 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fApplicationFilterChain.java*3a149&c=E,1,19pKUon60z33K_WmMwfnNMVHc18KirhbBq6O1sGxhUvEzfCHQv-8kpmVCoTKdZjNpRBlG9i1L6hSsevfEW4g3oEQ2demPaHgOBz0STxPqH71RVjdOw,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2wFFxX-Q$> > ) > > at org.apache.catalina.core.StandardWrapperValve.invoke( > StandardWrapperValve.java:167 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fStandardWrapperValve.java*3a167&c=E,1,iurW6Vg5HqcJe6goPvo4sb_OPJZ1ug8vrXm4M5nm_3WfLnrKdoR6AzqfMjjhLJEBGN1YTuVSr2KGjz7aPIdHNMWq5LroYkYwN_9ERWrPAw0qj-0qjeY,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2UbL4H-g$> > ) > > at org.apache.catalina.core.StandardContextValve.invoke( > StandardContextValve.java:90 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fStandardContextValve.java*3a90&c=E,1,y9KIkhbbvuOVtZ1BMVlp8LM8grRux9d-SB3Cx1ysNOkTTGAzWyKiaqYmfjtiyJ81e7xkDEvfQB5PBjpwh9D-qsgkZbLVC4OA7kBl9XhQ5D-wcaB8OGarKkgSpgs,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2aG4yB2A$> > ) > > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke( > AuthenticatorBase.java:482 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fAuthenticatorBase.java*3a482&c=E,1,870FeHRgrrt8FiPhpmG_J2QY0UhrzX4l2obWvn7GSTBkSokbpyaEh7DA5wlIFPxx8U8UucJ5jxsveJBiGv73ZL23tRaxKmL0sqlnVeg93LTGYv0,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3ZiI4UDg$> > ) > > at org.apache.catalina.core.StandardHostValve.invoke( > StandardHostValve.java:115 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fStandardHostValve.java*3a115&c=E,1,gtlhUbg7fRCSWrxRdn-xvp5rZiVVL14r6bmxhD8tbUZeO2HPLW42mnvDUaqWB165vffTw4pqzZBrvfcsJ0iwuZ0uL2tGE34M50Vn9JteanRmGYMeFJ4szU0g&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt06UD8C6Q$> > ) > > at org.apache.catalina.valves.ErrorReportValve.invoke( > ErrorReportValve.java:93 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fErrorReportValve.java*3a93&c=E,1,8fP04ef08snHBtrmso4oiWrZWAQg8unZ-PhOZT6v_YO0Udv7Sno4UnmuKPE-yPpc-WPoAJoSYfC_Kep5_Sf1xeu9i9jJAiyeGn-90SXaPOdE6PJa3eejOnM,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0_U4I5Lg$> > ) > > at org.apache.catalina.core.StandardEngineValve.invoke( > StandardEngineValve.java:74 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fStandardEngineValve.java*3a74&c=E,1,5XJYADhwE12w-qyHEEVFAnAJ_Fge20HRlvGrtNOUAjzT4UhVYhPeh7qsZCiR11GlnfnX7mxaIgwiug6752Gk5JURAIU1PgK6rk8vqsfvxM-7mbRJ&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2uszZYIA$> > ) > > at org.apache.catalina.connector.CoyoteAdapter.service( > CoyoteAdapter.java:340 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fCoyoteAdapter.java*3a340&c=E,1,B5JOWKWHEb6cVQgZlyltFnVbK4WhfVdrczK7jov9-MUlkwYC0zpCzTBNnohFstlGatoq7jDST-m-NLHFTHYVTuCNbqPIrZ8ufFHamINtG3N05p9UlkMwd5oEwSc,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1Yco2SdQ$> > ) > > at org.apache.coyote.http11.Http11Processor.service( > Http11Processor.java:391 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fHttp11Processor.java*3a391&c=E,1,iDXL8VhEKOyTibuCsANH061qEr0KkrGbiIOGzj_8fzqISA7HsdtnrEMfo09pTcpyQepntY3cgZKYSXPSK2eh3GO99UHkbTF4T7L5Z3IXl0NAyVjPSjm1ew,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2tgXW6Xg$> > ) > > at org.apache.coyote.AbstractProcessorLight.process( > AbstractProcessorLight.java:63 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fAbstractProcessorLight.java*3a63&c=E,1,A-vRfgPyPTwRwif1XCFkrF1_a4zD_DegwPnekB4SOkm04OpYSrniO6S-gm-ixgSAkSgUkZja9NWO2MYTIRlh6eLiLl2YKnrecwml7gwidbw2WF7bOGQ0Vw,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3EWG8C9w$> > ) > > at > org.apache.coyote.AbstractProtocol$ConnectionHandler.process( > AbstractProtocol.java:896 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fAbstractProtocol.java*3a896&c=E,1,pDFoRtt82LZU-FLdcBDdk-qlTk-Kt6gnRQpSQzYU8dEYEOGLmK01S-_w3L8iQgukNx49f3AUgrw9O1VhBVlRfoH4PNsuKkvr7sPvOf1bPaE,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1FxxcYyw$> > ) > > at > org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun( > NioEndpoint.java:1744 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fNioEndpoint.java*3a1744&c=E,1,-7FPn1c3059u04P9uwPVOkko51iX8BMN0E_rPpTPXwhN9hkjEjkHEsthD60wkwjFhcfxkwdzFjlIvshjkvlpopDv_JyVZj8MMCChGpV_jfnL&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2S7ef9Mg$> > ) > > at org.apache.tomcat.util.net.SocketProcessorBase.run > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2forg.apache.tomcat.util.net.SocketProcessorBase.run&c=E,1,9zXaF9vHvKVlE5NpCYjiHmbZrmqbG0B_GHQq4rSrzpeeXFCwFUs3MBzaE0j74-TGhcnMER1vqZG2FZFG0laisCtMlBX1R9nEaId_KoNNZqaYDU99jw,,&typo=1&ancr_add=1__;JSUl!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt37ct6vGA$> > (SocketProcessorBase.java:52 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fSocketProcessorBase.java*3a52&c=E,1,LZDkH_krRzHgwiDBEmnF4kCwabJiZvRp890kF2HGgehkBrUkK0u9dIj5At5lRRQzte2_ZWRYj5BFyVm-MOHGtHsq6PLN6c4vGPVzxDxCNw,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt39S_Es0w$> > ) > > at > org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1191 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fThreadPoolExecutor.java*3a1191&c=E,1,sFhkQGGY9T9drnsrqqTCqhYgt4P6sXplDPrKuhStXuq91fnWPdOAqtYJjsHH6ElilKWv63vhvYti2xx4zSQ6lUgtYswUBxUXuOC_L_roshPdeua_KBE,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0UC_pVvQ$> > ) > > at org.apache.tomcat.util.threads.ThreadPoolExecutor$ > Worker.run > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fWorker.run&c=E,1,AAJck1rx7huIcGTcZLVifmrqKKqn9tJ8_EiWLY_LgGGq07_vbhVv_TGyx999HyAkXdEQyZ2inpt_xkHU46VsugUd4KNB_lCB-NsAEnOJ9Q,,&typo=1&ancr_add=1__;JSUl!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0OfUx4Kw$> > (ThreadPoolExecutor.java:659 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fThreadPoolExecutor.java*3a659&c=E,1,HYWoCOj08fcNggtMqwtg8befDJ3OvA2zAifg0xQdLJm5e-OnXnaKI5_SCii3WvU3-9HCXwaChGG1bTuavL0vL7KEXHx7D7Xe9-vkD7UfIw,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1hFmOFBA$> > ) > > at org.apache.tomcat.util.threads.TaskThread$ > WrappingRunnable.run > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fWrappingRunnable.run&c=E,1,-v5NUrxqHwzGg-9dAW4Q5AoU0tfAeo30H8Ky80--szahAOLdpHSZRDXUFiK7lQ3Bf9c5gNtj9N1EPXf5bIt1IkQnBtFxU8AfBBf11UhBIsEZ6D10CQsc3DRj&typo=1&ancr_add=1__;JSUl!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3COcN9jw$> > (TaskThread.java:61 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fTaskThread.java*3a61&c=E,1,uRfcR0WsV0fQqEaa3HYCcLRIMMemHgiaz-A36uGG4a4HY2hQpshoi77gZxVZSANCQkQR10KWG0F8foV7t4Ae3MPXGUCZBhnuwUFqJQ0SvS6KEw,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0OboHauQ$> > ) > > at java.base/java.lang.Thread.run > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fjava.lang.Thread.run&c=E,1,6mclfOVKxSEC_VTgnVOWXrNwCcwYCqWYCtOBwdKeBT-A6z_62O805-qznvQOW6gHIZG-OKE3p1uXcX6TXJihVOENpw0vbkAd0DvI1_9GU_E,&typo=1&ancr_add=1__;JSUl!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0_EVu8fg$> > (Thread.java:833 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fThread.java*3a833&c=E,1,Xjdr2WgvR9se4-q8Y6ReY1YymidfVI1RqBfB-swMy0sAbyu3LQ3Gj7Oy8oDVv_49jgELRy8Q7kE1ZDlAmd26Yxr2iOdrJ6pPVb2qKYi2NQLH&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0gwDS_4w$> > ) > > Caused by: > ActiveMQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT > message=AMQ219014: Timed out after waiting 10000 ms for response when > sending packet 71] > > ... 65 more > > > > > > This is the stack trace at the second timeout message > > at > org.apache.activemq.artemis.jms.client.ActiveMQConnection$JMSFailureListener.connectionFailed( > ActiveMQConnection.java:714 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fActiveMQConnection.java*3a714&c=E,1,UTaiF_0hpU4wTQqz4_3iStrmfLszIHizjPpueBR5yv-5ilk0tbhwZ6KgeYpKKCaTEGJZLlNJziUbmKJSjQlA6nTOxihhnoWGSNbToAQ6zOSqEw,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1KndddZQ$> > ) > > at > org.apache.activemq.artemis.jms.client.ActiveMQConnection$JMSFailureListener.connectionFailed( > ActiveMQConnection.java:735 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fActiveMQConnection.java*3a735&c=E,1,M-b7JHMmR30vR5cCmk3NcNzfkC3m4MBhI7tJ_VStAzCaYBWqPVD1pyvEx9LD-I1LVDZjbHVeBfUMf5HOCEhANjQfJ84JNVn9pXV2_zh-kizyhMD05Yo,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0X4203Dw$> > ) > > at > org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.callSessionFailureListeners( > ClientSessionFactoryImpl.java:868 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fClientSessionFactoryImpl.java*3a868&c=E,1,htiR1-PgM2OgWpPbflNwb3TOVGuHjNJ46r1zIi_tR55EfzAJFETCEUZrDBv8nvJKD7vDL5puGHA7emmSLPMO_-0M1bkJoUpDZE6Y12NlqAbLdD30blmH1A,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt06tX_5VA$> > ) > > at > org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.callSessionFailureListeners( > ClientSessionFactoryImpl.java:856 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fClientSessionFactoryImpl.java*3a856&c=E,1,rG1HeinOiFfS-vAJeiDrP_tJ46W3ZwKfb5vP3JU86p789aoYHLT9_cltu1F556YuVfjAXcVJSdMX0vLZwS5Am1c9OjgzGUgiix9431pSXnGAFHg,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3yM95SKg$> > ) > > at > org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect( > ClientSessionFactoryImpl.java:802 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fClientSessionFactoryImpl.java*3a802&c=E,1,IpmoqWzjRsBWsWFRVyophC7xvobUYm_M-R07j2weNl0fMuXLc1F29q9TuV8AjEY2mxpBZJG12kK5SBibaGPeCrwCi0b0aUt56930OdwTuciH_pWkhR9SLw,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3q-oPNhg$> > ) > > at > org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure( > ClientSessionFactoryImpl.java:566 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fClientSessionFactoryImpl.java*3a566&c=E,1,jMFCSfeRebikH-AioEUI7cUUntw6-yuiUplnWJKFhMj_w-XlKte9e8HGTG5EbkHvQOJb6f__nDkxR20RBBxNTJx3swi3wt0dTLBbQ9Gc7YyL36YKbxoXhw,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2RA_btVg$> > ) > > at > org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingFailureListener.connectionFailed( > ClientSessionFactoryImpl.java:1407 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fClientSessionFactoryImpl.java*3a1407&c=E,1,SgODhYYd-iTgFlhmTmEi87435ahyCVF06rZr29wB4ZmpZT3fVrzB9y4Ukxyt3Qp4spxbR0RlCgSK6IE6eJ9EXZ3psySz49YvNJ1DF88roNqzWGPo8yzx&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0JG-v9YQ$> > ) > > at > org.apache.activemq.artemis.spi.core.protocol.AbstractRemotingConnection.callFailureListeners( > AbstractRemotingConnection.java:98 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fAbstractRemotingConnection.java*3a98&c=E,1,WDcthbTMBqHSmI-aOCsqLih30H8yDiQr_PVd_7nskCH4_kGGOYaGF9nA4AG-8L2m_jDAlXCYT933o1FU8neMfkT9g3kehZBPKrJ8ZpXgURBaYUqcOFCnWMpEQA,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1-IzgpLQ$> > ) > > at > org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2forg.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.fail&c=E,1,0pYKjuOHMOTL3UaLqDP6EqlqIjMTA8Xs2Y5pF6MJBTUbxDdQxIfbPyCavvMtGGypNVBbRbQ8-zZIrx7GQgfC3Q44sIhPeSqAk5BKHMNnPvxngsHc9VM,&typo=1&ancr_add=1__;JSUl!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1ykQsa-Q$> > (RemotingConnectionImpl.java:212 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fRemotingConnectionImpl.java*3a212&c=E,1,6CSm8GgMGGprYxpYP6IyjwroEMKSTd3ydQtGEw04aqh6Vdzvg4I112zsVdAt5yslOpmIMPxULflUZ7OL0Kim3JxwroK-Mk2vgWjNssTKwQ8,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2WbUq6qw$> > ) > > at > org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$ > CloseRunnable.run > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fCloseRunnable.run&c=E,1,eZvshsv6mememDd-0R4ghtzmYVOLSwz0A4FoHwzjGidDTToNBDzZezVfUnRD6-Fr6ZpJ7xgwcwgEUTXDxNytrkZYn7xVe_CCXLYp_k_endKN4c003YNrwdo,&typo=1&ancr_add=1__;JSUl!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt1rQHaGMg$> > (ClientSessionFactoryImpl.java:1172 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fClientSessionFactoryImpl.java*3a1172&c=E,1,E-urG5zq6gIV_DxAli9FjMYUOObRAtZL6qOaFUOOlJR2DFhySGzl0Ezt0enYkygsres1LWIiBlimOgEWcRQLo36pWLNBBOeWfcicG6Mj6XPUQg,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt075-XzRg$> > ) > > at > org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask( > OrderedExecutor.java:57 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fOrderedExecutor.java*3a57&c=E,1,9bznxhGY4EfYRHk3swYAryZ9Zv7cr16Ixkl_UjLyd_Rl0kFg3SXDBp0wvdSLtJCAdA5Cz3TwzASQ6lUgVrWM9Wtzwyg98MkPPCXX0xokHw,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3j1smHOQ$> > ) > > at > org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask( > OrderedExecutor.java:32 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fOrderedExecutor.java*3a32&c=E,1,t1lXLP_aqiYrMVYyb-bMDpM4mLF7TLWLbeerRhHX63W6TLf1jYUGt25oITkxOoxm4ZoQ3I2GZfQAcFmpZaR3AoFfQ8GxB-rpqID6_0FeePLZAysHVL5S&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3Gip9gYA$> > ) > > at > org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks( > ProcessorBase.java:68 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fProcessorBase.java*3a68&c=E,1,dqTUe0Hw6248go_69PKbcLIlvBQTpHuV3CcCFsk-FiPCM1O48nXbUV_guNPd_jI_IVZQt9VSlZUqjyeDy_6y-re8MDJqbOzruaHVVHU1-ApA5aWpoTmvgnEJo8A,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt3MXWDHKA$> > ) > > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1136 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fThreadPoolExecutor.java*3a1136&c=E,1,2nYUC5WAfJfnAGhEGid8vIIWld0ejYgY8lIQg25Q7UeeR6HyQEUIMvx5AdKZx7iRMN_Jbcqk6Z1y6DUYTgq1Ibe2ohbtuLOuknrK9Q2N&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2WsExbgQ$> > ) > > at java.base/java.util.concurrent.ThreadPoolExecutor$ > Worker.run > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fWorker.run&c=E,1,yYLSxR4Z6kEEBJHWE_U5Y9Yj1yJ5gCTjWnJG93-QTmhagOXIH88mEsnpEaGf68M9tjwmL6Qs2JkjN8ev6sHlkorNqTfDRuDd19bHUwXbIA,,&typo=1&ancr_add=1__;JSUl!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0sCtaj5A$> > (ThreadPoolExecutor.java:635 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fThreadPoolExecutor.java*3a635&c=E,1,2gx_LRp6l1TeZbPuvvGze5ybAfvTxvVesnh004mjFKDVBi_9Ef01UvfMvzF5H0rBG6jvG9JYYuENOCyvYC1mCYQZcTWUVRQS79tVx2dqhA,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2stqsCHw$> > ) > > at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$ > 1.run > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2f1.run&c=E,1,MYVb_RE5tczV-l7jvqgk1CydEYEAoiVV15yENbdhr6da1wfFTwF2_k6H9SruEgixQxL3QD--ryo5iePNIQ7Ve5LWsaWtoIlJ4w3dFSCQuomtJQ,,&typo=1&ancr_add=1__;JSUl!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt2NMmQF-w$> > (ActiveMQThreadFactory.java:118 > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fActiveMQThreadFactory.java*3a118&c=E,1,vb13BY-ZO2U4XSS8_k_oQ8L62v_m40T1EZTt5i_ls8XG5HX-X1d04kNXs9AhVlI65lduhnB5yAjB4aYNPPsm3Sda_IjNqgkbYkoUblIYcHB6j2kQuQ,,&typo=1&ancr_add=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt0fcfagXg$> > ) > > Caused by: ActiveMQDisconnectedException[errorType=DISCONNECTED > message=AMQ219015: The connection was disconnected because of server > shutdown] > > ... 7 more > > > > > > > > [image: rg] > <https://urldefense.com/v3/__https://linkprotect.cudasvc.com/url?a=https*3a*2f*2fwww.redpointglobal.com*2f&c=E,1,-sALdzxL74lBcP9YXBclI84IpjFZ6fxlUvA4KGryH_bBqlIZMivu2xrDDFqAM0Oyqr6m7m1ahJp7KQcXuo4ph4YG2nUn_O66R2CmaR3YseKA9dE3QA,,&typo=1__;JSUlJQ!!Ck4v2Rc!nzN3whXfPpxIweqDhJzenIYN05MWZUfHlopo7TBhpb6m8p-5EP7pvzzQD-SMZfQwgBJ_y4NGWq3NEStRIDMAEkW0wjT3tt23XDGlag$> > > *John Lilley * > > *Data Management Chief Architect, Redpoint Global Inc. * > > 34 Washington Street, Suite 205 Wellesley Hills, MA 02481 > > *M: *+1 7209385761 <+1%207209385761> | [email protected] > > > PLEASE NOTE: This e-mail from Redpoint Global Inc. (“Redpoint”) is > confidential and is intended solely for the use of the individual(s) to > whom it is addressed. If you believe you received this e-mail in error, > please notify the sender immediately, delete the e-mail from your computer > and do not copy, print or disclose it to anyone else. If you properly > received this e-mail as a customer, partner or vendor of Redpoint, you > should maintain its contents in confidence subject to the terms and > conditions of your agreement(s) with Redpoint. > > PLEASE NOTE: This e-mail from Redpoint Global Inc. (“Redpoint”) is > confidential and is intended solely for the use of the individual(s) to > whom it is addressed. If you believe you received this e-mail in error, > please notify the sender immediately, delete the e-mail from your computer > and do not copy, print or disclose it to anyone else. If you properly > received this e-mail as a customer, partner or vendor of Redpoint, you > should maintain its contents in confidence subject to the terms and > conditions of your agreement(s) with Redpoint. >
