Hi Justin, I appreciate you taking the time.
We've been hitting this issue in various forms for a while. There appear to be two conditions in which we see this: - The active member of our HA broker pair is moved by Kubernetes and fails over to the backup - The connection between our AMQ client and the broker is interrupted. When either of these happen, we first see a 10-second delay before the client emits this error: AMQ219014: Timed out after waiting 10000 ms for response when sending packet After ten seconds, the packet appears to be sent, but we log its delivery in our service. But then a full minute passes before the original (sending) client resumes communication with the broker. We want to shorten this one-minute reconnect interval, ideally to < 30 seconds. What combination of settings will do this? Thanks john -----Original Message----- From: Justin Bertram <jbert...@apache.org> Sent: Tuesday, November 19, 2024 2:52 PM To: users@activemq.apache.org Subject: Re: Warning around reconnect attempt *** [Caution] This email is from an external source. Please use caution responding, opening attachments or clicking embedded links. *** > 1- Should we be setting connectionFactory.setConfirmationWindowSize()? The docs for confirmationWindowSize are here [1]. Whether you should set this parameter is really up to you and the behavior you want for your application. > 2- Are the retry settings reasonable? "Reasonable" is hard to judge in a support context like this. I'm not sure we can really judge. > Ultimately we just want to reconnect in under a minute and avoid our timeout. It might be worth investigating why you got the timeout in the first place. Do you have any data from the broker or the network that might suggest what happened? > Or, maybe we just need to set our timeout longer? Without knowing the root cause of the timeout it's impossible to say. Justin [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__activemq.apache.org_components_artemis_documentation_latest_client-2Dfailover.html-23reconnect-2Dto-2Dthe-2Dsame-2Dserver&d=DwIFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=VpU5fPwEQ8wKbKY5SBkV1DYypS_WKWolD9Eue6u6sYo&m=Mist2dGO6lZfrTPPt2xcN8QXDIwlWhN4ks51T71JNe_2PLfyqLmfUjjOIR6C1_Gb&s=YmllMfJSJOseAoxT8ojTHhtLr2Q3cb3zuZD7KZ2rgAM&e= On Tue, Nov 19, 2024 at 3:30 PM John Lilley <https://urldefense.proofpoint.com/v2/url?u=http-3A__john.lilley-40redpointglobal.com&d=DwIFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=VpU5fPwEQ8wKbKY5SBkV1DYypS_WKWolD9Eue6u6sYo&m=Mist2dGO6lZfrTPPt2xcN8QXDIwlWhN4ks51T71JNe_2PLfyqLmfUjjOIR6C1_Gb&s=CovnqP20ifJlvniztfu3b6uty4AI-pRicMky6zqEi80&e=> wrote: > 2024-11-19 14:19:51,426 WARN > [org.apache.activemq.artemis.core.server] > AMQ222066: Reattach request from 10.147.133.72:53082 failed as there > is no confirmationWindowSize configured, which may be ok for your > system > > This was issued by the broker at the same time that our JMS client > issued this warning: > > 2024-11-19T14:19:51.417 [Thread-2 (ActiveMQ-client-global-threads)] > .client.connectionFailureDetected:293 [] WARN - AMQ212037: Connection > failure to dm-activemq-live-svc/10.0.219.91:61616 has been detected: > AMQ219014: Timed out after waiting 10000 ms for response when sending > packet 71 [code=CONNECTION_TIMEDOUT] > 2024-11-19T14:19:51.414 [main] .JmsProducerPool.send_:389 > [y6y5v5wm1s0x] WARN - Error sending message, will retry > jakarta.jms.JMSException: > AMQ219014: Timed out after waiting 10000 ms for response when sending > packet 71 at > org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBl > ocking(ChannelImpl.java:550) > at > org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBl > ocking(ChannelImpl.java:452) > at > org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionCon > text.sendFullMessage(ActiveMQSessionContext.java:588) > at > org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.sendRe > gularMessage(ClientProducerImpl.java:305) > at > org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.doSend > (ClientProducerImpl.java:277) > at > org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.send(C > lientProducerImpl.java:147) > at > org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.send(C > lientProducerImpl.java:129) > at > org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.doSendx > (ActiveMQMessageProducer.java:483) > at > org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.send(Ac > tiveMQMessageProducer.java:221) > at > net.redpoint.rpdm.ipc.jms.JmsProducerPool.send_(JmsProducerPool.java:3 > 85) > at > net.redpoint.rpdm.ipc.jms.JmsProducerPool.sendRequest(JmsProducerPool. > java:300) > at > net.redpoint.rpdm.ipc.jms.JmsRpcClientChannel.sendRequest(JmsRpcClient > Channel.java:230) > at > net.redpoint.rpdm.ipc.jms.JmsRpcClientChannel.invokeRaw(JmsRpcClientCh > annel.java:203) > Caused by: > ActiveMQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT > message=AMQ219014: Timed out after waiting 10000 ms for response when > sending packet 71] ... 13 more > > After 72 seconds, we hit our own 60-second timeout and fail. > > In our connection factory, we are setting > > connectionFactory.setRetryInterval(1000); > connectionFactory.setRetryIntervalMultiplier(1.5); > connectionFactory.setMaxRetryInterval(4000); > > > So my questions are: > 1- Should we be setting connectionFactory.setConfirmationWindowSize()? > 2- Are the retry settings reasonable? > > Ultimately we just want to reconnect in under a minute and avoid our > timeout. Or, maybe we just need to set our timeout longer? > > Our maven dependency for the JMS driver: > <dependency> > <groupId>org.apache.activemq</groupId> > <artifactId>artemis-jakarta-client</artifactId> > <version>2.32.0</version> > </dependency> > > Please advise. > Thanks > John > > PLEASE NOTE: This e-mail from Redpoint Global Inc. ("Redpoint") is > confidential and is intended solely for the use of the individual(s) > to whom it is addressed. If you believe you received this e-mail in > error, please notify the sender immediately, delete the e-mail from > your computer and do not copy, print or disclose it to anyone else. If > you properly received this e-mail as a customer, partner or vendor of > Redpoint, you should maintain its contents in confidence subject to > the terms and conditions of your agreement(s) with Redpoint. > ---------- This email has been scanned for spam and viruses by Proofpoint Essentials. Visit the following link to report this email as spam: https://us5.proofpointessentials.com/app/report_spam.php?mod_id=11&mod_option=logitem&report=1&type=easyspam&k=k1&payload=53616c7465645f5fc93a62ffd2f5b082f83c0a4b514b745939f337fbf3a27d426c9d84d149c1c8af15847b3f1403112a84fa6f63ecbe87688a9fdca2e37c7058e22d7eff721005809f48808015c98a65b7f60c124567c12f1f08b8c2e2daace4fc243cf762f0379e29d82d63113e53443993ab7ead863ca1bbe02d58a01b07e01548c0b87bc0888ecd7937a1df8e2d9efb2c4538ffe54c899ba63134a975b990
