Unfortunately the thread dump didn't provide any meaningful information. Did you acquire it during the 60-second window while the client was attempting to reconnect to the broker? There were essentially no active threads in the thread dump.
I'd recommend a few adjustments to your connection settings. First, it's recommended that clientFailureCheckPeriod be significantly lower than connectionTTL [1] (e.g. half). Second, I'd recommend increasing connectionTTL to be more than callTimeout. Lastly, I recommend you make all your connection-related settings in the same place rather than split between code & configuration. Putting all this together, you might try something like this: (tcp://dm-activemq-live-svc:61616,tcp://dm-activemq-backup-svc:61617)?ha=true&reconnectAttempts=200&initialConnectAttempts=200&clientFailureCheckPeriod=7500&connectionTTL=15000&callTimeout=10000&retryInterval=1000&retryIntervalMultiplier=1.5&maxRetryInterval=4000 Justin [1] https://activemq.apache.org/components/artemis/documentation/latest/connection-ttl.html#detecting-failure-from-the-client On Wed, Nov 20, 2024 at 4:31 PM John Lilley <john.lil...@redpointglobal.com> wrote: > Justin, > > Thanks again for taking a look at this,. > > I've attached the thread dump, as well as our broker configuration file. > > >> What logging do you see that indicates the packet was sent? > > We have a "client" side and a "server" side of the app. Each packet has a > unique ID, and our "server" logs the request when it is received, which is > right about when the client reports the warning at the 1-second mark.. > > >> Do you have any idea what's happening during this 60-second window? > Have you, for example, gathered any thread dumps from the client during > this time to see exactly what it's doing? If so, what did they show? If > not, could you? > > I attached the thread dump for our "client", which is the thing seeing the > timeout issues. Unfortunately I am not able to make much of it, not > knowing Artemis driver internals and there are a LOT of artemis threads. > > >> That said, it would be worth knowing *all* the connection-related > parameters you're setting on the client. Can you provide that info? > > These are the only things being set on the connection factory: > connectionFactory.setRetryInterval(1000); > connectionFactory.setRetryIntervalMultiplier(1.5); > connectionFactory.setMaxRetryInterval(4000); > > But we also set these on the client connection URL > clientFailureCheckPeriod=10000&connectionTTL=10000&callTimeout=10000 > > The overall connection URL is > > (tcp://dm-activemq-live-svc:61616,tcp://dm-activemq-backup-svc:61617)?ha=true&reconnectAttempts=200&initialConnectAttempts=200&clientFailureCheckPeriod=10000&connectionTTL=10000&callTimeout=10000 > > Thanks > john > PLEASE NOTE: This e-mail from Redpoint Global Inc. (“Redpoint”) is > confidential and is intended solely for the use of the individual(s) to > whom it is addressed. If you believe you received this e-mail in error, > please notify the sender immediately, delete the e-mail from your computer > and do not copy, print or disclose it to anyone else. If you properly > received this e-mail as a customer, partner or vendor of Redpoint, you > should maintain its contents in confidence subject to the terms and > conditions of your agreement(s) with Redpoint. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@activemq.apache.org > For additional commands, e-mail: users-h...@activemq.apache.org > For further information, visit: https://activemq.apache.org/contact >
