Hi Eric, you are getting this issue because the CORE clients uses topology for load balancing by default, you can solve this issue disabling topology for load balancing, i.e.
(tcp://server1.example.com:61616,tcp://server2.example.com:61616 )?sslEnabled=true&trustStorePath=/path/to/truststore.jks&trustStorePassword=foobar&useTopologyForLoadBalancing=false Regards, Domenico On Thu, 15 Feb 2024 at 15:43, <el...@melnib.one> wrote: > Hi folks, > > Trying to debug a tricky problem, and I'm beginning to suspect it's a > bug in the ActiveMQ server and/or client. The client sometimes ends up > using a wrong truststore, in spite of seemingly correct configuration. > > Server: 2.31.0 > Client: 2.38 (via Spring Boot) > > The client is connecting to the server with the following connection > string: > > (tcp://server1.example.com:61616,tcp://server1.example.com:61616 > )?sslEnabled=true&trustStorePath=/path/to/truststore.jks&trustStorePassword=foobar > > The client manages to connect to Server 1 and is able to consume and > produce messages, this is fine. > > However, the server seems to broadcast a topology update > (ClusterTopologyChangeMessage_V4) containing the following: > > > ?port=61616&keyStorePassword=****&sslEnabled=true&host=server1-example-com&keyStorePath=/path/to/broker-ks&verifyHostName=false > and > > ?port=61616&keyStorePassword=****&sslEnabled=true&host=server2-example-com&keyStorePath=/path/to/broker-ks&verifyHostName=false > > The client then seems to attempt to connect to Server 2 using the path > to the keystore provided in the ClusterTopologyChangeMessage, instead of > the the path I configured in the client connection string ??? > > I don't understand why this is happening. In fact, I don't even > understand why the server would broadcast the path to its own keystore > (and apparently also its password?) to clients. This information is > useless to the client, and it makes no sense for the client to attempt > to use this nonexistent path as a truststore. > > We're using Spring Boot, so I'm not 100% certain that the problem isn't > with Spring, but as far as I can tell the correct connection string is > passed to a connection factory, and then somehow it ends up using a > wrong path to a truststore anyway. > > The end result is that while things seem to work fine, the log is full > of errors (java.lang.Exception: Failed to find a store at > /path/to/broker.ks). Full stack trace at the bottom of the message. > Presumably things will stop working when we attempt to fail over to > Server 2. > > Could someone be so kind as to point me in the right direction for > getting to the bottom of this? > > Thanks, > > Elric > > > Full stacktrace: > > java.lang.Exception: Failed to find a store at /path/to/broker.ks > at > org.apache.activemq.artemis.core.remoting.impl.ssl.SSLSupport.validateStoreURL(SSLSupport.java:399) > > ~[artemis-core-client-2.28.0.jar!/:2.28.0] > at > org.apache.activemq.artemis.core.remoting.impl.ssl.SSLSupport.loadKeystore(SSLSupport.java:339) > > ~[artemis-core-client-2.28.0.jar!/:2.28.0] > at > org.apache.activemq.artemis.core.remoting.impl.ssl.SSLSupport.loadKeyManagerFactory(SSLSupport.java:375) > > ~[artemis-core-client-2.28.0.jar!/:2.28.0] > at > org.apache.activemq.artemis.core.remoting.impl.ssl.SSLSupport.loadKeyManagers(SSLSupport.java:355) > > ~[artemis-core-client-2.28.0.jar!/:2.28.0] > at > org.apache.activemq.artemis.core.remoting.impl.ssl.SSLSupport.createContext(SSLSupport.java:222) > > ~[artemis-core-client-2.28.0.jar!/:2.28.0] > at > org.apache.activemq.artemis.core.remoting.impl.ssl.DefaultSSLContextFactory.getSSLContext(DefaultSSLContextFactory.java:50) > > ~[artemis-core-client-2.28.0.jar!/:2.28.0] > at > org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector.loadJdkSslEngine(NettyConnector.java:786) > > ~[artemis-core-client-2.28.0.jar!/:2.28.0] > at > org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnector$1.initChannel(NettyConnector.java:690) > > ~[artemis-core-client-2.28.0.jar!/:2.28.0] > at > io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129) > > ~[netty-transport-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112) > > ~[netty-transport-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:1114) > > ~[netty-transport-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:609) > > ~[netty-transport-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.channel.DefaultChannelPipeline.access$100(DefaultChannelPipeline.java:46) > > ~[netty-transport-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.channel.DefaultChannelPipeline$PendingHandlerAddedTask.execute(DefaultChannelPipeline.java:1463) > > ~[netty-transport-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.channel.DefaultChannelPipeline.callHandlerAddedForAllHandlers(DefaultChannelPipeline.java:1115) > > ~[netty-transport-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.channel.DefaultChannelPipeline.invokeHandlerAddedIfNeeded(DefaultChannelPipeline.java:650) > > ~[netty-transport-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:514) > > ~[netty-transport-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429) > > ~[netty-transport-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486) > > ~[netty-transport-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) > > ~[netty-common-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) > > ~[netty-common-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) > > ~[netty-common-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:416) > ~[netty-transport-classes-epoll-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) > > ~[netty-common-4.1.97.Final.jar!/:4.1.97.Final] > at > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > ~[netty-common-4.1.97.Final.jar!/:4.1.97.Final] > at > org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) > > ~[artemis-commons-2.28.0.jar!/:na] > > > > > >
