I don’t have a ASF Jira account. I am happy to request one to open a Jira ticket for this.
Paul From: Justin Bertram <[email protected]> Date: Tuesday, October 28, 2025 at 2:20 PM To: [email protected] <[email protected]> Subject: Re: MQTT Last Will not sent because denied authentication I tend to agree with Matt here, but I think perhaps a feature could be added to Artemis so that LWT are authorized on connect to avoid this kind of problem. This behavior would be off by default so as not to impact existing users. Do you have a Jira account? If so, would you mind opening an issue [1] for this? Justin [1] https://urldefense.com/v3/__https://issues.apache.org/jira/browse/ARTEMIS__;!!NpxR!hdQbIYBgb_bBdoWH7TVo30l1yIKlErqg_yZOBlMZLt8SWQyPYex2pjuUSytWaI1zd5pSClzgpi0oxTF0wA$ On Tue, Oct 28, 2025 at 1:58 PM Matt Pavlovich <[email protected]> wrote: > Hi Paul- > > The usage of JWT and LWT are competing features, since JWT expires and LWT > is intended to alert for unplanned disconnect of long-running connections. > A possible solution is to perform periodic close-open of the connections > and re-register the LWT send would always fall within the timeframe of the > JWT expiry. > > Matt Pavlovich > > > On Oct 28, 2025, at 11:36 AM, Shields, Paul <[email protected]> > wrote: > > > > Hi Justin, > > > > I am struggling with this statement > > However, authorization for actually > > sending the will message is not performed at this point. Authorization is > > only performed when the LWT message is actually sent (e.g. when the > > client's connection fails). All sending operations are authorized at the > > time they occur whether that's via the normal PUBLISH packet from the > > client or for a LWT message sent on the client's behalf by the broker. > > If we were using a user name and password (even if that was stored in > LDAP) that would be ok. But we are using the machine ID for the user name > and a JWT is used in the password field of the MQTT connect. My problem is > that the JWTs for authentication have a short expiration time (5min). This > link explains the JWT concept we are using for authentication. > https://urldefense.com/v3/__https://stytch.com/blog/understanding-jwks/__;!!NpxR!hdQbIYBgb_bBdoWH7TVo30l1yIKlErqg_yZOBlMZLt8SWQyPYex2pjuUSytWaI1zd5pSClzgpi1KjsUsCg$ > but instead of using Stytch > as the authorization server JWKS https endpoint we are using Spire with a > plugin for the JWKS. The specific issue is that the JWT supplied with the > connect to the broker has expired by the time the LWT is being delivered. > We chose the MQTT protocol for its simplicity and the LWT feature for > server heartbeat. > > > > Not sure we have implemented the authorize function of the > securityManager class in alignment with how Artemis operates. We do not > create users up front but on the fly as each machine connects to publish > its status or scribe to another's status. We use the admin role for all of > the machine users/IDs. One on of the key features for using the JWT for us > is that we imbed the machine ID in the jwt payload and in the authorize > function reject any publish (SEND) request if the target topic does not > match the sender machine ID. Not sure how we would update the JWT for every > connection every 5min so that the JWT would be valid at the time it will be > sent? > > > > Is there a reference implementation of a pluggable securityManager that > uses JWTs? > > > > Any other suggestions? > > > > Regards, > > Paul > > > > From: Justin Bertram <[email protected]> > > Date: Tuesday, October 28, 2025 at 9:34 AM > > To: [email protected] <[email protected]> > > Subject: Re: MQTT Last Will not sent because denied authentication > > > > Just following up to ensure my explanation made sense. Do you need > anything > > further here? > > > > > > Justin > > > > On Fri, Oct 24, 2025 at 4:17 PM Justin Bertram <[email protected]> > wrote: > > > >> An MQTT client connects to the broker via a CONNECT packet. Typically > this > >> packet contains the client's credentials which are then authenticated by > >> the broker. > >> > >> Keep in mind that authentication and authorization are related but > >> separate things. A client may be authenticated but not authorized to > >> consume messages from or send messages to specific topics. This is a > >> fundamental concept of role-based access control. > >> > >> The CONNECT packet also contains all the details about the LWT message > >> (i.e. payload, properties, & topic). However, authorization for actually > >> sending the will message is not performed at this point. Authorization > is > >> only performed when the LWT message is actually sent (e.g. when the > >> client's connection fails). All sending operations are authorized at the > >> time they occur whether that's via the normal PUBLISH packet from the > >> client or for a LWT message sent on the client's behalf by the broker. > >> > >> Authorization is important here because the destination topic for the > LWT > >> message is arbitrary. If authorization was not performed then it would > be > >> simple for a client to send a message to a topic which it would not > >> otherwise have authorization. > >> > >> If authorization is failing when the broker attempts to send the > client's > >> LWT message and you're circumventing this so that the message is > actually > >> sent then it seems you may be undermining the security of your > environment. > >> You may inadvertently allow a clever, nefarious actor to send a message > to > >> a topic to which they are not authorized. > >> > >> How are you currently re-authenticating your clients when their JWTs > >> expire? > >> > >> > >> Justin > >> > >> On Fri, Oct 24, 2025 at 3:19 PM Shields, Paul > <[email protected]> > >> wrote: > >> > >>> Hi, > >>> > >>> Why are MQTT last will messages authenticated with Artemis before being > >>> sent? From my understanding of the MQTT last will feature is that > >>> authentication is a separate step handled during the client's initial > >>> connection. The LWT itself is a message prepared by a client and > stored by > >>> the broker to be published only if the client disconnects > unexpectedly. The > >>> security of the LWT message is therefore dependent on the security of > the > >>> initial client connection, which must be established through methods > like > >>> username/password or TLS and not when the LWT is delivered/published. > >>> > >>> We have written a custom securityManager plugin that uses Json Web > >>> Tokens as passwords for connecting MQTT clients. We use MQTT for server > >>> availability which has a state of either up or down. When a server > client > >>> connects with the MQTT Broker it is authenticated with the broker > using a > >>> JWT, registers a last will, and then publishes a server up message on a > >>> MQTT topic. We have other MQTT clients that also connect with the > Artemis > >>> broker using JWTs for auth and then subscribe to server state topics. > The > >>> MQTT last will is used to publish server down messages when the long > >>> running server dies for some reason. Some of the servers publish > >>> “re-announce” messages as they complete certain steps of processing > that > >>> they publish on a different topic using the initial client connection. > >>> This was initially developed using Artemis 2.28.0 and later used in > Artemis > >>> 2.34.0 and 2.40.0. Our securityManager plugin, JwtJassSecurityManager, > >>> implements the ActiveMQSecurityManager5 interface. During our initial > >>> development of the securityManager plugin we saw MQTT last will > messages > >>> failing authentication and put a workaround in place to check that if > there > >>> is a subject and a principal with the same user name (each MQTT client > has > >>> a unique user name) which means that this is the same session that has > >>> previously been authenticated, we will validate the JWT, but ignore the > >>> expiration time. We are now upgrading to Artemis 2.43.0 and would > like to > >>> remove the workaround. Our workaround for last will authentication > breaks > >>> down when the keys are rotated. It is about to become a real problem > as we > >>> are going to rotate the JWT key on a more frequent schedule. Here are > the > >>> errors produced: > >>> > >>> 2025-10-14 15:06:51,858 INFO > >>> [com.hpe.hpc.activemq.JwtJaasSecurityManager] validation failed: > username: > >>> x3000c0s9b0n0, details: auth is invalid > >>> InvalidJwtException: JWT processing failed. Additional details: [[17] > >>> Unable to process JOSE object (cause: > >>> org.jose4j.lang.UnresolvableKeyException: Unable to find a suitable > >>> verification key for JWS w/ header {"alg":"RS256","kid”:”previous > >>> KEY-A","typ":"JWT"} from JWKs [org.jose4j.jwk.RsaJsonWebKey{kty=RSA, > >>> kid=KEY-A, alg=RS256, n=CENSORED, e=CENSORED}, > >>> org.jose4j.jwk.RsaJsonWebKey{kty=RSA, kid=KEY-B, alg=RS256, n=CENSORED, > >>> e=CENSORED}, org.jose4j.jwk.RsaJsonWebKey{kty=RSA, kid=KEY-C, > alg=RS256, > >>> n=CENSORED, e=CENSORED}]): > JsonWebSignature{"alg":"RS256","kid”:”previous > >>> KEY-A","typ":”JWT”CENSORED > >>> ] > >>> 2025-10-14 15:06:51,861 INFO > >>> [com.hpe.hpc.activemq.JwtJaasSecurityManager] Invalid authentication > for > >>> user: x3000c0s9b0n0, subject: Subject: > >>> Principal: pzxQ2XQN > >>> Principal: admin > >>> Principal: x3000c0s9b0n0 > >>> > >>> 2025-10-14 15:06:51,863 WARN [org.apache.activemq.artemis.core.server] > >>> AMQ222216: Security problem while authenticating: AMQ229031: Unable to > >>> validate user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL > >>> certificate subject DN: unavailable > >>> 2025-10-14 15:06:51,863 ERROR > >>> [org.apache.activemq.artemis.core.protocol.mqtt] AMQ834002: Error > >>> processing control packet: > >>> MqttPublishMessage[fixedHeader=MqttFixedHeader[messageType=PUBLISH, > >>> isDup=false, qosLevel=AT_MOST_ONCE, isRetain=true, remainingLength=67], > >>> > variableHeader=MqttPublishVariableHeader[topicName=trusted/x3000c0s9b0n0/dvs/server/state, > >>> packetId=-1], payload=PooledSlicedByteBuf(ridx: 0, widx: 27, cap: > 27/27, > >>> unwrapped: PooledUnsafeDirectByteBuf(ridx: 69, widx: 69, cap: 80))] > >>> org.apache.activemq.artemis.api.core.ActiveMQSecurityException: > >>> AMQ229031: Unable to validate user from 127.0.0.6:49859. Username: > >>> x3000c0s9b0n0; SSL certificate subject DN: unavailable > >>> at > >>> > org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl.authenticationFailed(SecurityStoreImpl.java:449) > >>> at > >>> > org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl.check(SecurityStoreImpl.java:341) > >>> at > >>> > org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.securityCheck(ServerSessionImpl.java:527) > >>> at > >>> > org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.doSend(ServerSessionImpl.java:2365) > >>> at > >>> > org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1995) > >>> at > >>> > org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1934) > >>> at > >>> > org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendToQueue(MQTTPublishManager.java:242) > >>> at > >>> > org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler.handlePublish(MQTTProtocolHandler.java:322) > >>> at > >>> > org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler.act(MQTTProtocolHandler.java:164) > >>> at > >>> org.apache.activemq.artemis.utils.actors.Actor.doTask(Actor.java:32) > >>> at > >>> > org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:69) > >>> at > >>> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) > >>> at > >>> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) > >>> at > >>> > org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:120) > >>> 2025-10-14 15:06:51,866 WARN [org.apache.activemq.artemis.core.server] > >>> AMQ222216: Security problem while authenticating: AMQ229031: Unable to > >>> validate user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL > >>> certificate subject DN: unavailable > >>> 2025-10-14 15:06:51,866 ERROR > >>> [org.apache.activemq.artemis.core.protocol.mqtt] AMQ834007: > Authorization > >>> failure sending will message: AMQ229031: Unable to validate user from > >>> 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL certificate subject DN: > >>> unavailable > >>> > >>> Regards, > >>> Paul Shields > >>> > >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > For further information, visit: > https://urldefense.com/v3/__https://activemq.apache.org/contact__;!!NpxR!hdQbIYBgb_bBdoWH7TVo30l1yIKlErqg_yZOBlMZLt8SWQyPYex2pjuUSytWaI1zd5pSClzgpi1wpKbyCQ$ > > >
