Hi Matt, Thank you for the suggestion, But our backend s/w design is based around long term connections, So open and closing the connection has consequences in our backend. The JWT works well for the authenticate step of gaining access to the broker and setting up a session. It’s our authorize method that needs work. Does a session live as long as the MQTT client is connected to the broker? Any docs on how sessions are used and how they interact with the authorize method would be helpful.
Regards, Paul From: Matt Pavlovich <[email protected]> Date: Tuesday, October 28, 2025 at 1:59 PM To: [email protected] <[email protected]> Subject: Re: MQTT Last Will not sent because denied authentication Hi Paul- The usage of JWT and LWT are competing features, since JWT expires and LWT is intended to alert for unplanned disconnect of long-running connections. A possible solution is to perform periodic close-open of the connections and re-register the LWT send would always fall within the timeframe of the JWT expiry. Matt Pavlovich > On Oct 28, 2025, at 11:36 AM, Shields, Paul <[email protected]> > wrote: > > Hi Justin, > > I am struggling with this statement > However, authorization for actually > sending the will message is not performed at this point. Authorization is > only performed when the LWT message is actually sent (e.g. when the > client's connection fails). All sending operations are authorized at the > time they occur whether that's via the normal PUBLISH packet from the > client or for a LWT message sent on the client's behalf by the broker. > If we were using a user name and password (even if that was stored in LDAP) > that would be ok. But we are using the machine ID for the user name and a JWT > is used in the password field of the MQTT connect. My problem is that the > JWTs for authentication have a short expiration time (5min). This link > explains the JWT concept we are using for authentication. > https://urldefense.com/v3/__https://stytch.com/blog/understanding-jwks/__;!!NpxR!l0K48Qr3zBCaoFmyCZXVvGQOZMZY59F9n8cgHM4q5vNc5xz0PCpg7C4HOGannJ2cB0tn2U7ian9s7tNc7A$ > but instead of using Stytch as the authorization server JWKS https endpoint > we are using Spire with a plugin for the JWKS. The specific issue is that the > JWT supplied with the connect to the broker has expired by the time the LWT > is being delivered. We chose the MQTT protocol for its simplicity and the LWT > feature for server heartbeat. > > Not sure we have implemented the authorize function of the securityManager > class in alignment with how Artemis operates. We do not create users up > front but on the fly as each machine connects to publish its status or scribe > to another's status. We use the admin role for all of the machine users/IDs. > One on of the key features for using the JWT for us is that we imbed the > machine ID in the jwt payload and in the authorize function reject any > publish (SEND) request if the target topic does not match the sender machine > ID. Not sure how we would update the JWT for every connection every 5min so > that the JWT would be valid at the time it will be sent? > > Is there a reference implementation of a pluggable securityManager that uses > JWTs? > > Any other suggestions? > > Regards, > Paul > > From: Justin Bertram <[email protected]> > Date: Tuesday, October 28, 2025 at 9:34 AM > To: [email protected] <[email protected]> > Subject: Re: MQTT Last Will not sent because denied authentication > > Just following up to ensure my explanation made sense. Do you need anything > further here? > > > Justin > > On Fri, Oct 24, 2025 at 4:17 PM Justin Bertram <[email protected]> wrote: > >> An MQTT client connects to the broker via a CONNECT packet. Typically this >> packet contains the client's credentials which are then authenticated by >> the broker. >> >> Keep in mind that authentication and authorization are related but >> separate things. A client may be authenticated but not authorized to >> consume messages from or send messages to specific topics. This is a >> fundamental concept of role-based access control. >> >> The CONNECT packet also contains all the details about the LWT message >> (i.e. payload, properties, & topic). However, authorization for actually >> sending the will message is not performed at this point. Authorization is >> only performed when the LWT message is actually sent (e.g. when the >> client's connection fails). All sending operations are authorized at the >> time they occur whether that's via the normal PUBLISH packet from the >> client or for a LWT message sent on the client's behalf by the broker. >> >> Authorization is important here because the destination topic for the LWT >> message is arbitrary. If authorization was not performed then it would be >> simple for a client to send a message to a topic which it would not >> otherwise have authorization. >> >> If authorization is failing when the broker attempts to send the client's >> LWT message and you're circumventing this so that the message is actually >> sent then it seems you may be undermining the security of your environment. >> You may inadvertently allow a clever, nefarious actor to send a message to >> a topic to which they are not authorized. >> >> How are you currently re-authenticating your clients when their JWTs >> expire? >> >> >> Justin >> >> On Fri, Oct 24, 2025 at 3:19 PM Shields, Paul <[email protected]> >> wrote: >> >>> Hi, >>> >>> Why are MQTT last will messages authenticated with Artemis before being >>> sent? From my understanding of the MQTT last will feature is that >>> authentication is a separate step handled during the client's initial >>> connection. The LWT itself is a message prepared by a client and stored by >>> the broker to be published only if the client disconnects unexpectedly. The >>> security of the LWT message is therefore dependent on the security of the >>> initial client connection, which must be established through methods like >>> username/password or TLS and not when the LWT is delivered/published. >>> >>> We have written a custom securityManager plugin that uses Json Web >>> Tokens as passwords for connecting MQTT clients. We use MQTT for server >>> availability which has a state of either up or down. When a server client >>> connects with the MQTT Broker it is authenticated with the broker using a >>> JWT, registers a last will, and then publishes a server up message on a >>> MQTT topic. We have other MQTT clients that also connect with the Artemis >>> broker using JWTs for auth and then subscribe to server state topics. The >>> MQTT last will is used to publish server down messages when the long >>> running server dies for some reason. Some of the servers publish >>> “re-announce” messages as they complete certain steps of processing that >>> they publish on a different topic using the initial client connection. >>> This was initially developed using Artemis 2.28.0 and later used in Artemis >>> 2.34.0 and 2.40.0. Our securityManager plugin, JwtJassSecurityManager, >>> implements the ActiveMQSecurityManager5 interface. During our initial >>> development of the securityManager plugin we saw MQTT last will messages >>> failing authentication and put a workaround in place to check that if there >>> is a subject and a principal with the same user name (each MQTT client has >>> a unique user name) which means that this is the same session that has >>> previously been authenticated, we will validate the JWT, but ignore the >>> expiration time. We are now upgrading to Artemis 2.43.0 and would like to >>> remove the workaround. Our workaround for last will authentication breaks >>> down when the keys are rotated. It is about to become a real problem as we >>> are going to rotate the JWT key on a more frequent schedule. Here are the >>> errors produced: >>> >>> 2025-10-14 15:06:51,858 INFO >>> [com.hpe.hpc.activemq.JwtJaasSecurityManager] validation failed: username: >>> x3000c0s9b0n0, details: auth is invalid >>> InvalidJwtException: JWT processing failed. Additional details: [[17] >>> Unable to process JOSE object (cause: >>> org.jose4j.lang.UnresolvableKeyException: Unable to find a suitable >>> verification key for JWS w/ header {"alg":"RS256","kid”:”previous >>> KEY-A","typ":"JWT"} from JWKs [org.jose4j.jwk.RsaJsonWebKey{kty=RSA, >>> kid=KEY-A, alg=RS256, n=CENSORED, e=CENSORED}, >>> org.jose4j.jwk.RsaJsonWebKey{kty=RSA, kid=KEY-B, alg=RS256, n=CENSORED, >>> e=CENSORED}, org.jose4j.jwk.RsaJsonWebKey{kty=RSA, kid=KEY-C, alg=RS256, >>> n=CENSORED, e=CENSORED}]): JsonWebSignature{"alg":"RS256","kid”:”previous >>> KEY-A","typ":”JWT”CENSORED >>> ] >>> 2025-10-14 15:06:51,861 INFO >>> [com.hpe.hpc.activemq.JwtJaasSecurityManager] Invalid authentication for >>> user: x3000c0s9b0n0, subject: Subject: >>> Principal: pzxQ2XQN >>> Principal: admin >>> Principal: x3000c0s9b0n0 >>> >>> 2025-10-14 15:06:51,863 WARN [org.apache.activemq.artemis.core.server] >>> AMQ222216: Security problem while authenticating: AMQ229031: Unable to >>> validate user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL >>> certificate subject DN: unavailable >>> 2025-10-14 15:06:51,863 ERROR >>> [org.apache.activemq.artemis.core.protocol.mqtt] AMQ834002: Error >>> processing control packet: >>> MqttPublishMessage[fixedHeader=MqttFixedHeader[messageType=PUBLISH, >>> isDup=false, qosLevel=AT_MOST_ONCE, isRetain=true, remainingLength=67], >>> variableHeader=MqttPublishVariableHeader[topicName=trusted/x3000c0s9b0n0/dvs/server/state, >>> packetId=-1], payload=PooledSlicedByteBuf(ridx: 0, widx: 27, cap: 27/27, >>> unwrapped: PooledUnsafeDirectByteBuf(ridx: 69, widx: 69, cap: 80))] >>> org.apache.activemq.artemis.api.core.ActiveMQSecurityException: >>> AMQ229031: Unable to validate user from 127.0.0.6:49859. Username: >>> x3000c0s9b0n0; SSL certificate subject DN: unavailable >>> at >>> org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl.authenticationFailed(SecurityStoreImpl.java:449) >>> at >>> org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl.check(SecurityStoreImpl.java:341) >>> at >>> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.securityCheck(ServerSessionImpl.java:527) >>> at >>> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.doSend(ServerSessionImpl.java:2365) >>> at >>> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1995) >>> at >>> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1934) >>> at >>> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendToQueue(MQTTPublishManager.java:242) >>> at >>> org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler.handlePublish(MQTTProtocolHandler.java:322) >>> at >>> org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler.act(MQTTProtocolHandler.java:164) >>> at >>> org.apache.activemq.artemis.utils.actors.Actor.doTask(Actor.java:32) >>> at >>> org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:69) >>> at >>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) >>> at >>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) >>> at >>> org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:120) >>> 2025-10-14 15:06:51,866 WARN [org.apache.activemq.artemis.core.server] >>> AMQ222216: Security problem while authenticating: AMQ229031: Unable to >>> validate user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL >>> certificate subject DN: unavailable >>> 2025-10-14 15:06:51,866 ERROR >>> [org.apache.activemq.artemis.core.protocol.mqtt] AMQ834007: Authorization >>> failure sending will message: AMQ229031: Unable to validate user from >>> 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL certificate subject DN: >>> unavailable >>> >>> Regards, >>> Paul Shields >>> >> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] For further information, visit: https://urldefense.com/v3/__https://activemq.apache.org/contact__;!!NpxR!l0K48Qr3zBCaoFmyCZXVvGQOZMZY59F9n8cgHM4q5vNc5xz0PCpg7C4HOGannJ2cB0tn2U7ian-lWM-jFw$
