Hi Paul- The usage of JWT and LWT are competing features, since JWT expires and LWT is intended to alert for unplanned disconnect of long-running connections. A possible solution is to perform periodic close-open of the connections and re-register the LWT send would always fall within the timeframe of the JWT expiry.
Matt Pavlovich > On Oct 28, 2025, at 11:36 AM, Shields, Paul <[email protected]> > wrote: > > Hi Justin, > > I am struggling with this statement > However, authorization for actually > sending the will message is not performed at this point. Authorization is > only performed when the LWT message is actually sent (e.g. when the > client's connection fails). All sending operations are authorized at the > time they occur whether that's via the normal PUBLISH packet from the > client or for a LWT message sent on the client's behalf by the broker. > If we were using a user name and password (even if that was stored in LDAP) > that would be ok. But we are using the machine ID for the user name and a JWT > is used in the password field of the MQTT connect. My problem is that the > JWTs for authentication have a short expiration time (5min). This link > explains the JWT concept we are using for authentication. > https://stytch.com/blog/understanding-jwks/ but instead of using Stytch as > the authorization server JWKS https endpoint we are using Spire with a plugin > for the JWKS. The specific issue is that the JWT supplied with the connect to > the broker has expired by the time the LWT is being delivered. We chose the > MQTT protocol for its simplicity and the LWT feature for server heartbeat. > > Not sure we have implemented the authorize function of the securityManager > class in alignment with how Artemis operates. We do not create users up > front but on the fly as each machine connects to publish its status or scribe > to another's status. We use the admin role for all of the machine users/IDs. > One on of the key features for using the JWT for us is that we imbed the > machine ID in the jwt payload and in the authorize function reject any > publish (SEND) request if the target topic does not match the sender machine > ID. Not sure how we would update the JWT for every connection every 5min so > that the JWT would be valid at the time it will be sent? > > Is there a reference implementation of a pluggable securityManager that uses > JWTs? > > Any other suggestions? > > Regards, > Paul > > From: Justin Bertram <[email protected]> > Date: Tuesday, October 28, 2025 at 9:34 AM > To: [email protected] <[email protected]> > Subject: Re: MQTT Last Will not sent because denied authentication > > Just following up to ensure my explanation made sense. Do you need anything > further here? > > > Justin > > On Fri, Oct 24, 2025 at 4:17 PM Justin Bertram <[email protected]> wrote: > >> An MQTT client connects to the broker via a CONNECT packet. Typically this >> packet contains the client's credentials which are then authenticated by >> the broker. >> >> Keep in mind that authentication and authorization are related but >> separate things. A client may be authenticated but not authorized to >> consume messages from or send messages to specific topics. This is a >> fundamental concept of role-based access control. >> >> The CONNECT packet also contains all the details about the LWT message >> (i.e. payload, properties, & topic). However, authorization for actually >> sending the will message is not performed at this point. Authorization is >> only performed when the LWT message is actually sent (e.g. when the >> client's connection fails). All sending operations are authorized at the >> time they occur whether that's via the normal PUBLISH packet from the >> client or for a LWT message sent on the client's behalf by the broker. >> >> Authorization is important here because the destination topic for the LWT >> message is arbitrary. If authorization was not performed then it would be >> simple for a client to send a message to a topic which it would not >> otherwise have authorization. >> >> If authorization is failing when the broker attempts to send the client's >> LWT message and you're circumventing this so that the message is actually >> sent then it seems you may be undermining the security of your environment. >> You may inadvertently allow a clever, nefarious actor to send a message to >> a topic to which they are not authorized. >> >> How are you currently re-authenticating your clients when their JWTs >> expire? >> >> >> Justin >> >> On Fri, Oct 24, 2025 at 3:19 PM Shields, Paul <[email protected]> >> wrote: >> >>> Hi, >>> >>> Why are MQTT last will messages authenticated with Artemis before being >>> sent? From my understanding of the MQTT last will feature is that >>> authentication is a separate step handled during the client's initial >>> connection. The LWT itself is a message prepared by a client and stored by >>> the broker to be published only if the client disconnects unexpectedly. The >>> security of the LWT message is therefore dependent on the security of the >>> initial client connection, which must be established through methods like >>> username/password or TLS and not when the LWT is delivered/published. >>> >>> We have written a custom securityManager plugin that uses Json Web >>> Tokens as passwords for connecting MQTT clients. We use MQTT for server >>> availability which has a state of either up or down. When a server client >>> connects with the MQTT Broker it is authenticated with the broker using a >>> JWT, registers a last will, and then publishes a server up message on a >>> MQTT topic. We have other MQTT clients that also connect with the Artemis >>> broker using JWTs for auth and then subscribe to server state topics. The >>> MQTT last will is used to publish server down messages when the long >>> running server dies for some reason. Some of the servers publish >>> “re-announce” messages as they complete certain steps of processing that >>> they publish on a different topic using the initial client connection. >>> This was initially developed using Artemis 2.28.0 and later used in Artemis >>> 2.34.0 and 2.40.0. Our securityManager plugin, JwtJassSecurityManager, >>> implements the ActiveMQSecurityManager5 interface. During our initial >>> development of the securityManager plugin we saw MQTT last will messages >>> failing authentication and put a workaround in place to check that if there >>> is a subject and a principal with the same user name (each MQTT client has >>> a unique user name) which means that this is the same session that has >>> previously been authenticated, we will validate the JWT, but ignore the >>> expiration time. We are now upgrading to Artemis 2.43.0 and would like to >>> remove the workaround. Our workaround for last will authentication breaks >>> down when the keys are rotated. It is about to become a real problem as we >>> are going to rotate the JWT key on a more frequent schedule. Here are the >>> errors produced: >>> >>> 2025-10-14 15:06:51,858 INFO >>> [com.hpe.hpc.activemq.JwtJaasSecurityManager] validation failed: username: >>> x3000c0s9b0n0, details: auth is invalid >>> InvalidJwtException: JWT processing failed. Additional details: [[17] >>> Unable to process JOSE object (cause: >>> org.jose4j.lang.UnresolvableKeyException: Unable to find a suitable >>> verification key for JWS w/ header {"alg":"RS256","kid”:”previous >>> KEY-A","typ":"JWT"} from JWKs [org.jose4j.jwk.RsaJsonWebKey{kty=RSA, >>> kid=KEY-A, alg=RS256, n=CENSORED, e=CENSORED}, >>> org.jose4j.jwk.RsaJsonWebKey{kty=RSA, kid=KEY-B, alg=RS256, n=CENSORED, >>> e=CENSORED}, org.jose4j.jwk.RsaJsonWebKey{kty=RSA, kid=KEY-C, alg=RS256, >>> n=CENSORED, e=CENSORED}]): JsonWebSignature{"alg":"RS256","kid”:”previous >>> KEY-A","typ":”JWT”CENSORED >>> ] >>> 2025-10-14 15:06:51,861 INFO >>> [com.hpe.hpc.activemq.JwtJaasSecurityManager] Invalid authentication for >>> user: x3000c0s9b0n0, subject: Subject: >>> Principal: pzxQ2XQN >>> Principal: admin >>> Principal: x3000c0s9b0n0 >>> >>> 2025-10-14 15:06:51,863 WARN [org.apache.activemq.artemis.core.server] >>> AMQ222216: Security problem while authenticating: AMQ229031: Unable to >>> validate user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL >>> certificate subject DN: unavailable >>> 2025-10-14 15:06:51,863 ERROR >>> [org.apache.activemq.artemis.core.protocol.mqtt] AMQ834002: Error >>> processing control packet: >>> MqttPublishMessage[fixedHeader=MqttFixedHeader[messageType=PUBLISH, >>> isDup=false, qosLevel=AT_MOST_ONCE, isRetain=true, remainingLength=67], >>> variableHeader=MqttPublishVariableHeader[topicName=trusted/x3000c0s9b0n0/dvs/server/state, >>> packetId=-1], payload=PooledSlicedByteBuf(ridx: 0, widx: 27, cap: 27/27, >>> unwrapped: PooledUnsafeDirectByteBuf(ridx: 69, widx: 69, cap: 80))] >>> org.apache.activemq.artemis.api.core.ActiveMQSecurityException: >>> AMQ229031: Unable to validate user from 127.0.0.6:49859. Username: >>> x3000c0s9b0n0; SSL certificate subject DN: unavailable >>> at >>> org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl.authenticationFailed(SecurityStoreImpl.java:449) >>> at >>> org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl.check(SecurityStoreImpl.java:341) >>> at >>> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.securityCheck(ServerSessionImpl.java:527) >>> at >>> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.doSend(ServerSessionImpl.java:2365) >>> at >>> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1995) >>> at >>> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1934) >>> at >>> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendToQueue(MQTTPublishManager.java:242) >>> at >>> org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler.handlePublish(MQTTProtocolHandler.java:322) >>> at >>> org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler.act(MQTTProtocolHandler.java:164) >>> at >>> org.apache.activemq.artemis.utils.actors.Actor.doTask(Actor.java:32) >>> at >>> org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:69) >>> at >>> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) >>> at >>> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) >>> at >>> org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:120) >>> 2025-10-14 15:06:51,866 WARN [org.apache.activemq.artemis.core.server] >>> AMQ222216: Security problem while authenticating: AMQ229031: Unable to >>> validate user from 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL >>> certificate subject DN: unavailable >>> 2025-10-14 15:06:51,866 ERROR >>> [org.apache.activemq.artemis.core.protocol.mqtt] AMQ834007: Authorization >>> failure sending will message: AMQ229031: Unable to validate user from >>> 127.0.0.6:49859. Username: x3000c0s9b0n0; SSL certificate subject DN: >>> unavailable >>> >>> Regards, >>> Paul Shields >>> >> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] For further information, visit: https://activemq.apache.org/contact
