Thank you all for taking your time to respond to me. @Justin, no I don't use the pattern because I don't have any control over producers, they send text messages (3 in type header). I have come to the conclusion that I've to use the Core API because there I can access the ClientMessage and check if it is larger than what my client can handle, or I can just stream it and count. The good thing about Core API is that I don't have to trust producers and I can validate the message on a much grainer level before receiving it, which is just not possible with artemis-jakarta-client. Would you be interested in a PR that adds this type of control to the JMS consumer client? This would benefit consumers that want to protect themselves from badly configured & buggy producers. This would be specific to artemis-jakarta-client, text message. Since it is a safety feature it would be a configuration that consumers can set, and if the TextMessage is above that threshold a JMSException would be thrown. I can work on this PR.
Den tors 1 maj 2025 kl 21:46 skrev Timothy Bish <tabish...@gmail.com>: > On 5/1/25 15:31, Justin Bertram wrote: > >> I can see that it arrives as you describe as a large message but it > reads > > it all into memory (ActiveMQTextMessage) before reaching user code... > > > > Are you using the streaming pattern as outlined in the documentation I > > linked previously? If not then the entire message body will be loaded > into > > memory. Streaming large messages is a special feature and requires > > different client behavior from consuming normal messages. > > > >> It would be great if I could configure maximum allowed text message size > > on the consumer... > > > > To my knowledge no such feature exists for the Core protocol. However, > MQTT > > 5 has a feature like this if you're willing to switch clients. > > > There is also Qpid protonj2 which offers a streaming receiver for AMQP > in which you can control the maximum amount of bytes the broker can send > at any given time and read them from an InputStream which will open the > transmit window as you read the data. > > > https://github.com/apache/qpid-protonj2/blob/main/protonj2-client-examples/src/main/java/org/apache/qpid/protonj2/client/examples/StreamingFileReceiver.java > > https://qpid.apache.org/releases/qpid-protonj2-1.0.0-M23/api/index.html > > > Justin > > > > On Wed, Apr 30, 2025 at 4:25 PM Alexej Timonin <alexejtimo...@gmail.com> > > wrote: > > > >> Thank you for the response Justin. I get out of memory crashes when > using > >> artemis-jakarta-client on the consumer side when the message is of type > >> Text. > >> > >> The artemis-jakarta-client buffers the whole large message into memory > if > >> the message is of type Text and that crashes the consumer if the > message is > >> larger than consumers memory. > >> > >> For the experiment I tried to run my consumer with 256MB memory and > posted > >> a text message of size 300MB. I can see that it arrives as you describe > as > >> a large message but it reads it all into memory (ActiveMQTextMessage) > >> before reaching user code, it all happens inside the > artemis-jakarta-client > >> and the consumer crashes due to an out of memory error. I know the > example > >> is extreme but that is what I want to protect my consumer against. > >> > >> It would be great if I could configure maximum allowed text message > size on > >> the consumer, so when the consumer receives messages larger than the > >> configured max size it throws JMS exception and the message can be > subject > >> to DLQ policy on the broker side. > >> > >> On Wed, Apr 30, 2025, 21:50 Justin Bertram <jbert...@apache.org> wrote: > >> > >>> If the message was sent as a "large" message or converted into a large > >>> message by the broker (e.g. if it exceeded the journal-buffer-size) > then > >> it > >>> can be streamed from the broker in chunks via JMS as noted in the > >>> documentation [1]. However, if the message is not "large" then the > entire > >>> message will be transferred to the client. > >>> > >>> > >>> Justin > >>> > >>> [1] > >>> > >>> > >> > https://activemq.apache.org/components/artemis/documentation/latest/large-messages.html#streaming-over-jms > >>> On Tue, Apr 22, 2025 at 4:14 PM Alexej Timonin < > alexejtimo...@gmail.com> > >>> wrote: > >>> > >>>> Hi! > >>>> > >>>> I'm building a consumer which will be listening on text messages (I've > >> no > >>>> control over producers). I want to protect my consumer such that it > >> does > >>>> not read messages larger than 256KB to avoid crashing due to memory > >>>> limitations. Is there any way in artemis jms client implementation to > >> see > >>>> if a text message is larger than given size without having to load it > >>> into > >>>> memory? > >>>> > >>>> My understanding is that it's not possible when reading code of > >>>> ActiveMQTextMessage, specifically doBeforeReceive method, looks like > it > >>>> loads the entire message into memory - > >>>> > >>>> > >> > https://github.com/apache/activemq-artemis/blob/6f68668c867b23c7468f85c91c96a74136a561ad/artemis-jms-client/src/main/java/org/apache/activemq/artemis/jms/client/ActiveMQTextMessage.java#L104 > >>>> Is my understanding correct? > >>>> > >>>> Thank you in advance! > >>>> > > -- > Tim Bish > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@activemq.apache.org > For additional commands, e-mail: users-h...@activemq.apache.org > For further information, visit: https://activemq.apache.org/contact > > >
