Neither the client-side minLargeMessage or broker-side amqpMinLargeMessage config will have much effect if you are only consuming from camel / JMS client, since as before they mainly influence publishing. The Artemis JMS client sends messages 'large' slightly differently, doing this when its minLargeMessage size is exceeded (it has a default of 100kb). The broker persists published AMQP messages differently as they arrive if they exceed the brokers amqpMinLargeMessageSize (and has a default of 100kb).
AMQP doesnt have distinct 'large' and 'not large' messages, thats why "minLargeMessage" config isnt/wont-be supported for Camel AMQP endpoints, since it is an Artemis client/protocol specific configuration, and not something the Qpid JMS client uses. Again I'd suggest you give a newer Camel a try, perhaps there have been bugs there which have been fixed. For example you are using 3.11 but I see in the upgrade details for the very next version that they changed the default behaviour so you have to explicitly enable the seperate artemis-specific 'special input/output-stream Object property' handling stuff (via artemisStreamingEnabled option), which I'd guess is exactly what is causing your problem due to Camel trying it with the non-artemis client. https://github.com/apache/camel/blob/26b156bc3d7c85bc629f2ca14f852bc16273f023/docs/user-manual/modules/ROOT/pages/camel-3x-upgrade-guide-3_12.adoc#camel-jms On Mon, 4 Sept 2023 at 16:27, Modanese, Riccardo <riccardo.modan...@eurotech.com.invalid> wrote: > > I was just wondering if there was a way to disable, or at least force > globally from server side, a value for that parameter. > I mean, the default value is in some way enforced by the broker if no > override is done. So, in the same way, I was wondering if this default > parameter could be changed without adding any parameter to acceptors or > connection strings (according to the journal buffer size, obviously, that > anyway could be changed by configuration right? - > <journal-buffer-size>my_value</journal-buffer-size>) > > I mentioned MQTT clients to give a picture of our configuration. > Our Artemis exposes 3 acceptors. 2 MQTT for IoT clients with 2 ways load > (each client publishes and subscribes different topics) and one AMQP for > consumers (Camel + streams configured) implementing business logic. > > The Camel routes steps have generic signature (Exchange, Object) and in the > full stack trace (I partially reported here) there wasn't any of our class > involved. (I know Object is redundant, but I preferred instead of getting it > from exchange.in in every method) > > With my original route configuration, I tried to set the amqpMinLargeMessage > and minLargeMessage to both the acceptor and the connection string on client > side but without any effect. > > I already received support from Camel mailing list and they suggested me to > switch to Camel-jms since Camel-amqp doesn’t support the minLargeMessage > (though in my tests I was able to get the stream from Artemis once I switched > to use core instead of amqp) > > I’m looking forward to see the minLargeMessage supported by Camel streams > also with amqp endpoint. > > From: Robbie Gemmell <robbie.gemm...@gmail.com> > Date: Monday, 4 September 2023 at 14:11 > To: users@activemq.apache.org <users@activemq.apache.org> > Subject: Re: Disable large messages support at all > Though actually, another thing occurred to me...you said your "clients > are MQTT", in which case the Artemis client minLargeMessageSize config > and broker side amqpMinLargeMessageSize config wont necessarily be in > play if your Camel bits are only consuming messages, as those configs > primarily affect what happens when the Core and AMQP clients are > producing messages. Its not really clear so far what all you are doing > at each component. > > On Mon, 4 Sept 2023 at 12:34, Robbie Gemmell <robbie.gemm...@gmail.com> wrote: > > > > The Artemis broker / client handling of 'large' messages, and > > 'streaming payload to/from file at client' are 2 separate things and > > shouldnt be conflated. > > > > The Artemis 'Core' JMS client + broker consider messages over 100kb by > > default 'large' and treat them a little differently in terms of how > > they are sent by the producer client, and then the broker storing them > > differently on disk, and sending them to consuming clients. > > > > The Artemis client can separately leverage a custom Object property to > > read/write the BytesMessage body using an InputStream / OutputStream, > > e.g to from/to a file, at the application side. > > > > The exception below seems (not 100% clear, as the stack isnt present) > > to be saying that while trying to extract the body of a message Camel > > actually tried to write to a read-only JMS Message (specifically the > > properties of it), and so the qpid-jms client threw because of that, > > as it should. That seems like potentially either an issue with your > > configuration or a bug in Camel. Guessing, is it possible you/Camel > > perhaps configuring things to try inserting the custom 'input/output > > stream' Object property used by the Artemis JMS client, even though it > > isnt actually using the Artemis client? That certainly might explain > > the seeming 'set property while trying to extract body from read-only > > message' behaviour. As you are asking here to 'disable large messages' > > entirely, it sounds like your messages aren't necessarily all that big > > and you dont necessarily need/want the 'stream message to/from file' > > input/output stream behaviour at the client side anyway, and if so you > > can disable that? > > > > I see you mentioned Camel 3.11. That's pretty old now, perhaps also > > try a more recent version to see if it fixed anything that might be in > > play here? > > > > There is a second broker-side acceptor URL config parameter, > > amqpMinLargeMessageSize, that defaults to 102400 (100kb) and can > > change the point at which the broker considers AMQP messages to be > > 'large' and handles them on disk in similar way as it does the Artemis > > Core 'large' messages. Though there are also other considerations > > though as Justin said, such as the journal buffer size and file size, > > so its not a case of just setting it really large to disable the > > behaviour entirely. > > > > On Mon, 4 Sept 2023 at 08:32, Modanese, Riccardo > > <riccardo.modan...@eurotech.com.invalid> wrote: > > > > > > Hi Justin, > > > Thank you for your reply. > > > I had no way to get large messages support using Camel routes with > > > Camel-amqp endpoint. > > > With Camel-amqp, using qpid connection > > > (org.apache.qpid.jms.JmsConnectionFactory), messages over the default > > > 100kb weren’t processed. > > > Camel went to an infinite loop printing this error at each iteration: > > > > > > 2023-08-23 11:07:18 org.apache.camel.RuntimeCamelException: Failed to > > > extract body due to: javax.jms.MessageNotWriteableException: Message > > > properties are read-only. Message: JmsBytesMessage { > > > org.apache.qpid.jms.provider.amqp.message.AmqpJmsBytesMessageFacade@39dd152c<mailto:org.apache.qpid.jms.provider.amqp.message.AmqpJmsBytesMessageFacade@39dd152c> > > > } > > > 2023-08-23 11:07:18 at > > > org.apache.camel.component.jms.JmsBinding.extractBodyFromJms(JmsBinding.java:174) > > > 2023-08-23 11:07:18 at > > > org.apache.camel.component.jms.JmsMessage.createBody(JmsMessage.java:234) > > > > > > Switching to core the messages over 100k were handled but no way to > > > change the minimum large messages default value. > > > > > > From the support I got from Camel mailing list there is no other way than > > > using Camel-jms to have full large messages support: > > > > > > “I am afraid that in order to use minLargeMessageSize you need to use > > > *camel-jms* component and *artemis-jms-client* as jms implementation, > > > because camel-amqp uses *qpid-jms-client.*If you have some time, you can > > > take a look to camel-jms” > > > > > > This force us to use the Artemis connection factory > > > (org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory), in > > > our clients, to connect to the broker, decreasing the generality we like > > > to have on our opensource project. > > > > > > I’m still testing a working configuration with Camel-jms but I was just > > > wondering if there was a way to disable the large messages support. > > > > > > Riccardo > > > > > > From: Justin Bertram <jbert...@apache.org> > > > Date: Friday, 1 September 2023 at 20:56 > > > To: users@activemq.apache.org <users@activemq.apache.org> > > > Subject: Re: Disable large messages support at all > > > There is no way to completely disable large message support. This is > > > because clients are free to send messages larger than the configured > > > journal-buffer-size (490KB by default). When that happens the broker has > > > no > > > choice but to treat the message as "large" and stream it to disk rather > > > than storing it in the journal. You can increase the journal-buffer-size > > > to > > > make large messages less likely or even effectively eliminate them > > > depending on your use-case, but using the larger buffer may have an impact > > > on performance. > > > > > > What "difficulties" are you having currently with AMQP, MQTT, & large > > > messages? > > > > > > > > > Justin > > > > > > > > > On Wed, Aug 30, 2023 at 2:21 AM Modanese, Riccardo > > > <riccardo.modan...@eurotech.com.invalid> wrote: > > > > > > > Hello to everyone, > > > > > > > > I’m trying to configure my Camel routes (3.11) with Artemis (2.28.0) > > > > large messages but I’m still experiencing some difficulties. > > > > My clients are MQTT and my Camel routes are using AMQP endpoint. > > > > Is there a way to completely disable large message support on Artemis or > > > > to set the min value globally on Artemis to override any setting made on > > > > connectors bases? > > > > > > > > Regards, > > > > > > > > Riccardo Modanese > > > > > > > >
