Either that of have your Java producer to switch to AMQP. And that is true that the whole message will be read into memory before it's converted.
On Tue, Jul 26, 2022 at 12:46 AM Jan Šmucr <jan.sm...@aimtecglobal.com> wrote: > > Hello. I'm sorry for the late reply but somehow the message got caught by my > spam filter. > I believe we'd like to keep the broker from becoming overwhelmed by loading > the entire message to memory just for conversion purposes, so if we can avoid > it by sticking with the core protocol, I think it's going to be the way to go. > > Thank you though. 🙂 > Jan > > Dne 23. 7. 2022 21:23 napsal uživatel Clebert Suconic > <clebert.suco...@gmail.com>: > If you want help with your core producer to generate a compatible message > it’s a different thing. So far the issue was reported as a bug. > > I can certainly help next week. (From Monday ) > > I will make sure I connect on slack on Monday perhaps we could talk there. > > On Sat, Jul 23, 2022 at 2:49 PM Clebert Suconic <clebert.suco...@gmail.com> > wrote: > > > We could look at enhancing the converters. But the best would be to > > change your producer to have the exact format the converter would have. > > > > On Sat, Jul 23, 2022 at 2:41 PM Clebert Suconic <clebert.suco...@gmail.com> > > wrote: > > > >> Change your message in a way is compatible ? > >> > >> On Fri, Jul 22, 2022 at 3:36 PM Jan Šmucr <jan.sm...@aimtecglobal.com> > >> wrote: > >> > >>> The reason for this is that there's a whole infrastructure built using > >>> the core protocol, and now we need to connect a Python-based Lambda > >>> capable > >>> of receiving large messages. Is there any other, core-compatible method? > >>> > >>> Jan > >>> > >>> Dne 22. 7. 2022 21:21 napsal uživatel Clebert Suconic < > >>> clebert.suco...@gmail.com>: > >>> If you expect conversions to happen, you have to send messages from > >>> qpid-ms or core-jms client. > >>> > >>> > >>> On the case: > >>> > >>> > >>> ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(); > >>> Connection conn = factory.createConnection(...); > >>> Session session = conn.createSession(...) > >>> Producer producer = session.createProducer(....); > >>> producer.send(session.createTextMessage(YOUR-LARGE-BODY-GOES-HERE); > >>> > >>> > >>> > >>> Then the conversion from Core to AMQP should happen fine when you > >>> receive the message on the other side at your Python client. > >>> > >>> > >>> > >>> If you really must use the Core-API, then you will need to hack your > >>> way in and find what the JMS client would do.. but at that point it's > >>> just a non standard way of doing this... You would need to have a very > >>> good reason to not do the right thing on this case. > >>> > >>> On Fri, Jul 22, 2022 at 3:00 PM Clebert Suconic > >>> <clebert.suco...@gmail.com> wrote: > >>> > > >>> > If you’re not using qpid JMS it will not work. > >>> > > >>> > On Fri, Jul 22, 2022 at 8:06 AM Andy Yar <andyya...@gmail.com> wrote: > >>> >> > >>> >> There is another attempt employing AMQP test structures. It produces > >>> the same incorrect message body. > >>> >> > >>> >> On Fri, Jul 22, 2022 at 12:21 PM Andy Yar <andyya...@gmail.com> > >>> wrote: > >>> >>> > >>> >>> Hello, > >>> >>> Using the LargeServerMessageImpl was just an attempt providing a > >>> seemingly easy way to introduce a kind of Large Message into the test. I'm > >>> not familiar with the code base at all - so I simply tried to somehow > >>> provide the requested test replicating my "sending via Core -> receiving > >>> via Qpid" issue. The same applies to the explicit AMQP conversion - the > >>> encountered error simply pointed on it. Is there an intended "test way" to > >>> model the message transfer among different clients/protocols? > >>> >>> > >>> >>> Anyway, I've changed the test to employ > >>> ClientLargeMessageImpl/ClientMessageImpl instead of the server-related > >>> LargeServerMessageImpl with a very similar result. Does that > >>> implementation > >>> fit the simulated use case (sending via Core -> receiving via Qpid AMQP) > >>> better or is the explicit AMQP conversion misused? > >>> >>> > >>> >>> Please, see the attached patch. > >>> >>> > >>> >>> On Mon, Jul 18, 2022 at 3:50 PM Clebert Suconic < > >>> clebert.suco...@gmail.com> wrote: > >>> >>>> > >>> >>>> AMQP to Core and Core to AMQP conversions expect you using JMS > >>> clients > >>> >>>> to generate your message. There are certain caveats that both > >>> clients > >>> >>>> will add to the messages. > >>> >>>> > >>> >>>> > >>> >>>> Usually I have seen more users doing weird conversions in AMQP when > >>> >>>> they use a C++ or non Java Client.. but on your case you used a Core > >>> >>>> Client, using a non standard way to send a message (ServerMessage > >>> from > >>> >>>> the client)... and it's not clear the message is correctly set for a > >>> >>>> conversion to work. > >>> >>>> > >>> >>>> > >>> >>>> I need you to provide some explanation on what you're doing.. if > >>> this > >>> >>>> was a way to hack a bug your saw or if this is just the way you're > >>> >>>> using it. > >>> >>>> > >>> >>>> > >>> >>>> if this is how you're actually using, I would suggest you either use > >>> >>>> the proper APIs or if you're doing some hack for performance > >>> >>>> consideration you have to set the message in a better way the > >>> >>>> converters would understand... > >>> >>>> > >>> >>>> On Mon, Jul 18, 2022 at 9:38 AM Clebert Suconic > >>> >>>> <clebert.suco...@gmail.com> wrote: > >>> >>>> > > >>> >>>> > Are you using the LargeServerMessageImpl on your client, or that > >>> was > >>> >>>> > just a "hack" to reproduce your issue. > >>> >>>> > > >>> >>>> > > >>> >>>> > LargeServerMessageImpl was not meant to be used on the client. > >>> But if > >>> >>>> > you're doing that just to show something you faced in production > >>> it's > >>> >>>> > ok.. but using the LargeServerMessageImpl to send a Client > >>> Message is > >>> >>>> > a big not for me. > >>> >>>> > > >>> >>>> > The only reason we do that is for server to server transfer. > >>> >>>> > > >>> >>>> > On Mon, Jul 18, 2022 at 7:39 AM Andy Yar <andyya...@gmail.com> > >>> wrote: > >>> >>>> > > > >>> >>>> > > Hello, > >>> >>>> > > Yes, the 2.23.1 test instance was freshly created. > >>> >>>> > > > >>> >>>> > > I've attached a test as a part of ARTEMIS-3897 > >>> >>>> > > <https://issues.apache.org/jira/browse/ARTEMIS-3897>. I hope > >>> it helps. > >>> >>>> > > > >>> >>>> > > > >>> >>>> > > On Sat, Jul 16, 2022 at 5:21 PM Clebert Suconic < > >>> clebert.suco...@gmail.com> > >>> >>>> > > wrote: > >>> >>>> > > > >>> >>>> > > > Did you start with fresh data on 2.23.1. > >>> >>>> > > > > >>> >>>> > > > If you did please provide a self enclosing test reproducing > >>> your issue. > >>> >>>> > > > > >>> >>>> > > > On Fri, Jul 15, 2022 at 4:29 AM Andy Yar <andyya...@gmail.com> > >>> wrote: > >>> >>>> > > > > >>> >>>> > > > > A quick test using 2.23.1 results in the same error payload > >>> being > >>> >>>> > > > received: > >>> >>>> > > > > > >>> >>>> > > > > "Message(address='test', durable=True, priority=4, > >>> >>>> > > > > annotations=AnnotationDict({symbol('x-opt-jms-dest'): > >>> byte(0), > >>> >>>> > > > > symbol('x-opt-jms-msg-type'): byte(0)}), > >>> >>>> > > > > properties={'JMSXDeliveryCount': None, '_AMQ_LARGE_SIZE': > >>> 67}, > >>> >>>> > > > > body='Conversion to AMQP error: Error reading in > >>> simpleString, > >>> >>>> > > > > length=1330464032 is greater than readableBytes=62')" > >>> >>>> > > > > > >>> >>>> > > > > Best regards > >>> >>>> > > > > > >>> >>>> > > > > On Thu, Jul 14, 2022 at 9:18 PM Clebert Suconic > >>> >>>> > > > > <clebert.suco...@gmail.com> wrote: > >>> >>>> > > > > > > >>> >>>> > > > > > There's been a few fixes in AMQP Large message. > >>> >>>> > > > > > > >>> >>>> > > > > > > >>> >>>> > > > > > More prominently a fix with the JDBC implementation > >>> between 2.17 and > >>> >>>> > > > > HEAD. > >>> >>>> > > > > > > >>> >>>> > > > > > > >>> >>>> > > > > > I would definitely recommend you to upgrade. > >>> >>>> > > > > > > >>> >>>> > > > > > On Thu, Jul 14, 2022 at 10:26 AM Justin Bertram < > >>> jbert...@apache.org> > >>> >>>> > > > > wrote: > >>> >>>> > > > > > > > >>> >>>> > > > > > > I would expect this to work. Can you try this on the > >>> latest release > >>> >>>> > > > > (i.e. > >>> >>>> > > > > > > 2.23.1) [1]? > >>> >>>> > > > > > > > >>> >>>> > > > > > > If it still doesn't work please open a Jira with all > >>> the details > >>> >>>> > > > > necessary > >>> >>>> > > > > > > to reproduce the problem. An actual test-case would be > >>> ideal. > >>> >>>> > > > > > > > >>> >>>> > > > > > > Thanks! > >>> >>>> > > > > > > > >>> >>>> > > > > > > > >>> >>>> > > > > > > Justin > >>> >>>> > > > > > > > >>> >>>> > > > > > > [1] > >>> https://activemq.apache.org/components/artemis/download/ > >>> >>>> > > > > > > > >>> >>>> > > > > > > On Thu, Jul 14, 2022 at 9:20 AM Andy Yar < > >>> andyya...@gmail.com> > >>> >>>> > > > wrote: > >>> >>>> > > > > > > > >>> >>>> > > > > > > > Hello, > >>> >>>> > > > > > > > Sending a message as a Large Message via Core > >>> protocol and > >>> >>>> > > > receiving > >>> >>>> > > > > > > > it via AMQP on Artemis 2.17.0 ends with receiving the > >>> following > >>> >>>> > > > > string > >>> >>>> > > > > > > > as message body: > >>> >>>> > > > > > > > > >>> >>>> > > > > > > > "Conversion to AMQP error: Error reading in > >>> simpleString, length=y > >>> >>>> > > > is > >>> >>>> > > > > > > > greater than readableBytes=x" > >>> >>>> > > > > > > > > >>> >>>> > > > > > > > Broker lists the message as Type: Default and Large: > >>> True. > >>> >>>> > > > > > > > > >>> >>>> > > > > > > > The receiver is basically Qpid's: > >>> >>>> > > > > > > > > >>> >>>> > > > > > > > > >>> >>>> > > > > > >>> >>>> > > > > >>> https://github.com/apache/qpid-proton/blob/main/python/examples/queue_browser.py > >>> >>>> > > > > > > > > >>> >>>> > > > > > > > The doc > >>> >>>> > > > > > > > > >>> >>>> > > > > > >>> >>>> > > > > >>> https://activemq.apache.org/components/artemis/documentation/2.17.0/large-messages.html > >>> >>>> > > > > > > > states both Core and AMQP support Large Message. Is > >>> this approach > >>> >>>> > > > > > > > valid? > >>> >>>> > > > > > > > > >>> >>>> > > > > > > > Thanks > >>> >>>> > > > > > > > > >>> >>>> > > > > > > > > >>> >>>> > > > > > > >>> >>>> > > > > > > >>> >>>> > > > > > > >>> >>>> > > > > > -- > >>> >>>> > > > > > Clebert Suconic > >>> >>>> > > > > > >>> >>>> > > > -- > >>> >>>> > > > Clebert Suconic > >>> >>>> > > > > >>> >>>> > > >>> >>>> > > >>> >>>> > > >>> >>>> > -- > >>> >>>> > Clebert Suconic > >>> >>>> > >>> >>>> > >>> >>>> > >>> >>>> -- > >>> >>>> Clebert Suconic > >>> > > >>> > -- > >>> > Clebert Suconic > >>> > >>> > >>> > >>> -- > >>> Clebert Suconic > >>> > >> -- > >> Clebert Suconic > >> > > -- > > Clebert Suconic > > > -- > Clebert Suconic -- Clebert Suconic
