Perhaps we could add this to the next 1.5.3, if you cherry-pick the fix in 1.x
On Mon, Feb 6, 2017 at 10:55 AM, Martyn Taylor <mtay...@redhat.com> wrote: > Hi Francesco, > > It appears that there is a bug in the MessageImpl code that is not properly > setting the endOfBodyPosition when the server restarts (reads from journal) > and then there's a subsequent message copy (i.e. retain). I've enabled > correct pointer setting and added the fix to MQTT. > > I'm running the full suite atm to ensure this doesn't affect other areas of > the code. Should have this fix done shortly. > > Thanks for all the info, > Martyn > > On Thu, Jan 12, 2017 at 10:13 AM, Francesco PADOVANI < > francesco.padov...@bticino.it> wrote: > >> Hi Martyn, >> >> I've just opened a JIRA related to this issue: it's the tkt ARTEMIS-917 ( >> https://issues.apache.org/jira/browse/ARTEMIS-917). Of course it's >> currently unassigned: let me know if I have to assign it to someone in >> particular. >> >> >> A Thing That may help you (even if I understand you've already figured out >> where the problem lies): a developer of ours did put the Artemis code under >> debug and he has noticed that the message is yes, written correctly on the >> file system, but then it is read in a wrong way. It seems a matter of >> pointer position, which is kept to the end of metadata instead of the end >> of message body while reading it. The involved classes should be the >> followings: >> >> >> AbstractJournalStorageManager.java >> >> --> this is the class that probably doesn't correctly values the pointer. >> >> >> MessageImpl.java >> >> --> this (if I understand well) is the class containing the "decode" >> method used to read the message. A question: do you think the last bold >> line, below, could be a solution? >> >> >> public void decode(final ActiveMQBuffer buff) { >> >> int start = buff.readerIndex(); >> >> endOfBodyPosition = buff.readInt(); >> >> endOfMessagePosition = buff.getInt(endOfBodyPosition - >> BUFFER_HEADER_SPACE + start); >> >> int length = endOfMessagePosition - BUFFER_HEADER_SPACE; >> >> buffer.setIndex(0, BUFFER_HEADER_SPACE); >> >> buffer.writeBytes(buff, start, length); >> >> decode(); >> >> buff.readerIndex(start + length); >> >> buff.writerIndex(start + endOfBodyPosition); >> >> } >> >> >> >> Thanks a lot for your help and your patience. >> >> >> Francesco >> >> >> >> ________________________________ >> From: Martyn Taylor <mtay...@redhat.com> >> Sent: Wednesday, January 11, 2017 3:53:00 PM >> To: users@activemq.apache.org >> Subject: Re: MQTT retained messages with weird characters >> >> It looks to me like the full message buffer (not just the body) is getting >> returned to the client. Clebert I think you're right in that we do some >> hackery to handle retained messages. I suspect this is where the error is. >> >> Francesco could you create a JIRA with some reproducer steps and I'll aim >> to take a look. >> >> Thanks >> >> On Wed, Jan 11, 2017 at 2:03 PM, Clebert Suconic < >> clebert.suco...@gmail.com> >> wrote: >> >> > the MQTT ProtocolManager does some playing with retaining, perhaps >> > there's something wrong with reloading from journal on retaining. I >> > was wondering if you could send us a test showing the issue with MQTT >> > so we could see if there's anything specific to your test while using >> > MQTT. >> > >> > On Wed, Jan 11, 2017 at 4:22 AM, Francesco PADOVANI >> > <francesco.padov...@bticino.it> wrote: >> > > Unfortunately no (not yet). >> > > >> > > Anyway, it's a clean Artemis installation of version 1.5.1, made by >> > > following the user manual instructions at >> > > https://activemq.apache.org/artemis/docs/1.5.1/using-server.html. >> > > >> > > The installation was done by using a dedicated user and also the >> artemis >> > > java process runs with this dedicated user. Attached you can find my >> > > broker.xml configuration file: it's pretty much the same default >> created >> > > during the installation procedure, but for the acceptors (which I've >> > > customized for my MQTT purpose) and the addition of parameter >> > > "<last-value-queue>true</last-value-queue>" inside the address-setting >> > > section (but I tried with and without it and my issue persists). >> > > >> > > The only other configuration I changed is the heap size dedicated to >> the >> > > process, by setting "-Xms4096M -Xmx4096M" among the JAVA_ARGS in the >> > > artemis.profile file. >> > > >> > > >> > > The platform where Artemis broker is installed is: >> > > >> > > >> > > - Machine: >> > > >> > > It's an EC2 instance on AWS cloud of type m4.large: 2 vCPU, 8G RAM and >> > SSD >> > > storage. >> > > >> > > >> > > - OS: >> > > >> > > CentOS Linux release 7.3.1611 (Core) - x86_64 >> > > >> > > >> > > - JVM: >> > > >> > > java version "1.8.0_111" >> > > Java(TM) SE Runtime Environment (build 1.8.0_111-b14) >> > > Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode) >> > > >> > > >> > > - Distro package version of libaio: >> > > >> > > Arch : x86_64 >> > > Version : 0.3.109 >> > > Release : 13.el7 >> > > >> > > >> > > It seems like the problem relates the Persistence (File Journal?). >> Until >> > the >> > > broker keeps the retained messages in ram it works. When it needs to >> > recover >> > > the retained messages from the file system, clients receive them with >> > weird >> > > chars. >> > > And I suspect the same issue appears when RAM is full and messages >> start >> > to >> > > be paged... but this case is a bit more difficult to test in a >> systematic >> > > way. >> > > >> > > >> > > Francesco >> > > >> > > >> > > >> > > ________________________________ >> > > From: Clebert Suconic <clebert.suco...@gmail.com> >> > > Sent: Tuesday, January 10, 2017 7:43 PM >> > > To: users@activemq.apache.org >> > > Subject: Re: MQTT retained messages with weird characters >> > > >> > > Do you have a working test you can share? >> > > >> > > On Tue, Jan 10, 2017 at 12:07 PM, Francesco PADOVANI >> > > <francesco.padov...@bticino.it> wrote: >> > >> Hello, >> > >> >> > >> I'm trying the Apache Artemis Broker (ver. 1.5.1) for MQTT protocol. >> > >> >> > >> Exactly I'm testing the retained feature for messages of MQTT >> protocol. >> > >> >> > >> While the broker is up it seems to work well: >> > >> >> > >> a) a client publishes a retained message to a specific topic >> > >> >> > >> b) any client which then subscribes that topic receives the retained >> > >> message (any time it re-subscribes the topic it receives the last >> > retained >> > >> message ...perfect!) >> > >> >> > >> But it happens a strange thing when I restart the broker. And I mean I >> > >> restart the broker without any change on configuration: simply I make >> a >> > >> "artemis-service stop" and "artemis-service start". After The broker >> is >> > up >> > >> and running, if a client subscribes the previous topic it still >> > receives the >> > >> retained message, but with weird characters appended to it. For new >> > retained >> > >> messages published it work well again... but the previous ones (before >> > the >> > >> broker restart) are sent all with these weird characters. E.g., the >> > >> following is a retained message published before the broker restart: >> > >> >> > >> "test 25 for a retained message 20170110" >> > >> >> > >> And this is exactly how clients get it before the broker restart. >> > >> >> > >> Instead, after broker restart, the same retained message is received >> by >> > >> clients in the following way: >> > >> >> > >> test 25 for a retained message 20170110? >> > >> ? >> 6$sys.mqtt.retain..cro.test5 ? >> > >> &mqtt.message.retain ? mqtt.qos.level >> > >> >> > >> >> > >> Is this my bad configuration (but I don't know ehere)? Or anything >> else? >> > >> Or what? >> > >> >> > >> Someone can help me? >> > >> >> > >> >> > >> Thanks in advance. >> > >> >> > >> >> > >> Francesco >> > >> >> > >> ________________________________ >> > >> >> > >> Ce message, ainsi que tous les fichiers joints à ce message, peuvent >> > >> contenir des informations sensibles et/ ou confidentielles ne devant >> pas >> > >> être divulguées. Si vous n'êtes pas le destinataire de ce message (ou >> > que >> > >> vous recevez ce message par erreur), nous vous remercions de le >> notifier >> > >> immédiatement à son expéditeur, et de détruire ce message. Toute >> copie, >> > >> divulgation, modification, utilisation ou diffusion, non autorisée, >> > directe >> > >> ou indirecte, de tout ou partie de ce message, est strictement >> > interdite. >> > >> >> > >> >> > >> This e-mail, and any document attached hereby, may contain >> confidential >> > >> and/or privileged information. If you are not the intended recipient >> (or >> > >> have received this e-mail in error) please notify the sender >> > immediately and >> > >> destroy this e-mail. Any unauthorized, direct or indirect, copying, >> > >> disclosure, distribution or other use of the material or parts thereof >> > is >> > >> strictly forbidden. >> > > >> > > >> > > >> > > -- >> > > Clebert Suconic >> > > >> > > ________________________________ >> > > >> > > Ce message, ainsi que tous les fichiers joints à ce message, peuvent >> > > contenir des informations sensibles et/ ou confidentielles ne devant >> pas >> > > être divulguées. Si vous n'êtes pas le destinataire de ce message (ou >> que >> > > vous recevez ce message par erreur), nous vous remercions de le >> notifier >> > > immédiatement à son expéditeur, et de détruire ce message. Toute copie, >> > > divulgation, modification, utilisation ou diffusion, non autorisée, >> > directe >> > > ou indirecte, de tout ou partie de ce message, est strictement >> interdite. >> > > >> > > >> > > This e-mail, and any document attached hereby, may contain confidential >> > > and/or privileged information. If you are not the intended recipient >> (or >> > > have received this e-mail in error) please notify the sender >> immediately >> > and >> > > destroy this e-mail. Any unauthorized, direct or indirect, copying, >> > > disclosure, distribution or other use of the material or parts thereof >> is >> > > strictly forbidden. >> > >> > >> > >> > -- >> > Clebert Suconic >> > >> >> ________________________________ >> >> Ce message, ainsi que tous les fichiers joints à ce message, peuvent >> contenir des informations sensibles et/ ou confidentielles ne devant pas >> être divulguées. Si vous n'êtes pas le destinataire de ce message (ou que >> vous recevez ce message par erreur), nous vous remercions de le notifier >> immédiatement à son expéditeur, et de détruire ce message. Toute copie, >> divulgation, modification, utilisation ou diffusion, non autorisée, directe >> ou indirecte, de tout ou partie de ce message, est strictement interdite. >> >> >> This e-mail, and any document attached hereby, may contain confidential >> and/or privileged information. If you are not the intended recipient (or >> have received this e-mail in error) please notify the sender immediately >> and destroy this e-mail. Any unauthorized, direct or indirect, copying, >> disclosure, distribution or other use of the material or parts thereof is >> strictly forbidden. >> -- Clebert Suconic
