An easy way to check what is going on is with
Artemis data print (Preferably with the broker stopped) You can then check what is not being asked and why it became paging. You could maybe replicate a similar load pattern on a test? Like sending many messages a second instead of 2 per minute? I wonder what is being retained not acked leaving many messages in memory. On Mon, Jan 23, 2017 at 5:55 AM Francesco PADOVANI < francesco.padov...@bticino.it> wrote: > > > > > > > > > > > > > > > > > Hello, > > > I'm using Apache Artemis as MQTT broker for our IOT projects. > > > It's a clean Artemis installation of version 1.5.1., on a server (CentOS > 7) which has 2 vCPU, 8 GB of RAM (4GB of Heap Space dedicated to Artemis) > and 50 GB of SSD data disk. > > > After the installation of artemis Broker we started to test it with about > 10-15 clients constantly connected, 150 subscribed topics and an average of > 2 messages per minute per client. I think these are not huge numbers, right? > > > For the few days after the installation, all was good and the broker > worked perfectly: it was fast and reliable. But day by day performances > have decreased and after about 10 days of usage it is became almost > unusable: due to resources consumption. The following > > is my "top" situation on the server: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *top - 08:59:50 up 2 days, 22:09, 1 user, load average: 2.56, 2.88, > 3.00Tasks: 92 total, 1 running, 91 sleeping, 0 stopped, 0 > zombie%Cpu(s): 99.5 us, 0.5 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 > si, 0.0 stKiB Mem : 7747288 total, 290968 free, 4697976 used, 2758344 > buff/cacheKiB Swap: 0 total, 0 free, 0 used. 2734768 > avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM > TIME+ COMMAND 4352 mqtt 20 0 7043380 4.348g 15192 S 200.0 58.9 > 6153:54 > java > 1 root 20 0 128096 5176 2420 S 0.0 0.1 0:04.01 > systemd* > > > > > > > > > > Now I can see the broker starting to page on disk almost always. For sure > it's a wrong configuration of ours. Currently, it seems that very old > address/queue (and the related retained messages) are always kept, making > memory and cpu consuption growing > > more and more. And even after a restart, the broker takes so much to get > up. > > Before to become reachable, it starts to make many management operations > like, for example, retrieve data from paging, etc.. But I also see the > broker that starts to register old topics and queues we don't need. How to > clean them? How to make a topic/queue > > expire? > > > > > > > > Inside my broker.xml I did set up the parameter last-value-queue=true, > thinking that this was the problem ....but it doesn't solve. Or better: > probably it's me that I've not understood the correct meaning of the > parameter. > > I did check also the clients' parameters when they connect to the broker, > to be sure they don't set, for example, clean-session = false (saying the > broker to keep all messages also when a client disconnects). But they make > it in the right way. The only > > thing is that they don't specify a client-id. They connect by using a > username/password and certificate (over tls). So, every time a client > connects, Artemis automatically provide a random client-id for it (if I > understood well). > > > Attached you can find my broker.xml configuration file: it's pretty much > the same default created during the installation procedure, but for the > acceptors (which I've customized for my MQTT purpose) and the addition of > parameter > > last-value-queue = true inside the address-setting > > section. > > > > > > > > > > Please: some of you could help me? How I have to configure my broker > instance to understand and solve these performance issues? > > > > > > > > Thanks in advance > > > > > > > > Francesco > > > > > > > > > > > > > > > > > > > > ------------------------------ > > > > > Ce message, ainsi que tous les fichiers joints à ce message, peuvent > contenir des informations sensibles et/ ou confidentielles ne devant pas > être divulguées. Si vous n'êtes pas le destinataire de ce message (ou que > vous recevez ce message par erreur), nous > > vous remercions de le notifier immédiatement à son expéditeur, et de > détruire ce message. Toute copie, divulgation, modification, utilisation ou > diffusion, non autorisée, directe ou indirecte, de tout ou partie de ce > message, est strictement interdite. > > > > > > > This e-mail, and any document attached hereby, may contain confidential > and/or privileged information. If you are not the intended recipient (or > have received this e-mail in error) please notify the sender immediately > and destroy this e-mail. Any unauthorized, > > direct or indirect, copying, disclosure, distribution or other use of the > material or parts thereof is strictly forbidden. > > > > > > > > >
