Why did you move to 2.20 and not the latest ?
On Wed, Nov 23, 2022 at 2:37 AM Thomas Wahlberg <thomas.wahlb...@nextup.fi> wrote: > We have now upgraded one environment to 2.20.0 and notice some difference > in the logs. It might be pointing to a certain queue that receives over 100 > kb sized messages. 2.20.0 is still crashing in similar fashion. There's a > huge IOWAIT reported from our monitoring tools during the crash/restart and > wrapper.log is filled with hundreds, if not thousands, of errors like this, > all during 1-2 second time period: > > INFO | jvm 2 | 2022/11/23 08:02:12 | WARN > [org.apache.activemq.artemis.core.server] AMQ222061: Client connection > failed, clearing up resources for session > 90a07ca5-6a2e-11ed-9aa1-767acc7119a2 > INFO | jvm 2 | 2022/11/23 08:02:12 | WARN > [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources > for session 90a07ca5-6a2e-11ed-9aa1-767acc7119a2 > INFO | jvm 2 | 2022/11/23 08:02:12 | WARN > [org.apache.activemq.artemis.core.server] AMQ222061: Client connection > failed, clearing up resources for session > 90ce4367-6a2e-11ed-9aa1-767acc7119a2 > INFO | jvm 2 | 2022/11/23 08:02:12 | WARN > [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources > for session 90ce4367-6a2e-11ed-9aa1-767acc7119a2 > > > > > On Tue, Nov 8, 2022 at 2:27 PM Thomas Wahlberg <thomas.wahlb...@nextup.fi> > wrote: > > > Thanks for your reply. We can indeed update to 2.20.0 and we're moving > > forward with this now. Newer versions seemed to have new possible issues > so > > we'll try with 2.20.0 for starters. > > > > On Tue, Nov 8, 2022 at 12:52 PM Roskvist Anton <anton.roskv...@volvo.com > > > > wrote: > > > >> Hello Thomas, > >> > >> I might be mistaken but I believe this has been fixed in broker version > >> 2.20.0... it might be a good idea to update the broker version > regardless > >> since there has been quite a few bug fixes, added features and > improvements > >> made since 2.19.0. Be sure to follow the "Versions" documentation if > >> upgrading: > >> > https://activemq.apache.org/components/artemis/documentation/latest/versions.html > >> > >> If that's not an option you could try to set the > >> "critical-analyzer-policy" to "LOG" instead to keep it from killing the > >> broker. This has the drawback that if there are any actual critical > errors > >> in the broker it will not self-terminate. Also note that I am just > another > >> user of the broker so any advise regarding this should be taken with a > >> spoonful of "read the docs ". > >> > >> Br, > >> Anton > >> > >> -----Original Message----- > >> From: Thomas Wahlberg <thomas.wahlb...@nextup.fi> > >> Sent: den 8 november 2022 11:27 > >> To: users@activemq.apache.org > >> Subject: Artemis: Artemis crashing and restarting randomly due to IO > >> timeout > >> > >> [You don't often get email from thomas.wahlb...@nextup.fi. Learn why > >> this is important at https://aka.ms/LearnAboutSenderIdentification ] > >> > >> CAUTION: This email originated from outside of the organization. If > >> suspicious, please report it. > >> > >> Hi, > >> we're using artemis 2.19.0 and we've noticed this is happening randomly. > >> Sometimes it happens twice per day, sometimes once a month. It doesn't > >> seem to be related to any cron jobs or any server load. Sometimes during > >> the night, sometimes during the day. We're using Python with Artemis. > The > >> server monitoring tools don't report anything special regarding IO or > >> IOWAIT during the time of the crash. Can you help us in any way? Any > >> pointers on what to look for? > >> > >> Thank you in advance! > >> > >> Blow are some details: > >> Artemis: 2.19.0 > >> OS: Centos 7 > >> openjdk version "1.8.0_131" > >> OpenJDK Runtime Environment (build 1.8.0_131-b11) OpenJDK 64-Bit Server > >> VM (build 25.131-b11, mixed mode) > >> > >> INFO | jvm 6 | ***************** | WARN > >> [org.apache.activemq.artemis.utils.critical.CriticalMeasure] Component > >> org.apache.activemq.artemis.core.io.buffer.TimedBuffer is expired on > path 4 > >> INFO | jvm 6 | ***************** | ERROR > >> [org.apache.activemq.artemis.core.server] AMQ224079: The process for the > >> virtual machine will be killed, as component > >> org.apache.activemq.artemis.core.io.buffer.TimedBuffer@726de316 is not > >> responsive > >> INFO | jvm 6 | ***************** | WARN > >> [org.apache.activemq.artemis.core.server] AMQ222199: Thread dump: > >> > >> > ******************************************************************************* > >> INFO | jvm 6 | ***************** | Complete Thread dump > >> INFO | jvm 6 | ***************** | "Reference Handler" Id=2 > RUNNABLE > >> INFO | jvm 6 | ***************** | at > >> java.base@11.0.14 > >> /java.lang.ref.Reference.waitForReferencePendingList(Native > >> Method) > >> INFO | jvm 6 | ***************** | at java.base@11.0.14 > >> /java.lang.ref.Reference.processPendingReferences(Reference.java:241) > >> INFO | jvm 6 | ***************** | at java.base@11.0.14 > >> /java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213) > >> INFO | jvm 6 | ***************** | > >> > >> ... > >> > >> INFO | jvm 6 | ***************** | > >> INFO | jvm 6 | ***************** | > >> > >> > =============================================================================== > >> INFO | jvm 6 | ***************** | End Thread dump > >> INFO | jvm 6 | ***************** | > >> > >> > ******************************************************************************* > >> INFO | jvm 6 | ***************** | > >> ERROR | wrapper | ***************** | JVM exited unexpectedly. > >> STATUS | wrapper | ***************** | Reloading Wrapper > configuration... > >> STATUS | wrapper | ***************** | Launching a JVM... > >> > >> -- > >> > >> Thomas Wahlberg > >> Chief Technology Officer > >> Quriiri / NextUp Oy > >> Puh. +358407031990 > >> thomas.wahlb...@quriiri.fi > >> > >> This email message (including its attachments) is confidential and may > >> contain privileged information and is intended solely for the use of the > >> individual and/or entity to whom it is addressed. If you are not the > >> intended recipient of this e-mail you may not disseminate, distribute or > >> copy this e-mail (including its attachments), or any part thereof. If > this > >> e-mail is received in error, please notify the sender immediately by > return > >> e-mail and make sure that this e-mail (including its attachments), and > all > >> copies thereof, are immediately deleted from your system. Please further > >> note that when you communicate with us via email or visit our website we > >> process your personal data. See our privacy policy for more information > >> about how we process it: https://www.volvogroup.com/en-en/privacy.html > >> > > > > > > -- > > > > Thomas Wahlberg > > Chief Technology Officer > > Quriiri / NextUp Oy > > Puh. +358407031990 > > thomas.wahlb...@quriiri.fi > > > > > -- > > Thomas Wahlberg > Chief Technology Officer > Quriiri / NextUp Oy > Puh. +358407031990 > thomas.wahlb...@quriiri.fi > -- Clebert Suconic
