Hi, how large is the message? Have you checked max_allowed_packet configuration parameter, specifically https://dev.mysql.com/doc/mysql-replication-excerpt/8.0/en/replication-features-max-allowed-packet.html ?
-- Vilius -----Original Message----- From: Stephen Baker <stephen.ba...@rmssoftwareinc.com> Sent: Thursday, September 15, 2022 8:31 PM To: users@activemq.apache.org Subject: Re: Critical error sending large messages to mysql To follow up, I have tracked this down to a bug in connector/j. I am working on a simple proof of concept and a ticket for them now. For any artemis mysql users that are curious, the problem only happens when using a jdbc:mysql:replication:// connection string. It seems that updateBlob is broken in that case. From: Stephen Baker <stephen.ba...@rmssoftwareinc.com> Date: Thursday, September 15, 2022 at 10:57 AM To: users@activemq.apache.org <users@activemq.apache.org> Subject: Critical error sending large messages to mysql I don’t have exact reproduction steps yet (otherwise I would have filed an issue), but on Artemis 2.22 using a mysql backed journal our QA can reliably send messages that crash the server: ``` 2022-09-15 10:42:24,843 WARN [org.apache.activemq.artemis.journal] AMQ142021: Error on IO callback, class com.sun.proxy.$Proxy21 cannot be cast to class com.mysql.cj.jdbc.ClientPreparedStatement (com.sun.proxy.$Proxy21 and com.mysql.cj.jdbc.ClientPreparedStatement are in unnamed module of loader java.net.URLClassLoader @3a5ed7a6) 2022-09-15 10:42:24,845 ERROR [org.apache.activemq.artemis.core.server] AMQ222010: Critical IO Error, shutting down the server. file=org.apache.activemq.artemis.jdbc.store.file.JDBCSequentialFile@3452c05f, message=Error writing to JDBC file.: java.lang.ClassCastException: class com.sun.proxy.$Proxy21 cannot be cast to class com.mysql.cj.jdbc.ClientPreparedStatement (com.sun.proxy.$Proxy21 and com.mysql.cj.jdbc.ClientPreparedStatement are in unnamed module of loader java.net.URLClassLoader @3a5ed7a6) at com.mysql.cj.jdbc.result.UpdatableResultSet.syncUpdate(UpdatableResultSet.java:1143) [mysql-connector-java-8.0.28.jar:8.0.28] at com.mysql.cj.jdbc.result.UpdatableResultSet.updateBlob(UpdatableResultSet.java:1283) [mysql-connector-java-8.0.28.jar:8.0.28] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [java.base:] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [java.base:] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [java.base:] at java.base/java.lang.reflect.Method.invoke(Method.java:566) [java.base:] at com.mysql.cj.jdbc.ha.MultiHostConnectionProxy$JdbcInterfaceProxy.invoke(MultiHostConnectionProxy.java:107) [mysql-connector-java-8.0.28.jar:8.0.28] at com.sun.proxy.$Proxy22.updateBlob(Unknown Source) at org.apache.commons.dbcp2.DelegatingResultSet.updateBlob(DelegatingResultSet.java:1411) [commons-dbcp2-2.7.0.jar:2.7.0] at org.apache.commons.dbcp2.DelegatingResultSet.updateBlob(DelegatingResultSet.java:1411) [commons-dbcp2-2.7.0.jar:2.7.0] at org.apache.commons.dbcp2.DelegatingResultSet.updateBlob(DelegatingResultSet.java:1411) [commons-dbcp2-2.7.0.jar:2.7.0] at org.apache.activemq.artemis.jdbc.store.file.JDBCSequentialFileFactoryDriver.writeToFile(JDBCSequentialFileFactoryDriver.java:284) [artemis-jdbc-store-2.22.0.jar:2.22.0] at org.apache.activemq.artemis.jdbc.store.file.JDBCSequentialFile.internalWrite(JDBCSequentialFile.java:167) [artemis-jdbc-store-2.22.0.jar:2.22.0] at org.apache.activemq.artemis.jdbc.store.file.JDBCSequentialFile.internalWrite(JDBCSequentialFile.java:203) [artemis-jdbc-store-2.22.0.jar:2.22.0] at org.apache.activemq.artemis.jdbc.store.file.JDBCSequentialFile.lambda$scheduleWrite$1(JDBCSequentialFile.java:214) [artemis-jdbc-store-2.22.0.jar:2.22.0] at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.22.0.jar:] at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.22.0.jar:] at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) [artemis-commons-2.22.0.jar:] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [java.base:] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [java.base:] at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.22.0.jar:] 2022-09-15 10:42:24,848 WARN [org.apache.activemq.artemis.journal] AMQ142021: Error on IO callback, Error writing to JDBC file. 2022-09-15 10:42:24,854 WARN [org.apache.activemq.artemis.core.client] AMQ212004: Failed to connect to server. 2022-09-15 10:42:24,909 WARN [org.apache.activemq.artemis.journal] AMQ142021: Error on IO callback, class com.sun.proxy.$Proxy21 cannot be cast to class com.mysql.cj.jdbc.ClientPreparedStatement (com.sun.proxy.$Proxy21 and com.mysql.cj.jdbc.ClientPreparedStatement are in unnamed module of loader java.net.URLClassLoader @3a5ed7a6) 2022-09-15 10:42:24,926 WARN [org.apache.activemq.artemis.journal] AMQ142021: Error on IO callback, Error writing to JDBC file. ``` As you can see, whatever the original problem was it is hidden by the ClassCastException in Artemis. When I learn more I will file an issue (unless the problem has been resolved in a later version) and hopefully a fix. Stephen E. Baker [EXTERNAL]: This email originated from outside of Rave Mobile Safety. Do not click links or open attachments unless you recognize the sender and know the content is safe.