That’s the full output with regular logging levels. I can reproduce at will so I have enabled trace level logging and pasted the result in https://issues.apache.org/jira/browse/ARTEMIS-4045
Let’s take further discussion there? From: Clebert Suconic <clebert.suco...@gmail.com> Date: Wednesday, October 12, 2022 at 9:10 PM To: users@activemq.apache.org <users@activemq.apache.org> Subject: Re: Mirror compatibility across versions is this the actual trace? or you cut some to post here? Just puzzled by skipDelivery calling performAck.. artemis-test-artemis-1-m-1 | at org.apache.activemq.artemis.protocol.amqp.connect.mirror.AMQPMirrorControllerTarget.lambda$performAck$2(AMQPMirrorControllerTarget.java:377) [artemis-amqp-protocol-2.25.0.jar:2.25.0] artemis-test-artemis-1-m-1 | at org.apache.activemq.artemis.core.server.impl.QueueImpl$2.skipDelivery(QueueImpl.java:1203) [artemis-server-2.25.0.jar:2.25.0] can you post the full stack if this is not it? it definitely needs fixing... I'm investigating it. On Wed, Oct 12, 2022 at 6:05 PM Stephen Baker <stephen.ba...@rmssoftwareinc.com> wrote: > > Having updated both sides to 2.25 I’m seeing this error in the logs, is it a > concern that warrants further investigation? > > artemis-test-artemis-1-m-1 | 2022-10-12 22:01:43,632 ERROR > [org.apache.activemq.artemis.core.server] AMQ224041: Failed to deliver: > java.lang.IllegalStateException: this method requires to be called within the > handler, use the executor > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.protocol.amqp.proton.handler.ProtonHandler.requireHandler(ProtonHandler.java:210) > [artemis-amqp-protocol-2.25.0.jar:2.25.0] > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.protocol.amqp.proton.AMQPConnectionContext.requireInHandler(AMQPConnectionContext.java:197) > [artemis-amqp-protocol-2.25.0.jar:2.25.0] > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.protocol.amqp.proton.ProtonAbstractReceiver.settle(ProtonAbstractReceiver.java:185) > [artemis-amqp-protocol-2.25.0.jar:2.25.0] > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.protocol.amqp.connect.mirror.AMQPMirrorControllerTarget$ACKMessageOperation.run(AMQPMirrorControllerTarget.java:125) > [artemis-amqp-protocol-2.25.0.jar:2.25.0] > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.protocol.amqp.connect.mirror.AMQPMirrorControllerTarget.performAck(AMQPMirrorControllerTarget.java:388) > [artemis-amqp-protocol-2.25.0.jar:2.25.0] > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.protocol.amqp.connect.mirror.AMQPMirrorControllerTarget.lambda$performAck$2(AMQPMirrorControllerTarget.java:377) > [artemis-amqp-protocol-2.25.0.jar:2.25.0] > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.core.server.impl.QueueImpl$2.skipDelivery(QueueImpl.java:1203) > [artemis-server-2.25.0.jar:2.25.0] > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.core.server.impl.QueueImpl.doInternalPoll(QueueImpl.java:2932) > [artemis-server-2.25.0.jar:2.25.0] > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:2991) > [artemis-server-2.25.0.jar:2.25.0] > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4250) > [artemis-server-2.25.0.jar:2.25.0] > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:56) > [artemis-commons-2.25.0.jar:] > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) > [artemis-commons-2.25.0.jar:] > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:67) > [artemis-commons-2.25.0.jar:] > artemis-test-artemis-1-m-1 | at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > [java.base:] > artemis-test-artemis-1-m-1 | at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > [java.base:] > artemis-test-artemis-1-m-1 | at > org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) > [artemis-commons-2.25.0.jar:] > > > From: Stephen Baker <stephen.ba...@rmssoftwareinc.com> > Date: Wednesday, October 12, 2022 at 4:43 PM > To: users@activemq.apache.org <users@activemq.apache.org> > Subject: Re: Mirror compatibility across versions > I set up some docker images in this configuration as a preliminary step. One > oddity: > > Configure 2.25 side not to run the reaper > Send message to 2.25 side > Observe that after expiry the message shows up in the expiry queue on the > 2.20 side, but not on the 2.25 side, the message is removed from the original > queue on both sides. > > If the message is originally sent to the 2.20 side it shows up in both queues > as expected. > > There’s probably a reason for it, but I didn’t expect this change. I thought > that we would continue to see the old bugs until both sides were updated. > > From: Clebert Suconic <clebert.suco...@gmail.com> > Date: Tuesday, October 11, 2022 at 3:24 PM > To: users@activemq.apache.org <users@activemq.apache.org> > Subject: Re: Mirror compatibility across versions > Yeah.. something like that... not necessarily in there though. but a > similar test. > > On Tue, Oct 11, 2022 at 1:44 PM Stephen Baker > <stephen.ba...@rmssoftwareinc.com> wrote: > > > > Ok, I agree based on a cursory reading of that patch. The extra ackReason > > defaults to normal in one direction and isn’t read in the other direction. > > Killed, replaced, and expired being interpreted as normal just means that > > the 2.20 bugs will persist until both sides are updated. > > > > I’ll test it out with different version docker containers. I suppose as far > > as writing tests you mean something like the MultiVersionReplicaTest. > > > > Stephen E. Baker > > > > > > From: Clebert Suconic <clebert.suco...@gmail.com> > > Date: Tuesday, October 11, 2022 at 12:59 PM > > To: users@activemq.apache.org <users@activemq.apache.org> > > Subject: Re: Mirror compatibility across versions > > In theory it should work. > > > > > > Only change that might break compatibility is > > https://github.com/apache/activemq-artemis/commit/68f6d8263d8c795722805f0e4d6939e7a8b9ed48 > > which is ARTEMIS-3743 / ARTEMIS-3766 Use ACKReason on Mirror to > > determine target operations and fixing Delivering statistics on Mirror > > > > > > I tried to not break compatibility, but I just realized we should add > > a test to validate compatibility between mirrors. > > > > > > > > so, I will say it should be compatible, but I would test it before > > doing it in the real system. > > > > > > > > if you are willing to contribute to a compatibility test :) > > > > On Tue, Oct 11, 2022 at 10:06 AM Stephen Baker > > <stephen.ba...@rmssoftwareinc.com> wrote: > > > > > > We are planning our production upgrade from 2.20 to 2.25. These upgrades > > > involve a loss of service in the window between stopping the live and > > > when the backup instance becomes ready to process messages. > > > > > > I was wondering if the mirror protocol is expected to be compatible > > > between those versions. If so we could upgrade our cold site, and then > > > wait for a planned failover to avoid any additional down time. I know > > > that quite a bit of work was done by Clebert in 2.24 so I was hoping he > > > could weigh in. > > > > > > Stephen E Baker > > > > > > > > -- > > Clebert Suconic > > [EXTERNAL]: This email originated from outside of Rave Mobile Safety. Do > > not click links or open attachments unless you recognize the sender and > > know the content is safe. > > > > -- > Clebert Suconic > [EXTERNAL]: This email originated from outside of Rave Mobile Safety. Do not > click links or open attachments unless you recognize the sender and know the > content is safe. > [EXTERNAL]: This email originated from outside of Rave Mobile Safety. Do not > click links or open attachments unless you recognize the sender and know the > content is safe. -- Clebert Suconic [EXTERNAL]: This email originated from outside of Rave Mobile Safety. Do not click links or open attachments unless you recognize the sender and know the content is safe.
