there was a thread accessing paging as the server was stopping, what caused this exception.
I would try with latest release on paging as page-cache was removed. On Mon, Jul 24, 2023 at 7:26 AM Shiv Kumar Dixit <shivkumardi...@eurofins.com.invalid> wrote: > > We see below error in artemis.log and it is stopping the broker. Broker > version is 2.26.0. Before broker stops, we see many entries where paging on > different addresses are starting and stopping. When paging related logs on > different addresses are done, we see message 'Cannot find iter to remove: > java.lang.RuntimeException: Cannot find iter to remove' is logged and post > that broker stops. Any input/lead will be helpful. > > --------------Start of log------------------------- > 2023-07-17T12:22:12.999Z INFO --- [Thread-2 (Activ] > org.apache.activemq.artemis.core.server : AMQ224108: Stopped paging on > address 'XXXX'; size=0 bytes (0 messages); maxSize=-1 bytes (-1 messages); > globalSize=1035418769 bytes (76908 messages); globalMaxSize=1048576000 bytes > (-1 messages); > .............. > ............. > .............. > ............... > > 2023-07-17T12:22:39.721Z WARN --- [Thread-1 (Activ] > org.apache.activemq.artemis.utils.actors.OrderedExecutor : Cannot find iter > to remove: java.lang.RuntimeException: Cannot find iter to remove > at > org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl$CursorIterator.next(PageSubscriptionImpl.java:1353) > [artemis-server-2.26.0.jar:2.26.0] > at > org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl$CursorIterator.tryNext(PageSubscriptionImpl.java:1492) > [artemis-server-2.26.0.jar:2.26.0] > at > org.apache.activemq.artemis.core.server.impl.QueueImpl$ExpiryScanner.run(QueueImpl.java:2522) > [artemis-server-2.26.0.jar:2.26.0] > at > org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:56) > [artemis-commons-2.26.0.jar:] > at > org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) > [artemis-commons-2.26.0.jar:] > at > org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:67) > [artemis-commons-2.26.0.jar:] > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > [java.base:] > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > [java.base:] > at > org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) > [artemis-commons-2.26.0.jar:] > Caused by: java.lang.IllegalStateException: Cannot find iter to remove > at > org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl$CursorIterator.initPage(PageSubscriptionImpl.java:1295) > [artemis-server-2.26.0.jar:2.26.0] > at > org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl$CursorIterator.internalGetNext(PageSubscriptionImpl.java:1473) > [artemis-server-2.26.0.jar:2.26.0] > at > org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl$CursorIterator.moveNext(PageSubscriptionImpl.java:1378) > [artemis-server-2.26.0.jar:2.26.0] > at > org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl$CursorIterator.next(PageSubscriptionImpl.java:1348) > [artemis-server-2.26.0.jar:2.26.0] > ... 8 more > Caused by: java.lang.IllegalStateException: Cannot find iter to remove > at > org.apache.activemq.artemis.utils.collections.LinkedListImpl.removeIter(LinkedListImpl.java:370) > [artemis-commons-2.26.0.jar:] > at > org.apache.activemq.artemis.utils.collections.LinkedListImpl$Iterator.close(LinkedListImpl.java:527) > [artemis-commons-2.26.0.jar:] > at > org.apache.activemq.artemis.core.paging.cursor.impl.PageSubscriptionImpl$CursorIterator.initPage(PageSubscriptionImpl.java:1286) > [artemis-server-2.26.0.jar:2.26.0] > ... 11 more > > 2023-07-17T12:22:53.266Z INFO --- [Thread-27] org.apache.activemq.artemis : > AMQ241005: Stopping embedded web server > 2023-07-17T12:22:53.317Z INFO --- [Thread-27] > io.hawt.web.auth.AuthenticationFilter : Destroying hawtio authentication > filter > 2023-07-17T12:22:53.323Z INFO --- [Thread-27] io.hawt.HawtioContextListener > : Destroying hawtio services > 2023-07-17T12:22:53.356Z INFO --- [Thread-27] > org.apache.activemq.hawtio.plugin.PluginContextListener : Destroyed > artemis-plugin plugin > 2023-07-17T12:22:53.373Z INFO --- [Thread-27] > org.apache.activemq.hawtio.branding.PluginContextListener : Destroyed > activemq-branding plugin > 2023-07-17T12:22:53.405Z INFO --- [Thread-27] org.apache.activemq.artemis : > AMQ241006: Stopped embedded web server > 2023-07-17T12:22:53.405Z INFO --- [Thread-27] > org.apache.activemq.artemis.core.server : AMQ221002: Apache ActiveMQ Artemis > Message Broker version 2.26.0 [66ff429e-f80d-11eb-b357-0050568f9608] stopped, > uptime 6 minutes > --------------End of log------------------------- > -- Clebert Suconic
