ShuLian1984 opened a new issue, #7839: URL: https://github.com/apache/rocketmq/issues/7839
### Before Creating the Bug Report - [X] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq/discussions). - [X] I have searched the [GitHub Issues](https://github.com/apache/rocketmq/issues) and [GitHub Discussions](https://github.com/apache/rocketmq/discussions) of this repository and believe that this is not a duplicate. - [X] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ. ### Runtime platform environment ``` # os NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/"; BUG_REPORT_URL="https://bugs.centos.org/"; CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" # docker Client: Docker Engine - Community Version: 19.03.12 API version: 1.40 Go version: go1.13.10 Git commit: 48a66213fe Built: Mon Jun 22 15:46:54 2020 OS/Arch: linux/amd64 Experimental: false Server: Docker Engine - Community Engine: Version: 19.03.12 API version: 1.40 (minimum version 1.12) Go version: go1.13.10 Git commit: 48a66213fe Built: Mon Jun 22 15:45:28 2020 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.2.13 GitCommit: 7ad184331fa3e55e52b890ea95e65ba581ae3429 runc: Version: 1.0.0-rc10 GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd docker-init: Version: 0.18.0 GitCommit: fec3683 # docker-compose version docker-compose version 1.24.1, build 4667896b docker-py version: 3.7.3 CPython version: 3.6.8 OpenSSL version: OpenSSL 1.1.0j 20 Nov 2018 ``` ### RocketMQ version apache/rocketmq:5.1.1 (IMAGE ID: b44b9cc99a52) ``` # docker-compose.yml version: '2' services: rocketmq-nameserv: container_name: rocketmq-nameserv image: apache/rocketmq:5.1.1 restart: always environment: - TZ=Asia/Shanghai - JAVA_OPT_EXT=-server -Xms300m -Xmx300m -Xmn150m command: [ "bin/mqnamesrv" ] network_mode: host rocketmq-broker: container_name: rocketmq-broker image: apache/rocketmq:5.1.1 restart: always environment: - TZ=Asia/Shanghai - JAVA_OPT_EXT=-server -Xms1024m -Xmx1024m -Xmn150m command: [ "bin/mqbroker", "-n", "172.31.0.128:9876", "-c", "conf/broker.properties" ] volumes: - ./conf/broker.properties:/opt/rocketmq/conf/broker.properties:ro - ./store:/opt/rocketmq/store:rw network_mode: host # broker.properties brokerClusterName=YCluster brokerName=broker-a brokerIP1=10.10.2.32 brokerId=0 deleteWhen=04 fileReservedTime=48 brokerRole=ASYNC_MASTER flushDiskType=ASYNC_FLUSH listenPort=10911 autoCreateTopicEnable=true defaultTopicQueueNums=4 autoCreateSubscriptionGroup=true mapedFileSizeCommitLog=1073741824 mapedFileSizeConsumeQueue=300000 diskMaxUsedSpaceRatio=88 destroyMapedFileIntervalForcibly=120000 redeleteHangedFileInterval=120000 storePathRootDir=/opt/rocketmq/store storePathCommitLog=/opt/rocketmq/store/commitlog storePathConsumeQueue=/opt/rocketmq/store/consumequeue storePathIndex=/opt/rocketmq/store/index storePathCheckpoint=/opt/rocketmq/store/checkpoint abortFile=/opt/rocketmq/store/abort maxMessageSize=65536 ``` For security reasons, all IP addresses are virtual namesrv use SLB address :172.31.0.128 broker use ECS External Network Address: 10.10.2.32 ### JDK Version _No response_ ### Describe the Bug loop error in broker.log ``` 2024-01-16 17:50:30 INFO ReadSocketService - join thread[FlowMonitor], elapsed time: 0ms, join time:90000ms 2024-01-16 17:50:30 INFO WriteSocketService - makestop thread[WriteSocketService] 2024-01-16 17:50:30 INFO WriteSocketService - makestop thread[ReadSocketService] 2024-01-16 17:50:30 INFO WriteSocketService - Try to shutdown service thread:FlowMonitor started:false lastThread:Thread[FlowMonitor,5,] 2024-01-16 17:50:30 INFO DefaultAcceptSocketService - Try to start service thread:FlowMonitor started:false lastThread:null 2024-01-16 17:50:30 INFO DefaultAcceptSocketService - Start service thread:FlowMonitor started:true lastThread:Thread[FlowMonitor,5,main] 2024-01-16 17:50:30 INFO DefaultAcceptSocketService - Try to start service thread:ReadSocketService started:false lastThread:null 2024-01-16 17:50:30 INFO DefaultAcceptSocketService - Start service thread:ReadSocketService started:true lastThread:Thread[ReadSocketService,5,main] 2024-01-16 17:50:30 INFO DefaultAcceptSocketService - Try to start service thread:WriteSocketService started:false lastThread:null 2024-01-16 17:50:30 INFO DefaultAcceptSocketService - Start service thread:WriteSocketService started:true lastThread:Thread[WriteSocketService,5,main] 2024-01-16 17:50:30 INFO ReadSocketService - makestop thread[ReadSocketService] 2024-01-16 17:50:30 INFO ReadSocketService - makestop thread[WriteSocketService] 2024-01-16 17:50:30 INFO ReadSocketService - Try to shutdown service thread:FlowMonitor started:true lastThread:Thread[FlowMonitor,5,main] 2024-01-16 17:50:30 INFO ReadSocketService - shutdown thread[FlowMonitor] interrupt=true 2024-01-16 17:50:30 ERROR FlowMonitor - Interrupted java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at org.apache.rocketmq.common.CountDownLatch2.await(CountDownLatch2.java:114) at org.apache.rocketmq.common.ServiceThread.waitForRunning(ServiceThread.java:117) at org.apache.rocketmq.store.ha.FlowMonitor.run(FlowMonitor.java:37) at java.lang.Thread.run(Thread.java:748) 2024-01-16 17:50:30 INFO ReadSocketService - join thread[FlowMonitor], elapsed time: 1ms, join time:90000ms 2024-01-16 17:50:30 INFO WriteSocketService - makestop thread[WriteSocketService] 2024-01-16 17:50:30 INFO WriteSocketService - makestop thread[ReadSocketService] 2024-01-16 17:50:30 INFO WriteSocketService - Try to shutdown service thread:FlowMonitor started:false lastThread:Thread[FlowMonitor,5,] 2024-01-16 17:50:32 INFO DefaultAcceptSocketService - Try to start service thread:FlowMonitor started:false lastThread:null 2024-01-16 17:50:32 INFO DefaultAcceptSocketService - Start service thread:FlowMonitor started:true lastThread:Thread[FlowMonitor,5,main] 2024-01-16 17:50:32 INFO DefaultAcceptSocketService - Try to start service thread:ReadSocketService started:false lastThread:null 2024-01-16 17:50:32 INFO DefaultAcceptSocketService - Start service thread:ReadSocketService started:true lastThread:Thread[ReadSocketService,5,main] 2024-01-16 17:50:32 INFO DefaultAcceptSocketService - Try to start service thread:WriteSocketService started:false lastThread:null 2024-01-16 17:50:32 INFO DefaultAcceptSocketService - Start service thread:WriteSocketService started:true lastThread:Thread[WriteSocketService,5,main] 2024-01-16 17:50:32 INFO ReadSocketService - makestop thread[ReadSocketService] 2024-01-16 17:50:32 INFO ReadSocketService - makestop thread[WriteSocketService] 2024-01-16 17:50:32 INFO ReadSocketService - Try to shutdown service thread:FlowMonitor started:true lastThread:Thread[FlowMonitor,5,main] 2024-01-16 17:50:32 INFO ReadSocketService - shutdown thread[FlowMonitor] interrupt=true 2024-01-16 17:50:32 ERROR FlowMonitor - Interrupted java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at org.apache.rocketmq.common.CountDownLatch2.await(CountDownLatch2.java:114) at org.apache.rocketmq.common.ServiceThread.waitForRunning(ServiceThread.java:117) at org.apache.rocketmq.store.ha.FlowMonitor.run(FlowMonitor.java:37) at java.lang.Thread.run(Thread.java:748) 2024-01-16 17:50:32 INFO ReadSocketService - join thread[FlowMonitor], elapsed time: 1ms, join time:90000ms 2024-01-16 17:50:32 INFO WriteSocketService - makestop thread[WriteSocketService] 2024-01-16 17:50:32 INFO WriteSocketService - makestop thread[ReadSocketService] 2024-01-16 17:50:32 INFO WriteSocketService - Try to shutdown service thread:FlowMonitor started:false lastThread:Thread[FlowMonitor,5,] 2024-01-16 17:50:33 INFO DefaultAcceptSocketService - Try to start service thread:FlowMonitor started:false lastThread:null 2024-01-16 17:50:33 INFO DefaultAcceptSocketService - Start service thread:FlowMonitor started:true lastThread:Thread[FlowMonitor,5,main] 2024-01-16 17:50:33 INFO DefaultAcceptSocketService - Try to start service thread:ReadSocketService started:false lastThread:null 2024-01-16 17:50:33 INFO DefaultAcceptSocketService - Start service thread:ReadSocketService started:true lastThread:Thread[ReadSocketService,5,main] 2024-01-16 17:50:33 INFO DefaultAcceptSocketService - Try to start service thread:WriteSocketService started:false lastThread:null 2024-01-16 17:50:33 INFO DefaultAcceptSocketService - Start service thread:WriteSocketService started:true lastThread:Thread[WriteSocketService,5,main] 2024-01-16 17:50:33 INFO ReadSocketService - makestop thread[ReadSocketService] 2024-01-16 17:50:33 INFO ReadSocketService - makestop thread[WriteSocketService] 2024-01-16 17:50:33 INFO ReadSocketService - Try to shutdown service thread:FlowMonitor started:true lastThread:Thread[FlowMonitor,5,main] 2024-01-16 17:50:33 INFO ReadSocketService - shutdown thread[FlowMonitor] interrupt=true 2024-01-16 17:50:33 ERROR FlowMonitor - Interrupted java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at org.apache.rocketmq.common.CountDownLatch2.await(CountDownLatch2.java:114) at org.apache.rocketmq.common.ServiceThread.waitForRunning(ServiceThread.java:117) at org.apache.rocketmq.store.ha.FlowMonitor.run(FlowMonitor.java:37) at java.lang.Thread.run(Thread.java:748) ``` This error was recorded in 34 days and a 6.2GB log was generated. ### Steps to Reproduce Namesrv provides services externally through SLB, the broker address use ECS External Network Address, and connect SLB IP to register. ### What Did You Expect to See? what happen and how to fix it? ### What Did You See Instead? a single node deployment start the FlowMonitor thread. ### Additional Context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
