That gap seems a long time. Have you checked GC logs around the timeframe?
From: Jai Bheemsen Rao Dhanwada <jaibheem...@gmail.com> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org> Date: Monday, June 1, 2020 at 3:52 PM To: "user@cassandra.apache.org" <user@cassandra.apache.org> Subject: Cassandra Bootstrap Sequence Message from External Sender Hello Team, When I am bootstrapping/restarting a Cassandra Node, there is a delay between gossip settle and port opening. Can someone please explain me where this delay is configured and can this be changed? I don't see any information in the logs In my case if you see there is a ~3 minutes delay and this increases if I increase the #of tables and #of nodes and DC. INFO [main] 2020-05-31 23:51:07,554 Gossiper.java:1692 - Waiting for gossip to settle... INFO [main] 2020-05-31 23:51:15,555 Gossiper.java:1723 - No gossip backlog; proceeding INFO [main] 2020-05-31 23:54:06,867 NativeTransportService.java:70 - Netty using native Epoll event loop INFO [main] 2020-05-31 23:54:06,913 Server.java:155 - Using Netty Version: [netty-buffer=netty-buffer-4.0.44.Final.452812a, netty-codec=netty-codec-4.0.44.Final.452812a, netty-codec-haproxy=netty-codec-haproxy-4.0.44.Final.452812a, netty-codec-http=netty-codec-http-4.0.44.Final.452812a, netty-codec-socks=netty-codec-socks-4.0.44.Final.452812a, netty-common=netty-common-4.0.44.Final.452812a, netty-handler=netty-handler-4.0.44.Final.452812a, netty-tcnative=netty-tcnative-1.1.33.Fork26.142ecbb, netty-transport=netty-transport-4.0.44.Final.452812a, netty-transport-native-epoll=netty-transport-native-epoll-4.0.44.Final.452812a, netty-transport-rxtx=netty-transport-rxtx-4.0.44.Final.452812a, netty-transport-sctp=netty-transport-sctp-4.0.44.Final.452812a, netty-transport-udt=netty-transport-udt-4.0.44.Final.452812a] INFO [main] 2020-05-31 23:54:06,913 Server.java:156 - Starting listening for CQL clients on /x.x.x.x:9042 (encrypted)... Also during this 3 minutes delay, I am losing all my metrics from the C* nodes(basically the metrics are not returned within 10s). Can someone please help me understand the delay here? Cassandra Version: 3.11.3 Metrics: Using telegraf to collect metrics.
