Is there anything specific to for in GC logs? b/w this delay happens always whenever I bootstrap the node or restart a C* process.
I don't believe it's a GC issue and correction from initial question, it's not just bootstrap, but every restart of C* process is causing this. On Mon, Jun 1, 2020 at 3:22 PM Reid Pinchback <rpinchb...@tripadvisor.com> wrote: > That gap seems a long time. Have you checked GC logs around the timeframe? > > > > *From: *Jai Bheemsen Rao Dhanwada <jaibheem...@gmail.com> > *Reply-To: *"user@cassandra.apache.org" <user@cassandra.apache.org> > *Date: *Monday, June 1, 2020 at 3:52 PM > *To: *"user@cassandra.apache.org" <user@cassandra.apache.org> > *Subject: *Cassandra Bootstrap Sequence > > > > *Message from External Sender* > > Hello Team, > > > > When I am bootstrapping/restarting a Cassandra Node, there is a delay > between gossip settle and port opening. Can someone please explain me where > this delay is configured and can this be changed? I don't see any > information in the logs > > > > In my case if you see there is a ~3 minutes delay and this increases if I > increase the #of tables and #of nodes and DC. > > > > INFO [main] 2020-05-31 23:51:07,554 Gossiper.java:1692 - Waiting for > gossip to settle... > > *INFO [main] 2020-05-31 23:51:15,555 Gossiper.java:1723 - No gossip > backlog; proceeding *INFO [main] 2020-05-31 23:54:06,867 > NativeTransportService.java:70 - Netty using native Epoll event loop > INFO [main] 2020-05-31 23:54:06,913 Server.java:155 - Using Netty > Version: [netty-buffer=netty-buffer-4.0.44.Final.452812a, > netty-codec=netty-codec-4.0.44.Final.452812a, > netty-codec-haproxy=netty-codec-haproxy-4.0.44.Final.452812a, > netty-codec-http=netty-codec-http-4.0.44.Final.452812a, > netty-codec-socks=netty-codec-socks-4.0.44.Final.452812a, > netty-common=netty-common-4.0.44.Final.452812a, > netty-handler=netty-handler-4.0.44.Final.452812a, > netty-tcnative=netty-tcnative-1.1.33.Fork26.142ecbb, > netty-transport=netty-transport-4.0.44.Final.452812a, > netty-transport-native-epoll=netty-transport-native-epoll-4.0.44.Final.452812a, > netty-transport-rxtx=netty-transport-rxtx-4.0.44.Final.452812a, > netty-transport-sctp=netty-transport-sctp-4.0.44.Final.452812a, > netty-transport-udt=netty-transport-udt-4.0.44.Final.452812a] > *INFO [main] 2020-05-31 23:54:06,913 Server.java:156 - Starting listening > for CQL clients on /x.x.x.x:9042 (encrypted)...* > > > > Also during this 3 minutes delay, I am losing all my metrics from the C* > nodes(basically the metrics are not returned within 10s). > > > > Can someone please help me understand the delay here? > > > > Cassandra Version: 3.11.3 > > Metrics: Using telegraf to collect metrics. >
