NOTE: we are currently using gRPC v1.1.4. For various reasons, it's more challenging for us to upgrade but please advise if this is a fixed issue in a more recent version or any workaround that you'd suggest. Thanks for all your work on gRPC.
I'm trying to solve an issue with a persistent gRPC client where it takes 20 seconds to reconnect after a "Connect Failed" event. This is useful during testing when the service is repeatedly brought up and down, while the client is left running. See the console logs below that shows what repeatedly occurs. After bringing down the service, the client reports "Deadline Exceeded" - it has a 500ms and these failures are expected. When the service is restarted, including responding to other grpc clients, the already running grpc client continues to fail until 20 second after the original "Connect Failed". My impression is that this is part of the exponential backoff and retry. If I restart the persistent client, then it works immediately. I've tried playing with the grpc.max_reconnect_backoff_ms and grpc.initial_reconnect_backoff_ms settings without any success. # GRPC SERVICE BROUGHT DOWN I0714 12:13:17.325851 1793 xxxx.cc:118] gRPC status: Connect Failed I0714 12:13:18.826437 1794 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:19.826522 1792 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:20.826452 1795 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:21.826382 1798 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:22.826511 1797 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:23.826552 1791 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:24.826550 1795 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:25.826663 1793 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:26.826717 1794 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:27.826637 1792 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:28.826462 1798 xxxx.cc:118] gRPC status: Deadline Exceeded # GRPC SERVICE SUCCESSFULLY RESTARTED # expecting reconnect to work immediately I0714 12:13:29.826494 1795 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:30.826251 1791 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:31.827332 1792 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:32.826256 1796 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:33.826165 1795 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:34.827574 1791 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:35.826184 1792 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:36.825944 1798 xxxx.cc:118] gRPC status: Deadline Exceeded I0714 12:13:37.326781 1795 xxxx.cc:125] gRPC status: ok I0714 12:13:38.326337 1797 xxxx.cc:125] gRPC status: ok I0714 12:13:39.326292 1794 xxxx.cc:125] gRPC status: ok I0714 12:13:40.326501 1798 xxxx.cc:125] gRPC status: ok -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/f895ac08-c012-42da-8f49-aa13e4eb5a9f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
