This is indeed because gRPC does exponential connection backoff. To reconnect immediate means we would have to know when the service is back up. This is a problem because checking if the service is up is sometimes expensive, sometimes even enough to keep the service down.
At most it will wait 2 minutes after which it will re attach. I cannot comment on whether those parameters were broken at that release. On Friday, July 14, 2017 at 12:36:32 PM UTC-7, Bruno Bowden wrote: > > NOTE: we are currently using gRPC v1.1.4. For various reasons, it's more > challenging for us to upgrade but please advise if this is a fixed issue in > a more recent version or any workaround that you'd suggest. Thanks for all > your work on gRPC. > > I'm trying to solve an issue with a persistent gRPC client where it takes > 20 seconds to reconnect after a "Connect Failed" event. This is useful > during testing when the service is repeatedly brought up and down, while > the client is left running. See the console logs below that shows what > repeatedly occurs. After bringing down the service, the client reports > "Deadline Exceeded" - it has a 500ms and these failures are expected. When > the service is restarted, including responding to other grpc clients, the > already running grpc client continues to fail until 20 second after the > original "Connect Failed". > > My impression is that this is part of the exponential backoff and retry. > If I restart the persistent client, then it works immediately. I've tried > playing with the grpc.max_reconnect_backoff_ms > and grpc.initial_reconnect_backoff_ms settings without any success. > > # GRPC SERVICE BROUGHT DOWN > I0714 12:13:17.325851 1793 xxxx.cc:118] gRPC status: Connect Failed > I0714 12:13:18.826437 1794 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:19.826522 1792 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:20.826452 1795 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:21.826382 1798 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:22.826511 1797 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:23.826552 1791 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:24.826550 1795 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:25.826663 1793 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:26.826717 1794 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:27.826637 1792 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:28.826462 1798 xxxx.cc:118] gRPC status: Deadline Exceeded > # GRPC SERVICE SUCCESSFULLY RESTARTED > # expecting reconnect to work immediately > I0714 12:13:29.826494 1795 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:30.826251 1791 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:31.827332 1792 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:32.826256 1796 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:33.826165 1795 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:34.827574 1791 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:35.826184 1792 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:36.825944 1798 xxxx.cc:118] gRPC status: Deadline Exceeded > I0714 12:13:37.326781 1795 xxxx.cc:125] gRPC status: ok > I0714 12:13:38.326337 1797 xxxx.cc:125] gRPC status: ok > I0714 12:13:39.326292 1794 xxxx.cc:125] gRPC status: ok > I0714 12:13:40.326501 1798 xxxx.cc:125] gRPC status: ok > > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/cd863687-5cb7-46c3-965f-adb19f77299a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
