s/back-office/backoff/
Final note: although the goal here is not to resolve contention (as in the
aws article), I think we do still want a relatively smooth rate of
reconnects across all clients to avoid storm spikes. Full Jitter does that.
I expect that narrower jitter bands will lead to more clu
For some discussion of jitter and exponential back-office, I found this
article useful:
https://www.awsarchitectureblog.com/2015/03/backoff.html
My initial POC used the "Full Jitter" approach described therein. Equal
Jitter is good too, and may perform a little better. It is random
distribution b
Thanks for the feedback Gwen and Colin. I agree that the original formula
was not intuitive. I updated it to include a max jitter as was suggested. I
also updated the config name to include `ms`:
https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=69408222&selectedPageVersio
Thanks for the KIP, Ismael & Dana! This could be pretty important for
avoiding congestion collapse when there are a lot of clients.
It seems like a good idea to keep the "ms" suffix, like we have with
"reconnect.backoff.ms". So maybe we should use
"reconnect.backoff.max.ms"? In general unitless
This is a great suggestion. I like how we just do it by default instead of
making it a choice users need to figure out.
Avoiding connection storms is great.
One concern. If I understand the formula for effective maximum backoff
correctly, then with default maximum of 1000ms and default backoff of
Hi all,
Dana Powers posted a PR a while back for exponential backoff for broker
reconnect attempts. Because it adds a config, a KIP is required and Dana
seems to be busy so I posted it:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-
144%3A+Exponential+backoff+for+broker+reconnect+attempts
