Re: GC pauses and rebalance failures

2014-04-14 Thread David DeMaagd
Deliberate variation of the retry/backoff parameters on a per-client basis is probably an even more complicated work-around than bumping up the session timeout. I've never tried it because it doesn't really address the probable root cause (GC causing client stalls, zookeeper server dropping con

Re: GC pauses and rebalance failures

2014-04-14 Thread Clark Breyman
Thanks David. One hypothesis we have is that using different rebalance.backoff.ms settings for the different ConsumerConnectors on the same JVM will keep them from synchronizing their rebalance attempts enough so that one can finish. On Mon, Apr 14, 2014 at 12:58 PM, David DeMaagd wrote: > Corre

Re: GC pauses and rebalance failures

2014-04-14 Thread David DeMaagd
Correct - heavy client GC leads to numerous problems. There's two things you can do: 1) Tune the client JVM better to get GC to a more reasonable level 2) Increase the zookeeper session timeout value (this is generally a work-around for #1, but it can buy you time to dig into it) -- Dave D