Re: understanding consumer rebalance trigger(s)

2017-03-03 Thread Damian Guy
Hi Jon, On 0.10.0.1 it means that your processing is taking longer than the configured session.timeout.ms. "The timeout used to detect failures when using Kafka's " + "group management facilities. When a consumer's heartbeat is not received within the session timeout, " + "the bro

understanding consumer rebalance trigger(s)

2017-03-02 Thread Jon Yeargers
Im wondering what the parameters are to instantiate a consumer rebalance. I have a topic that turns roughly 50K / minute across 6 partitions. Each is serviced by a separate dockerized consumer. Roughly every 8-12 min this goes into a rebalance that may take up to a minute. When it returns it often