Re: Best approach to frequently restarting consumer process

2016-12-10 Thread Ewen Cheslack-Postava
Consumer groups aren't going to handle 'let it crash' particularly well (and really any session-based services, but particularly consumer groups since a single failure affects the entire group). That said, 'let it crash' doesn't necessarily have to mean 'don't try to clean up at all'. The consumer

Re: Best approach to frequently restarting consumer process

2016-12-07 Thread Harald Kirsch
With 'restart' I mean a 'let it crash' setup (as promoted by Erlang and Akka, e.g. http://doc.akka.io/docs/akka/snapshot/intro/what-is-akka.html). The consumer gets in trouble due to an OOM or a runaway computation or whatever that we want to preempt somehow. It crashes or gets killed external

Re: Best approach to frequently restarting consumer process

2016-12-06 Thread Gwen Shapira
Can you clarify what you mean by "restart"? If you call consumer.close() and consumer.subscribe() you will definitely trigger a rebalance. It doesn't matter if its "same consumer knocking", we already rebalance when you call consumer.close(). Since we want both consumer.close() and consumer.subsc