On Tue, Feb 13, 2018 at 2:59 PM, Subhash Sriram <subhash.sri...@gmail.com> wrote:
Hey Xavier, > > Within a consumer group, you can only have as many consumers as you have > partitions in a topic. If you start more consumers than partitions within > the same group, they will just be idle. > Excellent, I am reading the docs but cannot anticipate some scenarios with certainty. I am thinking about situations involving two machines, for fault-tolerance, deploy procedures without downtime, etc. Let's say we have machines A and B, running consumers that belong to the same consumer group. I guess the broker is then going to send messages only to the one that opened the TCP connection first, right? I correct, let's assume it is A. The broker is going to accept a connection from B, but there are no more partitions to balance, so B is idle and sending heartbeats just fine. If there is a network issue between the broker and A, and the socket of B is up, Kafka is then going to assign the unique partition to B automatically right? In that case, if the hiccup is solved one minute later and the connection in A is restored, then A is going to become the idle instance? If that is all correct, the only issue I foresee is committing offsets. If the client commits offsets every 10 seconds and the network cut happens, I believe the instance taking over is going to receive duplicated messages in the general case, right?
