Hi Vamsi,
Kafka Streams uses the new 0.9.0 Java consumer as its underlying client,
which do not have ZK dependency anymore. Instead it relies on a coordinator
on the server side, one assigned for each consumer group to assign
partitions among themselves.
You can find more details here:
https://c
Thanks Guozhang.
Is the below true?
1. There is no central controller for the consuemrs which assigns the
partitions to the active consumers.
2. Hence, the consumers each fight for the allocation by registering the
watch on active consumers (/ids path in zk).
But how do the consumers finally arr
Hi Vamsi,
Consumer partition assignment is always within a single group. You can find
the java docs for the PartitionAssignor interface here:
https://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/RangeAssignor.html
StreamPartitionAssignor is one implementation of the
Hi all,
I am going through the partition assignment logic in the Kafka new
consumer. Is there any documentation for the logic implemented? It would be
great if some one can write the algorithm for consumer partition assignment
(within a group and across groups).
--
Regards
Vamsi Subhash
