Re: Consumer Assignment documentation - StreamPartitionAssignment.java

2016-02-05 Thread Guozhang Wang
Hi Vamsi, Kafka Streams uses the new 0.9.0 Java consumer as its underlying client, which do not have ZK dependency anymore. Instead it relies on a coordinator on the server side, one assigned for each consumer group to assign partitions among themselves. You can find more details here: https://c

Re: Consumer Assignment documentation - StreamPartitionAssignment.java

2016-02-05 Thread Achanta Vamsi Subhash
Thanks Guozhang. Is the below true? 1. There is no central controller for the consuemrs which assigns the partitions to the active consumers. 2. Hence, the consumers each fight for the allocation by registering the watch on active consumers (/ids path in zk). But how do the consumers finally arr

Re: Consumer Assignment documentation - StreamPartitionAssignment.java

2016-02-02 Thread Guozhang Wang
Hi Vamsi, Consumer partition assignment is always within a single group. You can find the java docs for the PartitionAssignor interface here: https://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/RangeAssignor.html StreamPartitionAssignor is one implementation of the