> On May 18, 2015, 9:46 p.m., Guozhang Wang wrote: > > core/src/main/scala/kafka/coordinator/PartitionAssignor.scala, lines 87-89 > > <https://reviews.apache.org/r/34273/diff/1/?file=961395#file961395line87> > > > > I feel this logic's irregular assignment result may make it a bit > > harder to reason for debugging. For example if you have three consumers > > (1,2,3) with two topics each with three partitions, and consumer 1/2 > > subscribe to topic 1/2 while consumer 3 only subscribe to 2. The result > > would be: > > > > t1p0: c1 > > t1p1: c2 > > t1p2: c1 > > t2p0: c2 > > t2p0: c1 > > t2p0: c3 > > > > Instead it may be clearer to have: > > > > t1p0: c1 > > t1p1: c2 > > t1p2: c1 > > t2p0: c1 > > t2p1: c2 > > t2p2: c3 > > > > So I suggest grouping the consumers by subscribed topics, and for each > > group do the round-robin.
If I understood you correctly, you're suggesting we group consumers on a per-topic basis and reset your roundrobin consumer assignor position every topic. This would basically become range. The specific partitions being assigned would differ from range, but the ending quantity per consumer per topic would be the same. This also means the left-most consumers are going to get the most partitions overall. As an extreme example, imagine a group of 10 consumers subscribed to 10 topics, each topic with 1 partition. Reseting the consumer assignor means the leftmost consumer will get all 10 partitions. I think reseting the consumer assignor will in general bias assignments towards the left. Were you referring to a non-per-topic grouping? - Onur ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34273/#review84223 ----------------------------------------------------------- On May 15, 2015, 3:46 p.m., Onur Karaman wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/34273/ > ----------------------------------------------------------- > > (Updated May 15, 2015, 3:46 p.m.) > > > Review request for kafka. > > > Bugs: KAFKA-2196 > https://issues.apache.org/jira/browse/KAFKA-2196 > > > Repository: kafka > > > Description > ------- > > roundrobin doesn't need to make all consumers have identical topic > subscriptions. > > todo: > - run this and range through some simulations > > > Diffs > ----- > > core/src/main/scala/kafka/coordinator/PartitionAssignor.scala > 106982286ce7a9e4f0e9722da2812e3a8e7a6cc3 > core/src/test/scala/unit/kafka/coordinator/PartitionAssignorTest.scala > ba6d5cd85b89214247209d974701eb6c9eb1e2b2 > > Diff: https://reviews.apache.org/r/34273/diff/ > > > Testing > ------- > > > Thanks, > > Onur Karaman > >
