Thanks Guozhang, I agree that it seems to be a common reassignment strategy that should be one of the pre-defined strategies. Do you have a Jira ticket for this specific case, and/or a Jira ticket to add user defined partitions.assignment.strategies that I can watch?
/ Olof On 10/06/2015 14:35, "Guozhang Wang" <wangg...@gmail.com> wrote: >Hi Olof, > >Yes we have plans to allow user defined partitions.assignment strategy to >pass in to the consumer coordinator; I am not sure if this feature will >not >be available in the first release of the new consumer in 0.8.3 though. >Currently users still have to choose one from the server-defined strategy >upon registering themselves. > >On the other hand, I think "rebalance with a minimal number of >reassignment" should be quite a common reassignment strategy, and I think >it is possible to just add it into the server-defined strategies list. > >Guozhang > > >On Wed, Jun 10, 2015 at 9:32 AM, Johansson, Olof < >olof.johans...@thingworx.com> wrote: > >> Is it possible to have a consumer rebalancing partition assignment >> strategy that will rebalance with a minimal number of reassignments? >> >> Per the "Kafka 0.9 Consumer Rewrite Design" it should be possible to >> define my own partition assignment strategy: >> "partition.assignment.strategies - may take a comma separated list of >> properties that map the strategy's friendly name to to the class that >> implements the strategy. This is used for any strategy implemented by >>the >> user and released to the Kafka cluster. By default, Kafka will include a >> set of strategies that can be used by the consumer." >> >> Is there a Jira ticket that tracks adding user defined >> partitions.assignment.strategies? In the latest source, range, and >> roundrobin are still the only possible values (hard-coded). >> >> I assume that any user implemented strategy would have to implement the >> PartitionAssignor trait. If so, by naively looking at the 0.8.3 source, >>a >> strategy that should do a minimal number of partition reassignments >>would >> need the ConsumerMetaData. That's not currently available in the >> PartitionAssignor contract - assign(topicsPerConsumer, >>partitionsPerTopic). >> Have there been any discussion to change the contract to pass >> ConsumerMetaData? >> > > > >-- >-- Guozhang
