Hi Olof, Yes we have plans to allow user defined partitions.assignment strategy to pass in to the consumer coordinator; I am not sure if this feature will not be available in the first release of the new consumer in 0.8.3 though. Currently users still have to choose one from the server-defined strategy upon registering themselves.
On the other hand, I think "rebalance with a minimal number of reassignment" should be quite a common reassignment strategy, and I think it is possible to just add it into the server-defined strategies list. Guozhang On Wed, Jun 10, 2015 at 9:32 AM, Johansson, Olof < olof.johans...@thingworx.com> wrote: > Is it possible to have a consumer rebalancing partition assignment > strategy that will rebalance with a minimal number of reassignments? > > Per the "Kafka 0.9 Consumer Rewrite Design" it should be possible to > define my own partition assignment strategy: > "partition.assignment.strategies - may take a comma separated list of > properties that map the strategy's friendly name to to the class that > implements the strategy. This is used for any strategy implemented by the > user and released to the Kafka cluster. By default, Kafka will include a > set of strategies that can be used by the consumer." > > Is there a Jira ticket that tracks adding user defined > partitions.assignment.strategies? In the latest source, range, and > roundrobin are still the only possible values (hard-coded). > > I assume that any user implemented strategy would have to implement the > PartitionAssignor trait. If so, by naively looking at the 0.8.3 source, a > strategy that should do a minimal number of partition reassignments would > need the ConsumerMetaData. That's not currently available in the > PartitionAssignor contract - assign(topicsPerConsumer, partitionsPerTopic). > Have there been any discussion to change the contract to pass > ConsumerMetaData? > -- -- Guozhang
