[
https://issues.apache.org/jira/browse/KAFKA-1792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257377#comment-14257377
]
Dmitry Pekar edited comment on KAFKA-1792 at 12/23/14 7:14 PM:
---------------------------------------------------------------
[~nehanarkhede] Please see attached test results for test cluster of 5 brokers.
Typical scenarios are tested there such as broker addition/removal/replacement.
Also tested a scenario for decommission-broker which is just a special case of
the above algorithm application.
was (Author: dmitry pekar):
[~nehanarkhede] Please see attached test results, for test cluster of 5 brokers.
Typical scenarios are tested there such as broker addition/removal/replacement.
Also tested a scenario for decommission-broker which is just a special case of
above algorithm application.
> change behavior of --generate to produce assignment config with fair replica
> distribution and minimal number of reassignments
> -----------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-1792
> URL: https://issues.apache.org/jira/browse/KAFKA-1792
> Project: Kafka
> Issue Type: Sub-task
> Components: tools
> Reporter: Dmitry Pekar
> Assignee: Dmitry Pekar
> Fix For: 0.8.3
>
> Attachments: KAFKA-1792.patch, KAFKA-1792_2014-12-03_19:24:56.patch,
> KAFKA-1792_2014-12-08_13:42:43.patch, KAFKA-1792_2014-12-19_16:48:12.patch,
> generate_alg_tests.txt
>
>
> Current implementation produces fair replica distribution between specified
> list of brokers. Unfortunately, it doesn't take
> into account current replica assignment.
> So if we have, for instance, 3 brokers id=[0..2] and are going to add fourth
> broker id=3,
> generate will create an assignment config which will redistribute replicas
> fairly across brokers [0..3]
> in the same way as those partitions were created from scratch. It will not
> take into consideration current replica
> assignment and accordingly will not try to minimize number of replica moves
> between brokers.
> As proposed by [~charmalloc] this should be improved. New output of improved
> --generate algorithm should suite following requirements:
> - fairness of replica distribution - every broker will have R or R+1 replicas
> assigned;
> - minimum of reassignments - number of replica moves between brokers will be
> minimal;
> Example.
> Consider following replica distribution per brokers [0..3] (we just added
> brokers 2 and 3):
> - broker - 0, 1, 2, 3
> - replicas - 7, 6, 0, 0
> The new algorithm will produce following assignment:
> - broker - 0, 1, 2, 3
> - replicas - 4, 3, 3, 3
> - moves - -3, -3, +3, +3
> It will be fair and number of moves will be 6, which is minimal for specified
> initial distribution.
> The scope of this issue is:
> - design an algorithm matching the above requirements;
> - implement this algorithm and unit tests;
> - test it manually using different initial assignments;
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
