Thanks Grant, This is really helpful..
Regards, Nitin Kumar Sharma. On Tue, Oct 27, 2015 at 1:16 PM, Grant Henke <ghe...@cloudera.com> wrote: > Hi Nitin, > > Partitions and replicas are assigned in a round robin fashion. You can read > more about it in the docs under "4.7 Replication > <http://kafka.apache.org/documentation.html#replication>" look for the > header for "Replica Management". > > You can view the implementation here: > > https://github.com/apache/kafka/blob/21443f214fc6f1f51037e27f8ece155cf1eb288c/core/src/main/scala/kafka/admin/AdminUtils.scala#L48-L65 > > I copy-pasted the relevant javadoc below for convenience: > > /** > > * There are 2 goals of replica assignment: > > * 1. Spread the replicas evenly among brokers. > > * 2. For partitions assigned to a particular broker, their other > > replicas are spread over the other brokers. > > * > > * To achieve this goal, we: > > * 1. Assign the first replica of each partition by round-robin, > > starting from a random position in the broker list. > > * 2. Assign the remaining replicas of each partition with an > increasing > > shift. > > * > > * Here is an example of assigning (10 partitions, replication = 3) > > * broker-0 broker-1 broker-2 broker-3 broker-4 > > * p0 p1 p2 p3 p4 (1st replica) > > * p5 p6 p7 p8 p9 (1st replica) > > * p4 p0 p1 p2 p3 (2nd replica) > > * p8 p9 p5 p6 p7 (2nd replica) > > * p3 p4 p0 p1 p2 (3nd replica) > > * p7 p8 p9 p5 p6 (3nd replica) > > */ > > > Note that this is just the default implementation, you can manually assign > them however you like using the command line tools. There are also some > Jiras in progress to implement other options as well(KAFKA-2435 > <https://issues.apache.org/jira/browse/KAFKA-2435>, KAFKA-1215 > <https://issues.apache.org/jira/browse/KAFKA-1215>, KAFKA-1736 > <https://issues.apache.org/jira/browse/KAFKA-1736>). > > Thanks, > Grant > > On Tue, Oct 27, 2015 at 11:58 AM, nitin sharma < > kumarsharma.ni...@gmail.com> > wrote: > > > Hi All, > > > > I would like to know what kind of strategy Kafka adopts in splitting > > partitions of a topic to across the cluster. > > > > Ex: if i have 10 broker in a cluster. 1 topic with 5 partition and each > > with replication-factor of 2. then how would Kafka will assign the > > partitions ( 5*2 =10) across the cluster nodes? > > > > > > Regards, > > Nitin Kumar Sharma. > > > > > > -- > Grant Henke > Software Engineer | Cloudera > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke >
