Re: Topic to broker assignment

2015-04-13 Thread Gwen Shapira
Here's the algorithm as described in AdminUtils.scala: /** * There are 2 goals of replica assignment: * 1. Spread the replicas evenly among brokers. * 2. For partitions assigned to a particular broker, their other replicas are spread over the other brokers. * * To achieve this goa

Re: Topic to broker assignment

2015-04-13 Thread Bill Hastings
Thanks for the reference. But it doesn't seem to cover how a particular topic is assigned to a particular broker and also how the replicas are chosen. For eg if I have brokers A B C D and E what algorithm is used to assign a topic X and partition 1 to brokers B D and E if the chosen replication fac

Re: Topic to broker assignment

2015-04-13 Thread Jiangjie Qin
A quick reference. http://www.slideshare.net/junrao/kafka-replication-apachecon2013 On 4/12/15, 11:36 PM, "Bill Hastings" wrote: >Hi Guys > >How do topics get assigned to brokers? I mean if I were to create a topic >X >and publish to it how does Kafka assign the topic and the message to a >part

Topic to broker assignment

2015-04-12 Thread Bill Hastings
Hi Guys How do topics get assigned to brokers? I mean if I were to create a topic X and publish to it how does Kafka assign the topic and the message to a particular broker? If I have create a topic with say 10 partitions how does kafka assign each partition to a different broker? -- Cheers Bill