On Thu, Aug 29, 2013 at 11:09 AM, Mark <static.void....@gmail.com> wrote:
> 1) Should a producer be aware of which broker to write to or is this > somehow managed by Kafka itself. For example, If I have 2 brokers with a > configured partition size of 1 will my messages be written in a round-robin > type of fashion to each broker or is this completed dependent on which > broker the consume connects to? > Sounds like a question regarding the Producer, which I am not familiar with. We use our own Producers here, written in C++. > > 2) Similar question goes for a writing to a topic that has multiple > partitions. Are the writes randomly load-balanced between partitions or is > this something the client needs to do? > It depends. Producers can specify the partition, or use a partition of '-1', which instructs Kafka to randomly choose a partition. > > 3) If I have 1 topic with 1 partition and 2 brokers how does consuming > messages work. For example, if my consumer connects to broker 1 how does it > consume messages written to broker 2? > Don't think of partitions on one broker and partitions on another broker. Simply think of the aggregate partition count across the cluster. The high-level consumer that comes with Kafka will learn where all the partitions are for a given topic via Zookeeper, create connections to the Kafka brokers as needed, and drain all the partitions. > > Thanks > >
