Re: Kafka Cluster Setup

2018-12-07 Thread Vahid Hashemian
Hi Abhimanyu, Suman already answered your cluster upgrade question. To move all partitions from one broker to another, you can use the kafka-topics (with --describe) command to list all topic partitions and their current replicas. You can use that, replace all instances of broker B to broker A, a

Re: Kafka Cluster Setup

2018-12-07 Thread Suman B N
-On adding a new Kafka Broker(B) to the cluster. Will all data present on broker A will be distributed automatically? If not what I need to do distribute the data. Automatic distribution doesn't happen. Use kafka-reassign-partitions.sh command to distribute load manually. - Not let's supp

Re: Kafka Cluster Setup

2018-12-06 Thread Abhimanyu Nagrath
Hi Vahid, I missed one more point. - Can I have two brokers at the same time running at multiple versions just for Kafka version upgrade? Regards, Abhimanyu On Fri, Dec 7, 2018 at 1:03 PM Abhimanyu Nagrath wrote: > Thanks for clarifying all the points . Just a small doubt regarding my

Re: Kafka Cluster Setup

2018-12-06 Thread Abhimanyu Nagrath
Thanks for clarifying all the points . Just a small doubt regarding my second query: - *Not let's suppose somehow the case! is solved my data is distributed on both the brokers. Now due to some maintenance issue, I want to take down the server B.* - *How to transfer the data of Bro

Re: Kafka Cluster Setup

2018-12-06 Thread Vahid Hashemian
Hi Abhimanyu, I have answered your questions inline, but before that I just want to emphasize the notion of topics and partitions that are critical to Kafka's resiliency and scalability. Topics in Kafka can have multiple partitions. Each partition can be stored on one broker only. But the number