Kafka cluster setup issue

2019-11-28 Thread Uday Kiran A
Hi Team, We have setup the open source kafka_2.11 setup on ubuntu with 6 nodes . we are facing issue with the subscribers(dotnet) .Messages are failed to deliver and messages are missing. Can you help us in verifying the configuration. Thanks and Regards, Uday Kiran A Asociate Architect - Prod

Re: Kafka Cluster Setup

2018-12-07 Thread Vahid Hashemian
Hi Abhimanyu, Suman already answered your cluster upgrade question. To move all partitions from one broker to another, you can use the kafka-topics (with --describe) command to list all topic partitions and their current replicas. You can use that, replace all instances of broker B to broker A, a

Re: Kafka Cluster Setup

2018-12-07 Thread Suman B N
-On adding a new Kafka Broker(B) to the cluster. Will all data present on broker A will be distributed automatically? If not what I need to do distribute the data. Automatic distribution doesn't happen. Use kafka-reassign-partitions.sh command to distribute load manually. - Not let's supp

Re: Kafka Cluster Setup

2018-12-06 Thread Abhimanyu Nagrath
Hi Vahid, I missed one more point. - Can I have two brokers at the same time running at multiple versions just for Kafka version upgrade? Regards, Abhimanyu On Fri, Dec 7, 2018 at 1:03 PM Abhimanyu Nagrath wrote: > Thanks for clarifying all the points . Just a small doubt regarding my

Re: Kafka Cluster Setup

2018-12-06 Thread Abhimanyu Nagrath
Thanks for clarifying all the points . Just a small doubt regarding my second query: - *Not let's suppose somehow the case! is solved my data is distributed on both the brokers. Now due to some maintenance issue, I want to take down the server B.* - *How to transfer the data of Bro

Re: Kafka Cluster Setup

2018-12-06 Thread Vahid Hashemian
Hi Abhimanyu, I have answered your questions inline, but before that I just want to emphasize the notion of topics and partitions that are critical to Kafka's resiliency and scalability. Topics in Kafka can have multiple partitions. Each partition can be stored on one broker only. But the number

Kafka Cluster Setup

2018-12-06 Thread Abhimanyu Nagrath
Hi, I have a use case I want to set up a Kafka cluster initially at the starting I have 1 Kafka Broker(A) and 1 Zookeeper Node. So below mentioned are my queries: - On adding a new Kafka Broker(B) to the cluster. Will all data present on broker A will be distributed automatically? If not wh