Re: Help !! Apache-Kafka On-Premises Setup.

2021-06-18 Thread Ryanne Dolan
Nitin, I know this isn't a particularly helpful response, but you should know using a load balancer like that is really hard to set up with kafka due to the fact that it advertises broker addresses to clients. I've done it a few times but it's something of a dark art :) If you can get away without

Re: Help !! Apache-Kafka On-Premises Setup.

2021-06-18 Thread Marcus Horsley-Rai
Hi Nitin, It sounds like you are missing configuration for the advertised.listeners. See https://www.confluent.de/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/ for example. Once the client has initially connected to a broker, it's the DNS/IP exchanged in metadata that governs h

Re: Help !! Apache-Kafka On-Premises Setup.

2021-06-18 Thread Nitin Sharma
Actually we are using Load balancer in this setup like as follows:- 1. Public user/producer hits the public IP x.x.x.x:9092 which is a static NAT with load balancer private IP y.y.y.y: 9092. 2. In load balancer we have a pool of servers which contains the 3 node clustered Kafka broker servers. Lo

Re: Help !! Apache-Kafka On-Premises Setup.

2021-06-18 Thread Mich Talebzadeh
OK the three node Kafka cluster is classic.Pretty easy to set-up with three brokers and three zookeepers on three nodes. You quoted scala version 2.13. Your kafka version should be something like below *kafka_2.12-2.7.0* For Kafka 2.7.0 with Scala 2.12 For DNS refer to this SO thead How to Se

Re: Help !! Apache-Kafka On-Premises Setup.

2021-06-18 Thread Nitin Sharma
Hello Mich, Thanks for your response. :Please find the answers of your points :- 1. Have you set-up Apache Kafka on AWS as part of AWS offerings on Compute engines (VM) and if so how many nodes? - > *No it's in on-premises.* 2. Are you using containers in the Cloud? -- > *No container

Re: Help !! Apache-Kafka On-Premises Setup.

2021-06-18 Thread Mich Talebzadeh
Hi, Vital pieces are missing. How has the cluster been set up in AWS? 1. Have you set-up Apache Kafka on AWS as part of AWS offerings on Compute engines (VM) and if so how many nodes? 2. Are you using containers in the Cloud? 3. Which version of Kafka 4. Any third party offerings

Help !! Apache-Kafka On-Premises Setup.

2021-06-18 Thread Nitin Sharma
Hi Team, Hope you are well!! I need your help in setting up the Kafka On-Premises Setup. Problem Setup :- 1. We have tested the setup on the AWS environment and it is working fine. 2. We are trying to replicate the same in the On-Premises environment but it seems not working properly. Please