Hi Nitin, It sounds like you are missing configuration for the advertised.listeners. See https://www.confluent.de/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/ for example.
Once the client has initially connected to a broker, it's the DNS/IP exchanged in metadata that governs how further communication happens. I had to do this myself in an on-premises deployment with a mix of red/greenside - albeit not with a load balancer in the mix. Hope this helps Marcus On Fri, 18 Jun 2021, 22:08 Nitin Sharma, <mr09...@gmail.com> wrote: > Actually we are using Load balancer in this setup like as follows:- > > 1. Public user/producer hits the public IP x.x.x.x:9092 which is a static > NAT with load balancer private IP y.y.y.y: 9092. > > 2. In load balancer we have a pool of servers which contains the 3 node > clustered Kafka broker servers. Load balancer will distribute the > connection requests among the Kafka servers. > > 3. Kafka broker server/ leader will talk to primary zookeeper(which is also > 3 node cluster) and consumes & write the data on database servers. > > > Please suggest if this is a feasible or recommended solution by Kafka as > for on-premises setup. If yes then what are recommendations to make it > working. > > Because we are facing issue in name resolution after request landed onto > the Kafka broker server as it is not taking the same reverse path to > fulfill the request and name resolution as well. > > Thanks > Nitin Sharma > > On Fri, 18 Jun, 2021, 10:50 pm Mich Talebzadeh, <mich.talebza...@gmail.com > > > wrote: > > > OK the three node Kafka cluster is classic.Pretty easy to set-up with > three > > brokers and three zookeepers on three nodes. > > > > You quoted scala version 2.13. Your kafka version should be something > like > > below > > > > *kafka_2.12-2.7.0* > > > > For Kafka 2.7.0 with Scala 2.12 > > > > For DNS refer to this SO thead > > > > How to Setup a Public Kafka Broker Using a Dynamic DNS? > > < > > > https://stackoverflow.com/questions/53705573/how-to-setup-a-public-kafka-broker-using-a-dynamic-dns > > > > > > > HTH > > > > > > > > view my Linkedin profile > > <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> > > > > > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > > loss, damage or destruction of data or any other property which may arise > > from relying on this email's technical content is explicitly disclaimed. > > The author will in no case be liable for any monetary damages arising > from > > such loss, damage or destruction. > > > > > > > > > > On Fri, 18 Jun 2021 at 21:32, Nitin Sharma <mr09...@gmail.com> wrote: > > > > > Hello Mich, > > > > > > > > > Thanks for your response. > > > > > > :Please find the answers of your points :- > > > > > > 1. Have you set-up Apache Kafka on AWS as part of AWS offerings on > > > Compute engines (VM) and if so how many nodes? ----- > *No it's in > > > on-premises.* > > > 2. Are you using containers in the Cloud? -- > *No containers are > > being > > > used.* > > > 3. Which version of Kafka -- > *2.13 version * > > > 4. Any third party offerings as a service for this set-up in AWS > ---> > > > *No > > > third party offering are being used* > > > 5. Assuming that you have a three node cluster in Cloud with Kafka > and > > > zookeeper, do you have the same number of nodes on-premises ( Data > > > centre ) that you intend to utilise? ----> *Yes 3 node cluster but > > > on-premises* > > > 6. Finally what is not working in your Data centre -- > *We want to > > > connect Kafka cluster from public network with 3 node cluster. * > > > > > > *Note :- one more thing we want to know is , How important the DNS is > in > > > the Kafka setup while accessing it from open internet and what are the > > must > > > have requirements to achieve this setup to work in the On-premises > > > environment.* > > > > > > Regards > > > Nitin Sharma > > > > > > . > > > > > > On Fri, Jun 18, 2021 at 7:36 PM Mich Talebzadeh < > > mich.talebza...@gmail.com > > > > > > > wrote: > > > > > > > Hi, > > > > > > > > Vital pieces are missing. How has the cluster been set up in AWS? > > > > > > > > > > > > 1. Have you set-up Apache Kafka on AWS as part of AWS offerings on > > > > Compute engines (VM) and if so how many nodes? > > > > 2. Are you using containers in the Cloud? > > > > 3. Which version of Kafka > > > > 4. Any third party offerings as a service for this set-up in AWS > > > > 5. Assuming that you have a three node cluster in Cloud with Kafka > > and > > > > zookeeper, do you have the same number of nodes on-premises ( Data > > > > centre ) that you intend to utilise? > > > > 6. Finally what is not working in your Data centre > > > > > > > > > > > > HTH > > > > > > > > > > > > > > > > view my Linkedin profile > > > > <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> > > > > > > > > > > > > > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for > > any > > > > loss, damage or destruction of data or any other property which may > > arise > > > > from relying on this email's technical content is explicitly > > disclaimed. > > > > The author will in no case be liable for any monetary damages arising > > > from > > > > such loss, damage or destruction. > > > > > > > > > > > > > > > > > > > > On Fri, 18 Jun 2021 at 15:41, Nitin Sharma <mr09...@gmail.com> > wrote: > > > > > > > > > Hi Team, > > > > > > > > > > Hope you are well!! > > > > > > > > > > I need your help in setting up the Kafka On-Premises Setup. > > > > > > > > > > Problem Setup :- > > > > > > > > > > 1. We have tested the setup on the AWS environment and it is > working > > > > fine. > > > > > 2. We are trying to replicate the same in the On-Premises > environment > > > but > > > > > it seems not working properly. > > > > > > > > > > > > > > > Please suggest if it is possible to create the same setup in the > Data > > > > > centre premises instead of Cloud environment. > > > > > > > > > > > > > > > Thanks > > > > > Nitin Sharma > > > > > > > > > > > > > > >
