Actually we are using Load balancer in this setup like as follows:- 1. Public user/producer hits the public IP x.x.x.x:9092 which is a static NAT with load balancer private IP y.y.y.y: 9092.
2. In load balancer we have a pool of servers which contains the 3 node clustered Kafka broker servers. Load balancer will distribute the connection requests among the Kafka servers. 3. Kafka broker server/ leader will talk to primary zookeeper(which is also 3 node cluster) and consumes & write the data on database servers. Please suggest if this is a feasible or recommended solution by Kafka as for on-premises setup. If yes then what are recommendations to make it working. Because we are facing issue in name resolution after request landed onto the Kafka broker server as it is not taking the same reverse path to fulfill the request and name resolution as well. Thanks Nitin Sharma On Fri, 18 Jun, 2021, 10:50 pm Mich Talebzadeh, <mich.talebza...@gmail.com> wrote: > OK the three node Kafka cluster is classic.Pretty easy to set-up with three > brokers and three zookeepers on three nodes. > > You quoted scala version 2.13. Your kafka version should be something like > below > > *kafka_2.12-2.7.0* > > For Kafka 2.7.0 with Scala 2.12 > > For DNS refer to this SO thead > > How to Setup a Public Kafka Broker Using a Dynamic DNS? > < > https://stackoverflow.com/questions/53705573/how-to-setup-a-public-kafka-broker-using-a-dynamic-dns > > > > HTH > > > > view my Linkedin profile > <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > On Fri, 18 Jun 2021 at 21:32, Nitin Sharma <mr09...@gmail.com> wrote: > > > Hello Mich, > > > > > > Thanks for your response. > > > > :Please find the answers of your points :- > > > > 1. Have you set-up Apache Kafka on AWS as part of AWS offerings on > > Compute engines (VM) and if so how many nodes? ----- > *No it's in > > on-premises.* > > 2. Are you using containers in the Cloud? -- > *No containers are > being > > used.* > > 3. Which version of Kafka -- > *2.13 version * > > 4. Any third party offerings as a service for this set-up in AWS ---> > > *No > > third party offering are being used* > > 5. Assuming that you have a three node cluster in Cloud with Kafka and > > zookeeper, do you have the same number of nodes on-premises ( Data > > centre ) that you intend to utilise? ----> *Yes 3 node cluster but > > on-premises* > > 6. Finally what is not working in your Data centre -- > *We want to > > connect Kafka cluster from public network with 3 node cluster. * > > > > *Note :- one more thing we want to know is , How important the DNS is in > > the Kafka setup while accessing it from open internet and what are the > must > > have requirements to achieve this setup to work in the On-premises > > environment.* > > > > Regards > > Nitin Sharma > > > > . > > > > On Fri, Jun 18, 2021 at 7:36 PM Mich Talebzadeh < > mich.talebza...@gmail.com > > > > > wrote: > > > > > Hi, > > > > > > Vital pieces are missing. How has the cluster been set up in AWS? > > > > > > > > > 1. Have you set-up Apache Kafka on AWS as part of AWS offerings on > > > Compute engines (VM) and if so how many nodes? > > > 2. Are you using containers in the Cloud? > > > 3. Which version of Kafka > > > 4. Any third party offerings as a service for this set-up in AWS > > > 5. Assuming that you have a three node cluster in Cloud with Kafka > and > > > zookeeper, do you have the same number of nodes on-premises ( Data > > > centre ) that you intend to utilise? > > > 6. Finally what is not working in your Data centre > > > > > > > > > HTH > > > > > > > > > > > > view my Linkedin profile > > > <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> > > > > > > > > > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for > any > > > loss, damage or destruction of data or any other property which may > arise > > > from relying on this email's technical content is explicitly > disclaimed. > > > The author will in no case be liable for any monetary damages arising > > from > > > such loss, damage or destruction. > > > > > > > > > > > > > > > On Fri, 18 Jun 2021 at 15:41, Nitin Sharma <mr09...@gmail.com> wrote: > > > > > > > Hi Team, > > > > > > > > Hope you are well!! > > > > > > > > I need your help in setting up the Kafka On-Premises Setup. > > > > > > > > Problem Setup :- > > > > > > > > 1. We have tested the setup on the AWS environment and it is working > > > fine. > > > > 2. We are trying to replicate the same in the On-Premises environment > > but > > > > it seems not working properly. > > > > > > > > > > > > Please suggest if it is possible to create the same setup in the Data > > > > centre premises instead of Cloud environment. > > > > > > > > > > > > Thanks > > > > Nitin Sharma > > > > > > > > > >
