Thanks Jun for your quick reply. If I understood correctly, for the first step ( fetching metadata ), producer can still go through a VIP whereas for the second step ( once producer gets the metadata ), producer needs to connect to the correct instance of the broker that hosts the partition. I guess for the second step, producer should be able to directly connect to the right broker and this cant be achieved through a Load Balancer ( as LB may send the request to any one of the machines in the pool ).
Is my understanding correct ? Does it also mean that when a new broker is added or an existing broker is removed, we need to keep the list of brokers updated ? Thanks On Mon, Mar 31, 2014 at 9:22 PM, Jun Rao <jun...@gmail.com> wrote: > In Kafka, the broker list you provide to the producer is used only for > fetching metadata, which can be served on any broker. The producer then > connects to the right brokers according to the metadata response. > > Thanks, > > Jun > > > On Mon, Mar 31, 2014 at 6:04 PM, kafka user <kafkaraic...@gmail.com> > wrote: > > > If we have a kafka producer connecting to a Kafka cluster behind a > hardware > > Load Balancer (VIP), will producer be able to send a message to a right > > partition ? > > Can one of the brokers in a cluster do a broker discovery to forward the > > message to ? > > I guess my question is whether it makes sense to use a hardware load > > balancer at all. > > > > Typical deployment model at our company is to have a pool of machines and > > LB decides which machine in a pool would receive a request. Looking to > hear > > from kafka experts what is the best way of setting up a cluster given > that > > the cluster has be behind a VIP. > > > > > > Thanks > > >
