Thanks for starting this KIP Allen.
I agree with Gwen that having a RackLocator class that is pluggable seems
to be too complex. The KIP refers to potentially non-ZK storage for the
rack info which I don't think is necessary.
Perhaps we can persist this info in zk under /brokers/ids/<broker_id>
similar to other broker properties and add a config in KafkaConfig called
"rack".
{"jmx_port":-1,"endpoints":[...],"host":"xxx","port":yyy, "rack": "abc"}
Aditya
On Mon, Sep 28, 2015 at 2:30 PM, Gwen Shapira <g...@confluent.io> wrote:
> Hi,
>
> First, thanks for putting out a KIP for this. This is super important for
> production deployments of Kafka.
>
> Few questions:
>
> 1) Are we sure we want "as many racks as possible"? I'd want to balance
> between safety (more racks) and network utilization (traffic within a rack
> uses the high-bandwidth TOR switch). One replica on a different rack and
> the rest on same rack (if possible) sounds better to me.
>
> 2) Rack-locator class seems overly complex compared to adding a rack.number
> property to the broker properties file. Why do we want that?
>
> Gwen
>
>
>
> On Mon, Sep 28, 2015 at 12:15 PM, Allen Wang <allenxw...@gmail.com> wrote:
>
> > Hello Kafka Developers,
> >
> > I just created KIP-36 for rack aware replica assignment.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-36+Rack+aware+replica+assignment
> >
> > The goal is to utilize the isolation provided by the racks in data center
> > and distribute replicas to racks to provide fault tolerance.
> >
> > Comments are welcome.
> >
> > Thanks,
> > Allen
> >
>
