AFAIK (not actually using myself), for cross DC replication people tend to use MirrorMaker to transfer one cluster's data to another, usually a kind of central DC that unifies all "regional" DCs, but the layout depends on your business reqs. Then your consumer are assigned only with local brokers' addresses. Exactly because of the reason you mentioned, high latency of consuming from a remote broker and not being able to control partition assignment, i.e. which broker becomes the leader if current leader fails, since this is governed by the rule that says the most up to date in-sync replica becomes the leader.
Marko Bonaći Monitoring | Alerting | Anomaly Detection | Centralized Log Management Solr & Elasticsearch Support Sematext <http://sematext.com/> | Contact <http://sematext.com/about/contact.html> On Thu, Sep 29, 2016 at 7:30 PM, Ezra Stuetzel <ezra.stuet...@gmail.com> wrote: > Hi, > In kafka 0.10 is there a way to configure the consumer such that it is rack > aware? We replicate data across all our 'racks' and want consumers to > choose brokers that are rack local whenever possible. Our configured racks > are actually in different datacenters so there is much higher network cost > of not consuming from nearest replica. > > Configuring the consumer to only consume from specific hosts would also > achieve what we are trying to do if that is possible? > > Also, are there any major downsides to using the rack setting for cross > datacenter replication? > > Thanks, > Ezra >
