Apologies for piggybacking on a thread, figured the discussion was pretty relevant to a thought I had kicking around my brain.
In the event of complete failure or sustained loss of connectivity of the first cluster, could the secondary cluster act as a failover for a given configuration? Assuming the respective clusters are set up like the following: Cluster A brokerA01:9092 brokerA02:9092 brokerA03:9092 Cluster B brokerB01:9092 brokerB02:9092 brokerB03:9092 And the corresponding producer's properties line contained something like: bootstrap.servers="brokerA01:9092,brokerA02:9092,brokerA03:9092,brokerB01:9092,brokerB02:9092,brokerB03:9092" My initial theory is that a failover action would respond like: 1. Attempting to reach brokerA01:9092 with a metadata fetch, cannot reach, attempt to reach next broker 2. Attempting to reach brokerA02:9092 with a metadata fetch, cannot reach, attempt to reach next broker 3. Attempting to reach brokerA03:9092 with a metadata fetch, cannot reach, attempt to reach next broker 4. Attempting to reach brokerB01:9092 with a metadata fetch, able to reach, begin exchange with reachable cluster I have a feeling that this would not work, but I don't really have any evidence to substantiate it at this point. Anyone have any experience on this sort of a situation? The intended goal is more to ensure that the messages end up on *some *cluster, not necessarily a particular cluster. Best, On Wed, Aug 21, 2019 at 3:28 AM David Jacot <dja...@confluent.io> wrote: > Hello, > > As of today, the producer is able to talk to only one Kafka cluster. > *bootstrap.servers* is to provide a list of brokers belonging to the same > cluster to ensure at least one is available. > > In your case, you have to replicate the data from the first cluster to the > second cluster with mirror maker for instance. > > Best, > David > > On Mon, Aug 12, 2019 at 11:13 AM lsroudi abdel <lsro...@gmail.com> wrote: > > > I guess it more about replication, you can push your data in one choosen > > cluster and replicate your data to the second one by using mirror maker > or > > confluent replicator or the new open sourced project by LinkedIn, > > "Brooklin" > > > > Le lun. 12 août 2019 à 10:19, Garvit Sharma <eng.gar...@gmail.com> a > > écrit : > > > > > Thanks, Isroudi. But in my usecase there are two separate zookeeper as > > > well. Let me know how would it handle. > > > > > > On Mon, Aug 12, 2019 at 1:45 PM lsroudi abdel <lsro...@gmail.com> > wrote: > > > > > > > If it is two cluster it mean you have two zookeeper, I guess you > can't > > do > > > > that, I f you have one zookeeper it will be ok, in the case you have > > one > > > > zookeeper it's just an architecture with replica across two data > > center, > > > I > > > > hope it clear for you > > > > > > > > Le lun. 12 août 2019 à 09:22, Garvit Sharma <eng.gar...@gmail.com> a > > > > écrit : > > > > > > > > > Hi, > > > > > > > > > > I have 2 kafka clusters in different data-centers so can I provide > > the > > > > dns > > > > > hostnames of both the clusters separated by comma in > > > *bootstrap.servers* > > > > > key in producer config ? > > > > > > > > > > If I provide two different clusters in *bootstrap.servers *then how > > > would > > > > > the events get published ? > > > > > Would events get published to both the clusters or either of them > > > > randomly > > > > > ? > > > > > > > > > > Looking forward to hearing from you. > > > > > > > > > > Thanks > > > > > > > > > > > > > > >
