MirrorMaker is one reasonable way to do this, certainly it can replicate to another region, with most of the latency being the unavoidable kind, if you give it enough resources.
It'll require having a HA cluster running in another region, of course. One other caveat is that it doesn't preserve the offsets of the records. That's probably okay for your use-case, but you should be aware of it. Since what you want is a backup, there are many ways to do that which might be cheaper than another Kafka cluster. For example, you could use a Kafka Connect s3 sink. You'd have to write some disaster-recovery code to restore lost data from s3 into Kafka. https://www.confluent.io/blog/apache-kafka-to-amazon-s3-exactly-once/ There are many other sinks available, but s3 might be a reasonable choice for backup. It's inexpensive and reliable. On Fri, Mar 5, 2021, 2:48 AM Pushkar Deole <pdeole2...@gmail.com> wrote: > Yes.. so the requirement for me is to have data backed up or replicated in > a different 'region' to cater for disaster scenarios and recover from them > > On Fri, Mar 5, 2021 at 3:01 PM Ran Lupovich <ranlupov...@gmail.com> wrote: > > > I guess that in case of avoiding data lose you would need to use 3 > replica > > in different rack/sites awareness to avoid data lose, Confluent's > > Replicator or MirrorMaker are for copying data from one cluster to > another > > usually in different dc / regions, If I am not mistaken > > > > בתאריך יום ו׳, 5 במרץ 2021, 11:21, מאת Pushkar Deole < > > pdeole2...@gmail.com > > >: > > > > > Thanks Luke... is the mirror maker asynchronous? What will be typical > lag > > > between the replicated cluster and running cluster and in case of > > disaster, > > > what are the chances of data loss? > > > > > > On Fri, Mar 5, 2021 at 11:37 AM Luke Chen <show...@gmail.com> wrote: > > > > > > > Hi Pushkar, > > > > MirrorMaker is what you're looking for. > > > > ref: > > https://kafka.apache.org/documentation/#georeplication-mirrormaker > > > > > > > > Thanks. > > > > Luke > > > > > > > > On Fri, Mar 5, 2021 at 1:50 PM Pushkar Deole <pdeole2...@gmail.com> > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > I was looking for some options to backup a running kafka cluster, > for > > > > > disaster recovery requirements. Can someone provide what are the > > > > available > > > > > options to backup and restore a running cluster in case the entire > > > > cluster > > > > > goes down? > > > > > > > > > > Thanks.. > > > > > > > > > > > > > > >
