Hi Pushkar,

you could also look at the available Kafka-connect plugins. It provides
many connectors which could be leveraged to move the data in/out from Kafka.

On Sat, Mar 6, 2021 at 10:18 AM Blake Miller <blak3mil...@gmail.com> wrote:

> MirrorMaker is one reasonable way to do this, certainly it can replicate to
> another region, with most of the latency being the unavoidable kind, if you
> give it enough resources.
>
> It'll require having a HA cluster running in another region, of course. One
> other caveat is that it doesn't preserve the offsets of the records. That's
> probably okay for your use-case, but you should be aware of it.
>
> Since what you want is a backup, there are many ways to do that which might
> be cheaper than another Kafka cluster.
>
> For example, you could use a Kafka Connect s3 sink. You'd have to write
> some disaster-recovery code to restore lost data from s3 into Kafka.
>
> https://www.confluent.io/blog/apache-kafka-to-amazon-s3-exactly-once/
>
> There are many other sinks available, but s3 might be a reasonable choice
> for backup. It's inexpensive and reliable.
>
> On Fri, Mar 5, 2021, 2:48 AM Pushkar Deole <pdeole2...@gmail.com> wrote:
>
> > Yes.. so the requirement for me is to have data backed up or replicated
> in
> > a different 'region' to cater for disaster scenarios and recover from
> them
> >
> > On Fri, Mar 5, 2021 at 3:01 PM Ran Lupovich <ranlupov...@gmail.com>
> wrote:
> >
> > > I guess that in case of avoiding data lose you would need to use 3
> > replica
> > > in different rack/sites awareness to avoid data lose, Confluent's
> > > Replicator or MirrorMaker are for copying data from one cluster to
> > another
> > > usually in different dc / regions, If I am not mistaken
> > >
> > > בתאריך יום ו׳, 5 במרץ 2021, 11:21, מאת Pushkar Deole ‏<
> > > pdeole2...@gmail.com
> > > >:
> > >
> > > > Thanks Luke... is the mirror maker asynchronous? What will be typical
> > lag
> > > > between the replicated cluster and running cluster and in case of
> > > disaster,
> > > > what are the chances of data loss?
> > > >
> > > > On Fri, Mar 5, 2021 at 11:37 AM Luke Chen <show...@gmail.com> wrote:
> > > >
> > > > > Hi Pushkar,
> > > > > MirrorMaker is what you're looking for.
> > > > > ref:
> > > https://kafka.apache.org/documentation/#georeplication-mirrormaker
> > > > >
> > > > > Thanks.
> > > > > Luke
> > > > >
> > > > > On Fri, Mar 5, 2021 at 1:50 PM Pushkar Deole <pdeole2...@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > I was looking for some options to backup a running kafka cluster,
> > for
> > > > > > disaster recovery requirements. Can someone provide what are the
> > > > > available
> > > > > > options to backup and restore a running cluster in case the
> entire
> > > > > cluster
> > > > > > goes down?
> > > > > >
> > > > > > Thanks..
> > > > > >
> > > > >
> > > >
> > >
> >
>


-- 
Regards,
Himanshu Shukla

Reply via email to