Multiclustering can help. I would like to add good alternatives for Kafka
Mirror Maker which already has been mentioned: 1. Uber uReplication main
advantage: avoiding consumers' rebalances delay eng.uber.com eng.uber.com
2. Confluent Replicator. main advantage: simplifies administrating by wroking
as a source connector docs.confluent.io docs.confluent.io Best Wishes
Wojciech Obłąk Dnia 10 sierpnia 2020 14:18 Liam Clarke-Hutchinson
<liam.cla...@adscale.co.nz> napisał(a): Hi Kumar, You can restore but
you'll lose data since your last snapshot. I use KC -> SO instead for
obvious reasons. All that said, I've never had multiple Kafka nodes fail
to the extent that I need my S3 backup. But it's good to have it, for
peace of mind. Thanks, Liam Clarke-Hutchinson On Mon, 10 Aug. 2020, 2:27
pm kumar, <oracl...@gmail.com> wrote: Hi Liam - I did not understand
cloning kafka broker volumes. if you have 1 TB disk , Assuming the usage is
65% data in the volume is changing so fast. take 650 GB every hour or every
min ? how do we restore if there was failure? Oracle database provides point
in time recovery(incremental+full backup+archive logs) of their database. is
it possible to recover kafka like that? We had storage failure of the entire
site. We were not confident on the data recovered on kafka compared to oracle
database. We had 3 kafka nodes with no mirror maker. My understanding is
replication and mirror maker works until there is no lag on replication.
There is no guarantee of data loss. I never tested mirror maker with compact
topics. Thanks, pt On Sun, Aug 9, 2020 at 7:52 AM Liam
Clarke-Hutchinson < liam.cla...@adscale.co.nz> wrote: > Hi Dor,
> > There are multiple approaches. > > 1) Clone your Kafka
broker volumes > 2) Use Kafka Connect to stream all data to a different
storage system such > as Hadoop, S3, etc. > 3) Use Mirrormaker to
replicate all data to a backup cluster. > > Which approach is right for
you really depends on your needs, but > generally, if you have enough nodes
in your clusters, and a correct > replication setting for a topic, you
won't need to backup Kafka. As a rule > of thumb, a topic with a
replication factor of N can survive N - 1 node > failures without data
loss. > > If you can provide more information about the problems
you're trying to > solve, our advice can be more directed :) >
> Kind regards, > > Liam Clarke-Hutchinson > > On Sun, Aug
9, 2020 at 11:43 PM Dor Ben Dov <dor.ben-...@amdocs.com> > wrote:
> > > Hi All, > > What is the best recommended way, and tool
to backup kafka in production? > > Regards, > > Dor > >
> > This email and the information contained herein is proprietary and
> > confidential and subject to the Amdocs Email Terms of Service, which
you > > may review at www.amdocs.com www.amdocs.com < > >
www.amdocs.com www.amdocs.com > > > > >
