If I recall correctly, setting log.retention.ms and log.retention.bytes to
-1 disables both.

On Fri, Jul 10, 2015 at 1:55 PM, Daniel Schierbeck <
daniel.schierb...@gmail.com> wrote:

>
> > On 10. jul. 2015, at 15.16, Shayne S <shaynest...@gmail.com> wrote:
> >
> > There are two ways you can configure your topics, log compaction and with
> > no cleaning. The choice depends on your use case. Are the records
> uniquely
> > identifiable and will they receive updates? Then log compaction is the
> way
> > to go. If they are truly read only, you can go without log compaction.
>
> I'd rather be free to use the key for partitioning, and the records are
> immutable — they're event records — so disabling compaction altogether
> would be preferable. How is that accomplished?
> >
> > We have a small processes which consume a topic and perform upserts to
> our
> > various database engines. It's easy to change how it all works and simply
> > consume the single source of truth again.
> >
> > I've written a bit about log compaction here:
> > http://www.shayne.me/blog/2015/2015-06-25-everything-about-kafka-part-2/
> >
> > On Fri, Jul 10, 2015 at 3:46 AM, Daniel Schierbeck <
> > daniel.schierb...@gmail.com> wrote:
> >
> >> I'd like to use Kafka as a persistent store – sort of as an alternative
> to
> >> HDFS. The idea is that I'd load the data into various other systems in
> >> order to solve specific needs such as full-text search, analytics,
> indexing
> >> by various attributes, etc. I'd like to keep a single source of truth,
> >> however.
> >>
> >> I'm struggling a bit to understand how I can configure a topic to retain
> >> messages indefinitely. I want to make sure that my data isn't deleted.
> Is
> >> there a guide to configuring Kafka like this?
> >>
>

Reply via email to