If I recall correctly, setting log.retention.ms and log.retention.bytes to -1 disables both.
On Fri, Jul 10, 2015 at 1:55 PM, Daniel Schierbeck < daniel.schierb...@gmail.com> wrote: > > > On 10. jul. 2015, at 15.16, Shayne S <shaynest...@gmail.com> wrote: > > > > There are two ways you can configure your topics, log compaction and with > > no cleaning. The choice depends on your use case. Are the records > uniquely > > identifiable and will they receive updates? Then log compaction is the > way > > to go. If they are truly read only, you can go without log compaction. > > I'd rather be free to use the key for partitioning, and the records are > immutable — they're event records — so disabling compaction altogether > would be preferable. How is that accomplished? > > > > We have a small processes which consume a topic and perform upserts to > our > > various database engines. It's easy to change how it all works and simply > > consume the single source of truth again. > > > > I've written a bit about log compaction here: > > http://www.shayne.me/blog/2015/2015-06-25-everything-about-kafka-part-2/ > > > > On Fri, Jul 10, 2015 at 3:46 AM, Daniel Schierbeck < > > daniel.schierb...@gmail.com> wrote: > > > >> I'd like to use Kafka as a persistent store – sort of as an alternative > to > >> HDFS. The idea is that I'd load the data into various other systems in > >> order to solve specific needs such as full-text search, analytics, > indexing > >> by various attributes, etc. I'd like to keep a single source of truth, > >> however. > >> > >> I'm struggling a bit to understand how I can configure a topic to retain > >> messages indefinitely. I want to make sure that my data isn't deleted. > Is > >> there a guide to configuring Kafka like this? > >> >
