> On 10. jul. 2015, at 23.03, Jay Kreps <j...@confluent.io> wrote: > > If I recall correctly, setting log.retention.ms and log.retention.bytes to > -1 disables both.
Thanks! > > On Fri, Jul 10, 2015 at 1:55 PM, Daniel Schierbeck < > daniel.schierb...@gmail.com> wrote: > >> >>> On 10. jul. 2015, at 15.16, Shayne S <shaynest...@gmail.com> wrote: >>> >>> There are two ways you can configure your topics, log compaction and with >>> no cleaning. The choice depends on your use case. Are the records >> uniquely >>> identifiable and will they receive updates? Then log compaction is the >> way >>> to go. If they are truly read only, you can go without log compaction. >> >> I'd rather be free to use the key for partitioning, and the records are >> immutable — they're event records — so disabling compaction altogether >> would be preferable. How is that accomplished? >>> >>> We have a small processes which consume a topic and perform upserts to >> our >>> various database engines. It's easy to change how it all works and simply >>> consume the single source of truth again. >>> >>> I've written a bit about log compaction here: >>> http://www.shayne.me/blog/2015/2015-06-25-everything-about-kafka-part-2/ >>> >>> On Fri, Jul 10, 2015 at 3:46 AM, Daniel Schierbeck < >>> daniel.schierb...@gmail.com> wrote: >>> >>>> I'd like to use Kafka as a persistent store – sort of as an alternative >> to >>>> HDFS. The idea is that I'd load the data into various other systems in >>>> order to solve specific needs such as full-text search, analytics, >> indexing >>>> by various attributes, etc. I'd like to keep a single source of truth, >>>> however. >>>> >>>> I'm struggling a bit to understand how I can configure a topic to retain >>>> messages indefinitely. I want to make sure that my data isn't deleted. >> Is >>>> there a guide to configuring Kafka like this? >>
