Thanks Tom. I think thats good enough for my needs On Thu, Jun 30, 2016 at 4:20 PM, Tom Crayford <tcrayf...@heroku.com> wrote:
> The default cleanup policy is delete, which is the regular time based > retention. > > On Thursday, 30 June 2016, Sathyakumar Seshachalam < > sathyakumar_seshacha...@trimble.com> wrote: > > > Or may be am wrong, and Log cleaner only picks up topics with a > > cleanup.policy. > > From the documentation it is not very obvious what the behaviour is. > > > > On Thu, Jun 30, 2016 at 10:33 AM, Sathyakumar Seshachalam < > > sathyakumar_seshacha...@trimble.com <javascript:;>> wrote: > > > > > Hi, > > > > > > Thanks for the response. > > > > > > I still like to know what happens for topics which have not defined a > > > cleanup.policy. > > > I assume the default value is compact. And hence all topic's logs will > be > > > compacted which I want to avoid. > > > > > > Am running 0.9.0, So will have to manually set log.cleaner.enable=true > > > > > > Regards, > > > Sathya > > > > > > On Thu, Jun 30, 2016 at 10:20 AM, Manikumar Reddy < > > > manikumar.re...@gmail.com <javascript:;>> wrote: > > > > > >> Hi, > > >> > > >> Kafka internally creates the offsets topic (__consumer_offsets) with > > >> compact mode on. > > >> From 0.9.0.1 onwards log.cleaner.enable=true by default. This means > > >> topics > > >> with a > > >> cleanup.policy=compact will now be compacted by default, > > >> > > >> You can tweak the offset topic configuration by using below props > > >> offsets.topic.compression.codec > > >> offsets.topic.num.partitions > > >> offsets.topic.replication.factor > > >> offsets.topic.segment.bytes > > >> offsets.retention.minutes > > >> offsets.retention.check.interval.ms > > >> > > >> > > >> Thanks > > >> Manikumar > > >> > > >> On Thu, Jun 30, 2016 at 9:49 AM, Sathyakumar Seshachalam < > > >> sathyakumar_seshacha...@trimble.com <javascript:;>> wrote: > > >> > > >> > Am little confused about how log cleaner works. My use case is that > I > > >> want > > >> > to compact just selected topics (or in my case just the internal > topic > > >> > __consumers_offsets and want to leave other topics as is). > > >> > > > >> > Whats the right settings/configuration for this to happen. > > >> > > > >> > As I understand log cleaner enable/disable is a global setting. And > my > > >> > understanding is that they will clean all logs (compact logs based > on > > >> > cleanup policy), and so all topics' clean up policy will be > considered > > >> and > > >> > hence compacted - compact being the default policy. Is this correct > ? > > >> > > > >> > I have set all topic's retention duration to be a really > exorbitantly > > >> high > > >> > value. Does it mean __consumer_offsets wont be compacted at all ? If > > so, > > >> > how to set retention time just for offset topic it being an internal > > >> topic. > > >> > > > >> > Regards, > > >> > Sathya > > >> > > > >> > > > > > > > > >
