Or may be am wrong, and Log cleaner only picks up topics with a cleanup.policy. >From the documentation it is not very obvious what the behaviour is.
On Thu, Jun 30, 2016 at 10:33 AM, Sathyakumar Seshachalam < sathyakumar_seshacha...@trimble.com> wrote: > Hi, > > Thanks for the response. > > I still like to know what happens for topics which have not defined a > cleanup.policy. > I assume the default value is compact. And hence all topic's logs will be > compacted which I want to avoid. > > Am running 0.9.0, So will have to manually set log.cleaner.enable=true > > Regards, > Sathya > > On Thu, Jun 30, 2016 at 10:20 AM, Manikumar Reddy < > manikumar.re...@gmail.com> wrote: > >> Hi, >> >> Kafka internally creates the offsets topic (__consumer_offsets) with >> compact mode on. >> From 0.9.0.1 onwards log.cleaner.enable=true by default. This means >> topics >> with a >> cleanup.policy=compact will now be compacted by default, >> >> You can tweak the offset topic configuration by using below props >> offsets.topic.compression.codec >> offsets.topic.num.partitions >> offsets.topic.replication.factor >> offsets.topic.segment.bytes >> offsets.retention.minutes >> offsets.retention.check.interval.ms >> >> >> Thanks >> Manikumar >> >> On Thu, Jun 30, 2016 at 9:49 AM, Sathyakumar Seshachalam < >> sathyakumar_seshacha...@trimble.com> wrote: >> >> > Am little confused about how log cleaner works. My use case is that I >> want >> > to compact just selected topics (or in my case just the internal topic >> > __consumers_offsets and want to leave other topics as is). >> > >> > Whats the right settings/configuration for this to happen. >> > >> > As I understand log cleaner enable/disable is a global setting. And my >> > understanding is that they will clean all logs (compact logs based on >> > cleanup policy), and so all topics' clean up policy will be considered >> and >> > hence compacted - compact being the default policy. Is this correct ? >> > >> > I have set all topic's retention duration to be a really exorbitantly >> high >> > value. Does it mean __consumer_offsets wont be compacted at all ? If so, >> > how to set retention time just for offset topic it being an internal >> topic. >> > >> > Regards, >> > Sathya >> > >> > >
