Hi, Kafka internally creates the offsets topic (__consumer_offsets) with compact mode on. >From 0.9.0.1 onwards log.cleaner.enable=true by default. This means topics with a cleanup.policy=compact will now be compacted by default,
You can tweak the offset topic configuration by using below props offsets.topic.compression.codec offsets.topic.num.partitions offsets.topic.replication.factor offsets.topic.segment.bytes offsets.retention.minutes offsets.retention.check.interval.ms Thanks Manikumar On Thu, Jun 30, 2016 at 9:49 AM, Sathyakumar Seshachalam < sathyakumar_seshacha...@trimble.com> wrote: > Am little confused about how log cleaner works. My use case is that I want > to compact just selected topics (or in my case just the internal topic > __consumers_offsets and want to leave other topics as is). > > Whats the right settings/configuration for this to happen. > > As I understand log cleaner enable/disable is a global setting. And my > understanding is that they will clean all logs (compact logs based on > cleanup policy), and so all topics' clean up policy will be considered and > hence compacted - compact being the default policy. Is this correct ? > > I have set all topic's retention duration to be a really exorbitantly high > value. Does it mean __consumer_offsets wont be compacted at all ? If so, > how to set retention time just for offset topic it being an internal topic. > > Regards, > Sathya >
