Check the log-cleaner.log file on the server. When the thread runs you'll see output for every partition it compacts and the compaction ratio it achieved.
The __consumer_offsets topic is compacted, I see log output from it being compacted frequently. Depending on your settings for the topic it may take a while for it to compact. Compaction doesn't occur on the current log segment. Look at these settings for the topic, "segment.bytes" and "segment.ms". Lower them to force quicker compaction. On 0.8.2.2, occasionally, the compaction thread would die and then the __consumer_offets topic would grow out of control. Kafka logs the thread death in the log-cleaner.log. On Fri, Jul 29, 2016 at 4:10 PM David Yu <guans...@gmail.com> wrote: > Hi, > > We are using Kafka 0.9.0.0. One of our topic is set to use log compaction. > We have also set log.cleaner.enable. However, we suspected that the topic > is not being compacted. > > What is the best way for us to verify the compaction is happening? > > Thanks, > David >
