Thanks Joel for the quick turnaround! 1st, I took a look at "max-dirty-percent" and noticed it is "100", so that's a good sign that it *should* have something to clean.
2nd, I didn't realize the cleaner messages went to a different log! However I have NO log-cleaner.log file in the normal logs directory. It does seem to be configured in log4j (we haven't changed any of the defaults here): log4j.appender.cleanerAppender=org.apache.log4j.DailyRollingFileAppender log4j.appender.cleanerAppender.DatePattern='.'yyyy-MM-dd-HH log4j.appender.cleanerAppender.File=log-cleaner.log log4j.appender.cleanerAppender.layout=org.apache.log4j.PatternLayout log4j.appender.cleanerAppender.layout.ConversionPattern=[%d] %p %m (%c)%n log4j.logger.kafka.log.LogCleaner=INFO, cleanerAppender log4j.additivity.kafka.log.LogCleaner=false 3rd, no messages using compaction, all messages have a key. So, lastly the thread dump. Not sure exactly how to do that or what thread I'd be looking for specifically... Found a suggestion to run Jstack -l <JAVA_PID> > jstack.out So I did that, and looked for anything containing "Clean" or "clean" and no matches. I will be trying to set the log level for LogCleaner to debug to see if that helps at all. Thanks in advance, Thunder -----Original Message----- From: Joel Koshy [mailto:jjkosh...@gmail.com] Sent: Tuesday, February 03, 2015 3:07 PM To: users@kafka.apache.org Subject: Re: Turning on cleanup.policy=compact for a topic => not starting cleanup ? - Can you check the log cleaner logs? - Do you have any compressed messages in your log? Or messages without a key? - By default it is in a log-cleaner.log file unless you modified that. - Can you take a thread-dump to see if the log cleaner is still alive? - Also, there is an mbean that you can poke: "max-dirty-percent" under log cleaner. Can you check on its value? Thanks, Joel On Tue, Feb 03, 2015 at 10:59:27PM +0000, Thunder Stumpges wrote: > Hi guys, I am having a difficult time getting the log compaction to run on a > topic I created initially with cleanup.policy=delete. Here's the details: > > 3 brokers, 0.8.1.1 all have log.cleaner.enable=true (other than this there > are no non-defaults set in our server.properties) > > Create the topic first with policy=delete and a short window. Wanted to see > that the segments were being created and deleted by the "normal" delete > retention. This seemed to work fine. New segments created every 6 seconds > while I add some test data, then the cleaner comes along and deletes log > segments after 1min. > > sudo bin/kafka-topics.sh --zookeeper zk1:2181/kafka --create --topic > dev_testcompact --partitions 10 --replication-factor 3 --config > cleanup.policy=delete --config segment.ms=6000 --config retention.ms=60000 > > > Then I tried to update the policy to do compact: > sudo bin/kafka-topics.sh --zookeeper zk1:2181/kafka --alter --topic > dev_testcompact --config cleanup.policy=compact --config > min.cleanable.dirty.ratio=0.3 -config delete.retention.ms=60000 > > From this point, the deletion retention stopped cleaning up, but the compact > cleaner doesn't seem to be running at all. I was expecting that after 15 > seconds (default for setting log.cleaner.backoff.ms) the cleaner would come > in and compact the old segments. I now have generated 400 rows across 40 keys > (so there is plenty to compact). Segments are being created every 6 seconds > per the config. > > I have also tried removing the deletion related config: > > sudo bin/kafka-topics.sh --zookeeper zk1:2181/kafka --alter --topic > dev_testcompact --deleteConfig retention.ms > > I then restarted one of the three brokers wondering if it needed to restart > to pick up new configs, still no clenup. What am I doing wrong?!
