Hi There is a couple of metrics and log produced by the log cleaner https://github.com/apache/kafka/blob/1.0/core/src/main/scala/kafka/log/LogCleaner.scala
You can try to monitor that (metrics could be fetched with JMX and for the log you can tweak the log4j properties) Don't forget that the log cleaner can only work on "inactive" segments, i.e a topic/partition should have 2 segments in order for the log cleaner to run Best regards On Fri, Apr 12, 2019 at 6:49 PM Karthik Yadav <thekarthikya...@gmail.com> wrote: > I have a few brokers which take care of the data of a few applications. I > have the log-cleaner for all the brokers enabled with thecleanup.policy as > delete. There was an issue a few weeks back where the log-cleaner stopped > working in one of the brokers and this led complete disk usage, and it > wasn't detectable. > > I am looking for an efficient way to *check if the log-cleaner of a broker > has stopped working*. > > Firstly, I am using the *Kafka 1.0.2 version*. > > As a simple method, I have tried looking into the logs of the log-cleaner, > and when the log wouldn't update for a considerable amount of time meant > the log-cleaner stopped working. > > But some of the brokers' log-cleaner's log was not getting updated for some > reason even though the log-cleaner was running(understood because the size > of the log of the applications didn't keep increasing after a considerable > amount of time). > > I have tried playing around with a few configs, > log.cleaner.min.cleanable.ratiolog.retention.check.interval.ms etc. on why > the log-cleaner's log was not getting updated, but I couldn't conclude on > anything. *Any ideas on how to check if the log-cleaner is running or why > the log-cleaner's log wouldn't log?* > > TLDR; > > Had some trouble as the log-cleaner of a Kafka broker stopped and couldn't > detect it in time. Looking for an efficient method to check the termination > of the log-cleaner. > ᐧ >
