If there are no "dirty" logs then the cleaner does not log anything.
You can try changing the dirty ratio config (min.cleanable.dirty.ratio) to something smaller than the default (which is 0.5). Joel On Tue, Aug 26, 2014 at 03:56:20PM -0400, Philippe Laflamme wrote: > Yes, and in order to "force" it to compact regardless of the volume, we've > set the "segment.ms" configuration key on the topic. According to the > docs[1], that should force a compaction at a certain time interval. > > We're seeing the segment rolling, but not the compaction. > > [1]http://kafka.apache.org/documentation.html#configuration > > > On Tue, Aug 26, 2014 at 2:20 PM, Guozhang Wang <wangg...@gmail.com> wrote: > > > Log cleaner will only wakeup and start the cleaning work when there are > > logs that are "dirty" enough to be cleaned. So if the topic-partitions does > > not get enough traffic to make it dirty the log cleaner will not kicks in > > to that partition again. > > > > Guozhang > > > > > > On Tue, Aug 26, 2014 at 9:02 AM, Philippe Laflamme <plafla...@hopper.com> > > wrote: > > > > > Here's the thread dump: > > > https://gist.github.com/plaflamme/634411b162f56d8f48f6 > > > > > > There's a log-cleaner thread sleeping. Would there be any reason why it's > > > not writing to it's log-cleaner.log file if it's still running? > > > > > > We are not using compression (unless it's on by default?) > > > > > > Thanks, > > > Philippe > > > > > > > > > On Tue, Aug 26, 2014 at 11:25 AM, Guozhang Wang <wangg...@gmail.com> > > > wrote: > > > > > > > Hello Philippe, > > > > > > > > You can get a thread dump and check if the log cleaner thread is still > > > > alive, or it is blocked. > > > > > > > > Also, are you using some compression on the messages stored on server? > > > > > > > > Guozhang > > > > > > > > Gu > > > > > > > > > > > > On Tue, Aug 26, 2014 at 8:15 AM, Philippe Laflamme < > > plafla...@hopper.com > > > > > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > We're using compaction on some of our topics. The log cleaner output > > > > showed > > > > > that it kicked in when the broker was restarted. But now after > > several > > > > > months of uptime, the log cleaner output is empty. The compacted > > topics > > > > > segment files don't seem to be cleaned up (compacted) anymore. > > > > > > > > > > If there anyway to confirm that the log cleaner is really stopped > > (the > > > > log > > > > > file doesn't mention any shutdown / error)? > > > > > > > > > > I've searched the mailing list about a similar problem but found > > > nothing. > > > > > Is there anything that might explain our issue? We're using 0.8.1.1 > > and > > > > the > > > > > topics have very low traffic. > > > > > > > > > > Philippe > > > > > > > > > > > > > > > > > > > > > -- > > > > -- Guozhang > > > > > > > > > > > > > > > -- > > -- Guozhang > >
