Log cleaner will only wakeup and start the cleaning work when there are logs that are "dirty" enough to be cleaned. So if the topic-partitions does not get enough traffic to make it dirty the log cleaner will not kicks in to that partition again.
Guozhang On Tue, Aug 26, 2014 at 9:02 AM, Philippe Laflamme <plafla...@hopper.com> wrote: > Here's the thread dump: > https://gist.github.com/plaflamme/634411b162f56d8f48f6 > > There's a log-cleaner thread sleeping. Would there be any reason why it's > not writing to it's log-cleaner.log file if it's still running? > > We are not using compression (unless it's on by default?) > > Thanks, > Philippe > > > On Tue, Aug 26, 2014 at 11:25 AM, Guozhang Wang <wangg...@gmail.com> > wrote: > > > Hello Philippe, > > > > You can get a thread dump and check if the log cleaner thread is still > > alive, or it is blocked. > > > > Also, are you using some compression on the messages stored on server? > > > > Guozhang > > > > Gu > > > > > > On Tue, Aug 26, 2014 at 8:15 AM, Philippe Laflamme <plafla...@hopper.com > > > > wrote: > > > > > Hi, > > > > > > We're using compaction on some of our topics. The log cleaner output > > showed > > > that it kicked in when the broker was restarted. But now after several > > > months of uptime, the log cleaner output is empty. The compacted topics > > > segment files don't seem to be cleaned up (compacted) anymore. > > > > > > If there anyway to confirm that the log cleaner is really stopped (the > > log > > > file doesn't mention any shutdown / error)? > > > > > > I've searched the mailing list about a similar problem but found > nothing. > > > Is there anything that might explain our issue? We're using 0.8.1.1 and > > the > > > topics have very low traffic. > > > > > > Philippe > > > > > > > > > > > -- > > -- Guozhang > > > -- -- Guozhang
