Jason,

Kafka closes the handler of all delete files. Otherwise, the broker will
run out of file handler quickly.

Thanks,

Jun


On Wed, May 22, 2013 at 10:17 PM, Jason Rosenberg <j...@squareup.com> wrote:

> So, does this indicate kafka (or the jvm itself) is not aggressively
> closing file handles of deleted files?  Is there a fix for this?  Or is
> there not likely anything to be done?  What happens if the disk fills up
> with file handles for phantom deleted files?
>
> Jason
>
>
> On Wed, May 22, 2013 at 9:50 PM, Jonathan Creasy <j...@box.com> wrote:
>
> > It isn't uncommon if a process has an open file handle on a file that is
> > deleted, the space is not freed until the handle is closed. So restarting
> > the process that has a handle on the file would cause the space to be
> freed
> > also.
> >
> > You can troubleshoot that with lsof.
> > Normally, I see 2-4 log segments deleted every hour in my brokers.  I see
> > log lines like this:
> >
> > 2013-05-23 04:40:06,857  INFO [kafka-logcleaner-0] log.LogManager -
> > Deleting log segment 00000000035434043157.kafka from <redacted topic>
> >
> > However, it seems like if I restart the broker, a massive amount of disk
> > space is freed (without a corresponding flood of these log segment
> deleted
> > messages).  Is there an explanation for this?  Does kafka keep reference
> to
> > file segments around, and reuse them as needed or something?  And then or
> > restart, the references to those free segment files are dropped?
> >
> > Thoughts?
> >
> > This is with 0.7.2.
> >
> > Jason
> >
>

Reply via email to