Re: .deleted file descriptors

2015-03-02 Thread Guangle Fan
I see DEL file descriptors as well. java2978 root DELREG 202,16 1074192884 /mnt/data1/kafka/flirt_tasks-1/000143874814.index.deleted Using java from Oracle java version "1.7.0_72" Java(TM) SE Runtime Environment (build 1.7.0_72-b14) Java HotSpot(TM) 64-Bit

Re: .deleted file descriptors

2015-03-01 Thread Jaikiran Pai
One thing to remember is that the .index files are memory-mapped [1] which in Java means that the file descriptors may not be released even when the program is done using it. A garbage collection is expected to close such resources, but forcing a System.gc() is only a hint and thus doesn't guar

Re: .deleted file descriptors

2015-03-01 Thread Guozhang Wang
Did you check if log.delete.delay.ms is set to 6? Guozhang On Sun, Mar 1, 2015 at 9:00 PM, Guangle Fan wrote: > Slightly different from what I observed. > > Broker box has 800GB disk space. By setting the appropriate log retention, > it's supposed to hold the log size. But then the usage of

Re: .deleted file descriptors

2015-03-01 Thread Guangle Fan
Slightly different from what I observed. Broker box has 800GB disk space. By setting the appropriate log retention, it's supposed to hold the log size. But then the usage of disk hits 90%, and by doing nothing but restarting broker server. It free 40% disk space. It's for sure the speed of the tra

Re: .deleted file descriptors

2015-03-01 Thread Mayuresh Gharat
Also I suppose when the broker starts up it will remove the files that are marked with suffix .deleted and that's why you can see the free disk space on restarting. Guozhang can correct me if I am wrong. Thanks, Mayuresh On Sat, Feb 28, 2015 at 9:27 PM, Guozhang Wang wrote: > Guangle, > > The

Re: .deleted file descriptors

2015-02-28 Thread Guozhang Wang
Guangle, The deletion of the segment log / index files are async, i.e. when Kafka decide to clean the logs, it only adds a suffix ".deleted" to the files such that it will not be access any more by other Kafka threads. The actual file deletion will be executed later, with period controlled by " fi

.deleted file descriptors

2015-02-27 Thread Guangle Fan
Hi, After Kafka cleaned .log / .index files based on topic retention. I can still lsof a lot of .index.deleted files. And df shows usage on disk space is accumulated to full. When this happened, just by restarting broker, it will immediately free those disk space. I seems to me kafka after cleani