It sounds like you are trying to forcibly delete the files that build up
the segments used by the partitions. If that is the case then I would
recommend not using external tools and leave to Kafka manage its
filesystem. If you set the retention policy (either by size or time) in
your topics the log cleaner threads will take care of purging any unused
space from the segments automatically. No need to have a external
process that clean those out.
In order to recover from this; kill any processes that might be acting
upon those files and then restart your Kafka brokers once again. Only
the broker JVM should have file handles to the segments to avoid any
lock problems during file write/read.
Thanks,
-- Ricardo
On 6/15/20 4:50 AM, Ncir, Kawther (external - Service) wrote:
Hi,
I am working on a project using kafka broker and I face a problem with kafka
log retention ,
We need to delete logs every 24h but after the retention time the broker was
stopped and make an issue like this :
"ERROR Uncaught exception in scheduled task 'kafka-log-retention'
(kafka.utils.KafkaScheduler)
org.apache.kafka.common.errors.KafkaStorageException: Error while deleting segme
nts for test-1 in dir C:\tmp\log Caused by: java.nio.file.FileSystemException:
C:\tmp\log\test-1\0000000000000000
0000.timeindex -> C:\tmp\log\test-1\00000000000000000000.timeindex.deleted: The
process cannot access the file because it is being used by another process."
Could you please support me to find a solution.
Regards,
kawther
