Dao Quang Minh created KAFKA-7569:
-------------------------------------
Summary: Kafka doesnt appear to cleanup dangling partitions
Key: KAFKA-7569
URL: https://issues.apache.org/jira/browse/KAFKA-7569
Project: Kafka
Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Dao Quang Minh
In our current cluster running kafka 1.0.0, we recently observed that Kafka
doesnt cleanup dangling partitions ( i.e. partion data on disk, but partition
is not assigned to the current broker anymore ).
For example of the dangling partition data, we have:
{code}
total 26G
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:19 00000000352433304663.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:16 00000000352414164340.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:24 00000000352466972892.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:23 00000000352457368236.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:17 00000000352423709566.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:21 00000000352447702369.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:20 00000000352442921890.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:22 00000000352452551548.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:17 00000000352418945305.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:18 00000000352428477361.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:15 00000000352409416538.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:24 00000000352462192103.log
-rw-r--r-- 1 kafka kafka 1.9G Aug 6 16:20 00000000352438136012.log
-rw-r--r-- 1 kafka kafka 1.8G Aug 6 17:43 00000000352471757311.log
-rw-r--r-- 1 kafka kafka 10M Oct 16 21:44 00000000352471757311.index
-rw-r--r-- 1 kafka kafka 10M Oct 16 21:44 00000000352471757311.timeindex
drwxr-xr-x 2 kafka kafka 4.0K Oct 8 15:27 .
drwxr-xr-x 49 kafka kafka 4.0K Oct 30 11:21 ..
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352414164340.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352423709566.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352433304663.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352447702369.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352457368236.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352466972892.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352409416538.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352418945305.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352428477361.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352438136012.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352442921890.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352452551548.timeindex
-rw-r--r-- 1 kafka kafka 2.3K Oct 16 21:44 00000000352462192103.timeindex
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352414164340.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352423709566.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352433304663.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352447702369.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352457368236.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352466972892.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352409416538.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352418945305.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352428477361.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352438136012.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352442921890.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352452551548.index
-rw-r--r-- 1 kafka kafka 1.5K Oct 16 21:44 00000000352462192103.index
-rw-r--r-- 1 kafka kafka 20 Aug 6 16:23 leader-epoch-checkpoint
-rw-r--r-- 1 kafka kafka 10 Aug 6 16:24 00000000352466972892.snapshot
-rw-r--r-- 1 kafka kafka 10 Aug 6 16:24 00000000352471757311.snapshot
-rw-r--r-- 1 kafka kafka 10 Oct 8 15:27 00000000352476186724.snapshot
{code}
I'm unsure how we ended up in this situation as partition data should be marked
as removed and eventually remove when it's not assigned to the broker anymore.
But in this edge case, should Kafka detect that automatically when it loads the
partition and re-mark it as to be deleted again ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
