Hi All,
I have a kafka cluster with 3 brokers running in kubernetes. kafka version
i am using is 2.7.1
Frequently we are observing that log cleaner is not able to comapct and
clean log of some partitions. it mark the partition as uncleanable.
Following is the error message we see in the log...
[2021-11-14 23:04:32,215] WARN [kafka-log-cleaner-thread-0]: Unexpected
exception thrown when cleaning log Log(dir=/home/kafka/data/order-0,
topic=order, partition=0, highWatermark=50158328,
lastStableOffset=50158328, logStartOffset=0, logEndOffset=50158328).
Marking its partition (order-0) as uncleanable (kafka.log.LogCleaner)
kafka.log.LogCleaningException: Record size 0 is less than the minimum
record overhead (14)
at
kafka.log.LogCleaner$CleanerThread.cleanFilthiestLog(LogCleaner.scala:357)
at
kafka.log.LogCleaner$CleanerThread.tryCleanFilthiestLog(LogCleaner.scala:333)
at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:322)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
Caused by: org.apache.kafka.common.errors.CorruptRecordException: Record
size 0 is less than the minimum record overhead (14)
If i delete the topic and recrete it partition remain uncleanable unless
brokers are restarted after deleteing and recreating the topic.
what is causing this issue and how to fix it ?
Thanks,
Dhirendra.
