Kamal Chandraprakash created KAFKA-15682:
--------------------------------------------
Summary: Ensure internal remote log metadata topic does not expire
its segments before deleting user-topic segments
Key: KAFKA-15682
URL: https://issues.apache.org/jira/browse/KAFKA-15682
Project: Kafka
Issue Type: Task
Reporter: Kamal Chandraprakash
One of the implementation of RemoteLogMetadataManager is
TopicBasedRemoteLogMetadataManager which uses an internal Kafka topic. Unlike
other internal topics which are compaction enabled, this topic is not enabled
with compaction and retention is set to unlimited.
Keeping this internal topic retention to unlimited is not practical in real
world use-case where the topic disk usage footprint grows large over a period
of time.
It is assumed that the user will set the retention to a reasonable time such
that it is the max of all the user-created topics (max + X). We can't just rely
on it and need an assertion before deleting the internal
{{__remote_log_metadata}} segments, otherwise there will be dangling remote log
segments which won't be cleared once all the brokers are restarted post the
topic truncation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
