jsancio opened a new pull request, #12884: URL: https://github.com/apache/kafka/pull/12884
Now that Kafka is generating a metadata snapshot every hour and the default metadata retention is to delete snapshots after 7 days, every cluster metadata partition will have 168 (1 snapshot per hour * 24 hours per day * 7 days) snapshots. If we assume that in most cases the size of the snapshot is determined by the number of partitions in a cluster, a cluster with 100K partitions will have a snapshot size of roughly 10MB (100 bytes per partition * 100k partitions). For this kind of clusters the cluster metadata partition will always consume around 1.7GB. KIP-876 changed the default value for metadata.max.retention.bytes to 100MB. This should limit the size of the cluster metadata partition for large clusters but keep 7 days worth of snapshots for small clusters. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
