Jimmy Wang created KAFKA-15118:
----------------------------------
Summary: Need for a Centralized Configuration Management System in
Apache Kafka
Key: KAFKA-15118
URL: https://issues.apache.org/jira/browse/KAFKA-15118
Project: Kafka
Issue Type: Improvement
Components: core
Affects Versions: 2.7.1
Reporter: Jimmy Wang
Hi all,
In our use of Apache Kafka, we found something strange following:
Despite our partitions being well-distributed across brokers, we noticed a
significant discrepancy in disk usage between different brokers. Specifically,
the same partition takes up different amounts of disk space on different
brokers. Upon investigating, we found that the root cause of this discrepancy
was a variation in {{log.retention.hours}} setting between different brokers.
On the one hand, we know that we should make sure that the server.properties
should be same across the whole cluster. But on the other hand, could kafka
provide guarantee or advance check to make sure all the configurations of
brokers to be the same to avoid such situations?
Here is some of my opinions:
# Provide centralized configuration center or just manage them in internal
topics(kraft mode) or zookeeper like dynamic configuration.
# Warn users about potential inconsistencies during the broker startup process.
Best regards,
Jimmy Wang
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
