[
https://issues.apache.org/jira/browse/KAFKA-9590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gokul Ramanan Subramanian reassigned KAFKA-9590:
------------------------------------------------
Assignee: Gokul Ramanan Subramanian
> Add read-only configuration to limit number of topic partitions
> ---------------------------------------------------------------
>
> Key: KAFKA-9590
> URL: https://issues.apache.org/jira/browse/KAFKA-9590
> Project: Kafka
> Issue Type: Improvement
> Components: admin
> Reporter: Gokul Ramanan Subramanian
> Assignee: Gokul Ramanan Subramanian
> Priority: Minor
>
> Currently, a Kafka cluster does not limit the number of topic partitions
> despite the fact that on a given system, having more than a certain number of
> topic partitions (without even any messages written / read) renders the
> cluster unusable.
>
> The ask here is to have a read-only Kafka configuration named something like
> *max.broker.partitions* (an integer, which applies at the broker level and
> defaults to INT.MAX) that the administrator the cluster can set to ensure
> smooth operation.
>
> Things to keep in mind:
> # If a broker already has more than *max.broker.partitions* number of
> partitions at the time when the *max.broker.partitions* configuration is
> first applied on the broker, the broker should disallow any more topic
> partitions. This allows for graceful reassignments and in-place upgrade to a
> Kafka version that supports the *max.broker.partitions* configuration.
> # Different brokers can technically have different *max.broker.partitions*
> in case of a heterogenous cluster, where not all broker nodes operate with
> the same system limits (such as CPU, memory etc.)
> # When creating a topic partition, the controller has to identify brokers
> which will replicate the topic partition. Currently, the controller
> distributes the leadership of the topic partitions to ensure uniform load.
> This algorithm will need to account for the *max.broker.partitions*
> configuration.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
