[
https://issues.apache.org/jira/browse/KAFKA-20030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated KAFKA-20030:
-----------------------------------
Description:
We should update the documentation to list important considerations when users
dynamically increase the number of partitions for a topic.
Key points to include:
*Key Distribution Changes:* The default partitioner’s mapping logic (e.g., hash
of the key) changes when the partition count increases, which may affect
message ordering for existing keys.
*Potential Data Loss with {{{}latest{}}}:* Existing consumers configured with
{{auto.offset.reset=latest}} might miss messages produced to the new partitions
during the window between partition creation and consumer discovery.
*Metadata Propagation Delay:* New partitions are not immediately visible to
producers and consumers due to metadata refresh intervals.
was:
We should update the documentation to list important considerations when users
dynamically increase the number of partitions for a topic.
Key points to include:
# *Key Distribution Changes:* The default partitioner’s mapping logic (e.g.,
hash of the key) changes when the partition count increases, which may affect
message ordering for existing keys.
# *Potential Data Loss with {{{}latest{}}}:* Existing consumers configured
with {{auto.offset.reset=latest}} might miss messages produced to the new
partitions during the window between partition creation and consumer discovery.
# *Metadata Propagation Delay:* New partitions are not immediately visible to
producers and consumers due to metadata refresh intervals.
> Document considerations and side effects of dynamic partition expansion
> -----------------------------------------------------------------------
>
> Key: KAFKA-20030
> URL: https://issues.apache.org/jira/browse/KAFKA-20030
> Project: Kafka
> Issue Type: Improvement
> Reporter: Chia-Ping Tsai
> Assignee: Lan Ding
> Priority: Minor
>
> We should update the documentation to list important considerations when
> users dynamically increase the number of partitions for a topic.
> Key points to include:
> *Key Distribution Changes:* The default partitioner’s mapping logic (e.g.,
> hash of the key) changes when the partition count increases, which may affect
> message ordering for existing keys.
> *Potential Data Loss with {{{}latest{}}}:* Existing consumers configured with
> {{auto.offset.reset=latest}} might miss messages produced to the new
> partitions during the window between partition creation and consumer
> discovery.
> *Metadata Propagation Delay:* New partitions are not immediately visible to
> producers and consumers due to metadata refresh intervals.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
