Append a related Issue
### Issue both partitioned DLQ and non-partitioned DLQ were created
automatically.
- Background
- Broker settings: `{allowAutoTopicCreation=true, defaultNumPartitions=1,
allowAutoTopicCreationType=partitioned}`
- client version `< 3.1`
- broker version `>= 3.1`
- Flows of creating DLQ.
- Trigger the DLQ creation.
- Get partitioned metadata of DLQ: `partitions: 1`.
- Broker created the partitioned metadata at this step.
- Before `3.1`, the client mistakenly assumed
`{tenant}/{namespace}/{topic}-partition-0-{subscription}-DLQ-partition-0`
is `{tenant}/{namespace}/{topic}-partition-0-{subscription}-DLQ` due to the
bug that was fixed by https://github.com/apache/pulsar/pull/19841.
- Start a dead-letter producer under
`{tenant}/{namespace}/{topic}-partition-0-{subscription}-DLQ`
- Triggered a non-partitioned topic creation.
On Tue, May 14, 2024 at 2:56 AM Yubiao Feng <yubiao.f...@streamnative.io>
wrote:
> Hi community
>
> I want to change the behavior of DLQ/Retry Topic auto creation.
>
> ### Motivation
> After you set `defaultNumPartitions` to `16`, you will get `16` partitions
> per topic, and `16*16` DLQ partitions per subscription, you will get a huge
> number of DLQs if you have more than one subscription under one topic. For
> example:
> - create topic `t1` with `16` partitions
> - you will get `t1-partition-0`, `t1-partition-1`...`tp-partition-15`.
> - after you enable DLQ, you will get the following `16*16` DLQs per
> subscription
> - `t1-partition-0-{subscription}-DLQ-partition-0`
> - `t1-partition-0-{subscription}-DLQ-partition-1`
> - ...
> - `t1-partition-15-{subscription}-DLQ-partition-15`
> - `t1-partition-1-{subscription}-DLQ-partition-0`
> - `t1-partition-1-{subscription}-DLQ-partition-1`
> - ...
> - ...
> - ...
> - `t1-partition-15-{subscription}-partition-15`
>
> ### Goal
> - Do not create partitioned DLQs/Retry topics automatically.
> - Users can also create partitioned DLQ manually if they need it.
>
> I have drafted a PR, please take a look
> - https://github.com/apache/pulsar/pull/22705
>
> Thanks
> Yubiao Feng
>
