Re: [DISCUSS] PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

I agree with you. wasAutoCreated seems a bit like a patch in light of the recent issues. On Fri, Jun 9, 2023 at 9:03 AM Michael Marshall wrote: > > In general, these problems hint to me that we need better definitions > > for "system" and "special" topics that break the rules of auto created >

Re: [DISCUSS] PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

> In general, these problems hint to me that we need better definitions > for "system" and "special" topics that break the rules of auto created > topics in well defined ways. This point reminds me of this discussion from a year and a half ago: https://lists.apache.org/thread/qgbpzr6o3k5rqbs2jvpk

Re: [DISCUSS] PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

Hi Michael, Enrico > The identifier is the name. The topics have `-DLQ` and `-RETRY` > as suffixes, which makes them "special" (which reminds me of this > thread [0]). We could choose to always make these non partitioned > or to make them configurable in some other way. I changed the design like

Re: [DISCUSS] PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

We've had several bugs related to this topic of auto created partitioned topics recently. These three PRs come to mind: * https://github.com/apache/pulsar/pull/20370 * https://github.com/apache/pulsar/pull/20392 * https://github.com/apache/pulsar/pull/20397 Those PRs rely on server side inference

Re: [DISCUSS] PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

Hi @Enrico > I think that it is better to add flags like: > - allowAutoTopiCreation: default "true", if "false" the broker > won't create the topic in any case > autoTopicCreationMode: undefined/partitioned/non-partitioned I agree with you, but now the problem is that we still need an identifier

Re: [DISCUSS] PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

Hi Enrico > One problem with partitioned topic and DLQ is that you > probably want the same number of partitions on the > side topic as the original topic, so probably you have > to carry that parameter (number of partitions). I understand your concern, but this problem does not exist. If the m

Re: [VOTE] PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

-1 I had missed the discussion thread. Please follow up there Enrico Il giorno ven 5 mag 2023 alle ore 17:04 Yubiao Feng ha scritto: > > Hi Pulsar Community > > This thread is to start the vote for PIP 259. > > Discussion: *https://lists.apache.org/thread/tp25lom2ggztljlo76krsldo270f293j >

Re: [VOTE] PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

-1 I had missed the discussion thread. Please follow up there Enrico Il giorno mer 31 mag 2023 alle ore 09:07 Yubiao Feng ha scritto: > > Hi Pulsar Community > > This thread is to start the vote for PIP 263. > I'm starting the voting process for PIP-263. > > DISCUSS: https://lists.apache.org/thr

Re: [DISCUSS] PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

Yubiao, I understand your problem: DLQ and retry topic don't play well with automatic topic creation. We are introducing wire protocol changes and also starting to add topic properties for every new topic (they come with a cost on metadata). I am not sure that adding the purposeOfAutoCreatedTopi

Re: PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

I forgot to add [vote] in the title of this email and sent another one On Wed, May 31, 2023 at 3:05 PM Yubiao Feng wrote: > Hi Pulsar Community > > This thread is to start the vote for PIP 263. > I'm starting the voting process for PIP-263. > > DISCUSS: https://lists.apache.org/thread/tp25lom2gg

[VOTE] PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

Hi Pulsar Community This thread is to start the vote for PIP 263. I'm starting the voting process for PIP-263. DISCUSS: https://lists.apache.org/thread/tp25lom2ggztljlo76krsldo270f293j PR: https://github.com/apache/pulsar/pull/20445 Thanks Yubiao Feng

PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

Hi Pulsar Community This thread is to start the vote for PIP 263. I'm starting the voting process for PIP-263. DISCUSS: https://lists.apache.org/thread/tp25lom2ggztljlo76krsldo270f293j PR: https://github.com/apache/pulsar/pull/20445 Thanks Yubiao Feng

[VOTE] PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

Hi Pulsar Community This thread is to start the vote for PIP 259. Discussion: *https://lists.apache.org/thread/tp25lom2ggztljlo76krsldo270f293j * Issue: https://github.com/apache/pulsar/issues/20033 Voting will stay open for at l

Re: [DISCUSS] PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

Hi Yubiao, Appreciate your effort in initiating this PIP. I believe these changes will address the existing issues and make DLQ and Retry Topic handling more efficient and straightforward. The goals you outlined are clear and, upon implementation, will improve the overall functionality of Pulsar.

[DISCUSS] PIP-263: Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ

Hi community I started a PIP about "Just auto-create no-partitioned DLQ And Prevent auto-create a DLQ for a DLQ". PIP link: https://github.com/apache/pulsar/issues/20033 ### Motivation Just auto-create no-partitioned DLQ/Retry Topic If enabled the config `allowAutoTopicCreation,` Pulsar wi