I support this PIP, thank you for driving it forward Mattison. I support using the prefix __SYSTEM__ for system topics, and I agree that DLQ and RETRY topics do not qualify as system topics because they are produced to and consumed by regular clients.
I think we might benefit from a generic definition for a system topic. I provided a loose definition for system topics in this email [0]. I copy it here: A system topic is a topic that is completely internal to Pulsar components. Internally, it is a normal topic. It requires elevated permission to produce/consume when authorization is enabled, even if topic level policies are not enabled. Generic calls like `clearNamespaceBacklog` should not affect system topics. Deleting a namespace or tenant should delete the system topics within it. Thanks, Michael [0] https://lists.apache.org/thread/sr01hvqmdrnk4lxwfwzcpg7y21psj6tt On Tue, Feb 14, 2023 at 9:55 AM <mattisonc...@gmail.com> wrote: > > > Could you list all existing system topic names that are used so we > > canunderstand the rule better? > Yes, sure. I will list it later. > > And I saw that this proposal only forbids the creation of thesetopics. What > > about writing messages to them? I think it's better notto allow Pulsar > > clients to write messages to them. The Pulsar clientshould configure > > something to get the permission to write messages tothem. To keep the > > compatibility, maybe we can make use of the protocolversion. > Sure, I agree with your point, because the current proposal just want to > have topic name restriction. maybe we can revise the system topic permission > and etc in the next proposal. (relate to system topic) > > Best, > Mattison > On Feb 14, 2023, 21:22 +0800, Yunze Xu <y...@streamnative.io.invalid>, wrote: > > Could you list all existing system topic names that are used so we can > > understand the rule better? Such as the "-RETRY" and "-DLQ" topics > > mentioned above. > > > > And I saw that this proposal only forbids the creation of these > > topics. What about writing messages to them? I think it's better not > > to allow Pulsar clients to write messages to them. The Pulsar client > > should configure something to get the permission to write messages to > > them. To keep the compatibility, maybe we can make use of the protocol > > version. > > > > Thanks > > Yunze > > > > On Tue, Feb 14, 2023 at 5:38 PM Yubiao Feng > > <yubiao.f...@streamnative.io.invalid> wrote: > > > > > > Hi Qiang > > > > > > ### System topic name > > > '__system__<name>' I think this format is clearer. > > > Now the system automatically creates topics of type retry consumer letters > > > and dead letters. > > > These topics all end in uppercase letters, such as `-RETRY,` `-DLQ.` > > > Is it better to define the system topic name in uppercase( > > > '__SYSTEM__<name>' )? > > > > > > ### Another comment > > > If you now redesign the topic name restrictions, should we make `-RETRY` > > > and `-DLQ` keywords? > > > > > > Thanks > > > Yubiao Feng > > > > > > On Tue, Feb 14, 2023 at 5:06 PM Asaf Mesika <asaf.mes...@gmail.com> wrote: > > > > > > > > On Tue, Feb 14, 2023 at 3:46 AM <mattisonc...@gmail.com> wrote: > > > > > > > > > > > > Hi, Asaf > > > > > > > > > > > > > > Welcome to join this discussion. > > > > > > > > > You mean that allows the *system* to use it when it's a > > > > > > > > > partitioned > > > > > > > topic? > > > > > > > Sorry, I didn't get your point. What do you mean by *system*? > > > > > > > > > > > > > > > > > This sentence was a reply to: > > > > > > > > > > 2. Make the `-partition-` string the keyword. That allows the user to > > > > > use > > > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > I wanted to say that this sentence should be: > > > > > Make the `-partition-` string the keyword, that allows the *system* > > > > > to use > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > > > > > > > > Why postfix of `__`?Why uppercase ?Maybe `__system__<name>`? > > > > > > > Yes, That is a key point that I want to discuss in this > > > > > > > thread. `__system__<name>` is good for me. > > > > > > > > > Can you please elaborate what it means to make it dynamic > > > > > > > > > exactly? > > > > > > > Sorry, I will refine it. it means we can update this configuration > > > > > > > dynamically. (using rest api or sth) > > > > > > > > > > > > > > > > > I'm unfamiliar with how Pulsar supports dynamic configuration. I would > > > > > love it if you can share a link or explain it briefly, thus > > > > > explaining what > > > > > exactly you are going to change to support dynamic configuration. > > > > > > > > > > > > General question: In the last thread you said something about > > > > > > > configurablerules, etc? You decided not to use this idea? > > > > > > > IMO, That idea is an advanced feature. we may need more time to > > > > > > > discuss > > > > > > > the details and for the topic name restriction, maybe we don't > > > > > > > have > > > > > strong > > > > > > > reason to use that. > > > > > > > > > > > > > > We can introduce this advanced feature when we have a need for it. > > > > > > > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > I agree. Future PIP and discussion. > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > Mattison > > > > > > > On Feb 13, 2023, 22:21 +0800, Asaf Mesika <asaf.mes...@gmail.com>, > > > > > wrote: > > > > > > > > > You mean that allows the *system* to use it when it's a > > > > > > > > > partitioned > > > > > > > topic? > > > > > > > > > > > >
