Il giorno gio 16 feb 2023 alle ore 14:39 Asaf Mesika <asaf.mes...@gmail.com> ha scritto: > > On Wed, Feb 15, 2023 at 4:36 PM <mattisonc...@gmail.com> wrote: > > > Hi, All > > > > First of all, I want to list all of the system topics as follows. That > > Yunze has mentioned before. > > > > Namespace level: > > > > • pulsar/system > > • transaction_coordinator_assign > > • __transaction_log_ > > • resource-usage > > • pulsar/<host> > > • healthcheck > > > > Topic level: > > > > • __change_events > > • __transaction_buffer_snapshot_segment > > • ... > > > > We can check the details here. [0] > > > > Secondly. I think we need focus on the system topic name prefix. we have > > some options as follows: > > > > • __SYSTEM__ > > • __system__ > > > > > > Both them make sense for me. As there are already two people prefer > > `__SYSTEM`. if no other people have concern, I will chose it. > > > > I vote `__system__` since I think upper casing is not that common for > internal names (for me it's "shouting" and requires an extra key press to > write it). > Examples: > > Cassandra System tables > <https://docs.datastax.com/en/dse/5.1/cql/cql/cql_using/useQuerySystem.html> > : > * batches > > Postgres database catalog > <https://www.postgresql.org/docs/current/catalogs.html>: > * pg_database > <https://www.postgresql.org/docs/current/catalog-pg-database.html> > > OpenSearch system indices > <https://opensearch.org/docs/1.0/security-plugin/configuration/system-indices/> > : > * .opendistro-alerting-config > > > So I think we should stick to `__system__`
__system__ may work for me, but how do we deal with existing clusters ? Enrico > > > > > > Best, > > Mattison > > > > -[0] > > https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/java/org/apache/pulsar/common/naming/SystemTopicNames.java > > On Feb 15, 2023, 11:36 +0800, Michael Marshall <mmarsh...@apache.org>, > > wrote: > > > I support this PIP, thank you for driving it forward Mattison. > > > > > > I support using the prefix __SYSTEM__ for system topics, and I agree > > > that DLQ and RETRY topics do not qualify as system topics because they > > > are produced to and consumed by regular clients. > > > > > > I think we might benefit from a generic definition for a system topic. > > > I provided a loose definition for system topics in this email [0]. I > > > copy it here: > > > > > > A system topic is a topic that is completely internal to Pulsar > > > components. Internally, it is a normal topic. It requires elevated > > > permission to produce/consume when authorization is enabled, even if > > > topic level policies are not enabled. Generic calls like > > > `clearNamespaceBacklog` should not affect system topics. Deleting a > > > namespace or tenant should delete the system topics within it. > > > > > > Thanks, > > > Michael > > > > > > [0] https://lists.apache.org/thread/sr01hvqmdrnk4lxwfwzcpg7y21psj6tt > > > > > > > > > On Tue, Feb 14, 2023 at 9:55 AM <mattisonc...@gmail.com> wrote: > > > > > > > > > > Could you list all existing system topic names that are used so we > > canunderstand the rule better? > > > > Yes, sure. I will list it later. > > > > > > And I saw that this proposal only forbids the creation of > > thesetopics. What about writing messages to them? I think it's better notto > > allow Pulsar clients to write messages to them. The Pulsar clientshould > > configure something to get the permission to write messages tothem. To keep > > the compatibility, maybe we can make use of the protocolversion. > > > > Sure, I agree with your point, because the current proposal just want > > to have topic name restriction. maybe we can revise the system topic > > permission and etc in the next proposal. (relate to system topic) > > > > > > > > Best, > > > > Mattison > > > > On Feb 14, 2023, 21:22 +0800, Yunze Xu <y...@streamnative.io.invalid>, > > wrote: > > > > > > Could you list all existing system topic names that are used so we > > can > > > > > > understand the rule better? Such as the "-RETRY" and "-DLQ" topics > > > > > > mentioned above. > > > > > > > > > > > > And I saw that this proposal only forbids the creation of these > > > > > > topics. What about writing messages to them? I think it's better > > not > > > > > > to allow Pulsar clients to write messages to them. The Pulsar > > client > > > > > > should configure something to get the permission to write messages > > to > > > > > > them. To keep the compatibility, maybe we can make use of the > > protocol > > > > > > version. > > > > > > > > > > > > Thanks > > > > > > Yunze > > > > > > > > > > > > On Tue, Feb 14, 2023 at 5:38 PM Yubiao Feng > > > > > > <yubiao.f...@streamnative.io.invalid> wrote: > > > > > > > > > > > > > > > > Hi Qiang > > > > > > > > > > > > > > > > ### System topic name > > > > > > > > '__system__<name>' I think this format is clearer. > > > > > > > > Now the system automatically creates topics of type retry > > consumer letters > > > > > > > > and dead letters. > > > > > > > > These topics all end in uppercase letters, such as `-RETRY,` > > `-DLQ.` > > > > > > > > Is it better to define the system topic name in uppercase( > > > > > > > > '__SYSTEM__<name>' )? > > > > > > > > > > > > > > > > ### Another comment > > > > > > > > If you now redesign the topic name restrictions, should we > > make `-RETRY` > > > > > > > > and `-DLQ` keywords? > > > > > > > > > > > > > > > > Thanks > > > > > > > > Yubiao Feng > > > > > > > > > > > > > > > > On Tue, Feb 14, 2023 at 5:06 PM Asaf Mesika < > > asaf.mes...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > On Tue, Feb 14, 2023 at 3:46 AM < > > mattisonc...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, Asaf > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Welcome to join this discussion. > > > > > > > > > > > > > > > > > > > > You mean that allows the *system* to > > use it when it's a partitioned > > > > > > > > > > > > > > > > topic? > > > > > > > > > > > > > > > > Sorry, I didn't get your point. What do you > > mean by *system*? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This sentence was a reply to: > > > > > > > > > > > > > > > > > > > > > > > > 2. Make the `-partition-` string the keyword. That > > allows the user to use > > > > > > > > > > > > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I wanted to say that this sentence should be: > > > > > > > > > > > > Make the `-partition-` string the keyword, that allows > > the *system* to use > > > > > > > > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Why postfix of `__`?Why uppercase > > ?Maybe `__system__<name>`? > > > > > > > > > > > > > > > > Yes, That is a key point that I want to > > discuss in this > > > > > > > > > > > > > > > > thread. `__system__<name>` is good for me. > > > > > > > > > > > > > > > > > > > > Can you please elaborate what it means > > to make it dynamic exactly? > > > > > > > > > > > > > > > > Sorry, I will refine it. it means we can > > update this configuration > > > > > > > > > > > > > > > > dynamically. (using rest api or sth) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'm unfamiliar with how Pulsar supports dynamic > > configuration. I would > > > > > > > > > > > > love it if you can share a link or explain it briefly, > > thus explaining what > > > > > > > > > > > > exactly you are going to change to support dynamic > > configuration. > > > > > > > > > > > > > > > > > > > > > > > > > > > > General question: In the last thread you said > > something about > > > > > > > > > > > > > > > > configurablerules, etc? You decided not to use > > this idea? > > > > > > > > > > > > > > > > IMO, That idea is an advanced feature. we may > > need more time to discuss > > > > > > > > > > > > > > > > the details and for the topic name > > restriction, maybe we don't have > > > > > > > > > > > > strong > > > > > > > > > > > > > > > > reason to use that. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We can introduce this advanced feature when we > > have a need for it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I agree. Future PIP and discussion. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > Mattison > > > > > > > > > > > > > > > > On Feb 13, 2023, 22:21 +0800, Asaf Mesika < > > asaf.mes...@gmail.com>, > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > You mean that allows the *system* to > > use it when it's a partitioned > > > > > > > > > > > > > > > > topic? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
