Re: [DISCUSS] PIP-242 Topic name restrictions

2023-05-09 Thread Yubiao Feng
Hi Mattison Now we can create a topic with special characters in its name, but Prometheus does not support it. Should we increase restrictions to avoid this scenario? See: https://github.com/apache/pulsar/pull/20230 Thanks Yubiao Feng On Sat, Feb 11, 2023 at 12:01 PM wrote: > Hi guys. > > Th

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-19 Thread Asaf Mesika
Is https://github.com/apache/pulsar/pull/19235 somehow related? On Sat, Feb 18, 2023 at 10:38 AM wrote: > Hi, All > > After discussing with Enrico and Michael offline. > I will split the discussed topic into two PIP. > > 1. Topic name restrictions > a. `-partition-` keyword. > b. enable

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-18 Thread mattisonchao
Hi, All After discussing with Enrico and Michael offline. I will split the discussed topic into two PIP. 1. Topic name restrictions a. `-partition-` keyword. b. enable topic name character pattern. 2. System topic a. System topic name pattern. b. System topic authorisation. c.

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-17 Thread Michael Marshall
I support breaking this into two PIPs. It was my fault the two PIPs were merged in the first place. I am sorry if I created any confusion. My intention was only to point out that names are a meaningful way to simplify logic, and we should reserve certain names for Pulsar's own usage with a well def

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-16 Thread Enrico Olivelli
Mattison, Il giorno gio 16 feb 2023 alle ore 00:27 ha scritto: > > > I am sorry but I am not sure that this is enough to preventreads/writes > > from unallowed clients. > IMO, We can consider the authorisation part in another PIP because We are > just focusing on adding the topic name constrain

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-16 Thread Enrico Olivelli
Il giorno gio 16 feb 2023 alle ore 14:39 Asaf Mesika ha scritto: > > On Wed, Feb 15, 2023 at 4:36 PM wrote: > > > Hi, All > > > > First of all, I want to list all of the system topics as follows. That > > Yunze has mentioned before. > > > > Namespace level: > > > > • pulsar/system > > • trans

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-16 Thread Asaf Mesika
On Wed, Feb 15, 2023 at 4:36 PM wrote: > Hi, All > > First of all, I want to list all of the system topics as follows. That > Yunze has mentioned before. > > Namespace level: > > • pulsar/system > • transaction_coordinator_assign > • __transaction_log_ > • resource-usage > • pulsar/ >

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-15 Thread mattisonchao
> I am sorry but I am not sure that this is enough to preventreads/writes from > unallowed clients. IMO, We can consider the authorisation part in another PIP because We are just focusing on adding the topic name constraint of topic creation. Maybe we can use another PIP to clearify all of syste

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-15 Thread Enrico Olivelli
Il giorno mer 15 feb 2023 alle ore 17:07 ha scritto: > > Hi Enrico > > I think it's a good question. We can introduce a new method in the > BrokerService to help brokers create the topic internally first(maybe just > metadata is enough), and then to use a pulsar client to connect to it. I am so

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-15 Thread mattisonchao
Hi Enrico I think it's a good question. We can introduce a new method in the BrokerService to help brokers create the topic internally first(maybe just metadata is enough), and then to use pulsar client connect to it. WDYT? Best, Mattison On Feb 16, 2023, 00:01 +0800, Enrico Olivelli , wrote:

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-15 Thread Enrico Olivelli
I have one question (apologies for the top posting). The Broker (and the other Pulsar components) use the regular Pulsar client to connect to "system topics" and in general they use the Pulsar wire protocol. The question is "how do you distinguish an internal component from a user component ?" Ho

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-15 Thread mattisonchao
Hi Asaf There is a link to introduce the dynamic configuration. https://pulsar.apache.org/docs/2.10.x/admin-api-brokers/#dynamic-broker-configuration Best, Mattison On Feb 14, 2023, 17:06 +0800, Asaf Mesika , wrote: > On Tue, Feb 14, 2023 at 3:46 AM wrote: > > > Hi, Asaf > > > > Welcome to join

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-15 Thread mattisonchao
Hi, All First of all, I want to list all of the system topics as follows. That Yunze has mentioned before. Namespace level: • pulsar/system • transaction_coordinator_assign • __transaction_log_ • resource-usage • pulsar/ • healthcheck Topic level: • __change_events • __transac

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-14 Thread Michael Marshall
I support this PIP, thank you for driving it forward Mattison. I support using the prefix __SYSTEM__ for system topics, and I agree that DLQ and RETRY topics do not qualify as system topics because they are produced to and consumed by regular clients. I think we might benefit from a generic defin

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-14 Thread mattisonchao
> Could you list all existing system topic names that are used so we > canunderstand the rule better? Yes, sure. I will list it later. > And I saw that this proposal only forbids the creation of thesetopics. What > about writing messages to them? I think it's better notto allow Pulsar > clients

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-14 Thread mattisonchao
>These topics all end in uppercase letters, such as `-RETRY,` `-DLQ.` Is it better to define the system topic name in uppercase( '__SYSTEM__' )? I think we don't need to mark `-DLQ` and `-RETRY` to be the keyword. because they are the same as the normal topic for the broker side. Best, Mattison

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-14 Thread Yunze Xu
Could you list all existing system topic names that are used so we can understand the rule better? Such as the "-RETRY" and "-DLQ" topics mentioned above. And I saw that this proposal only forbids the creation of these topics. What about writing messages to them? I think it's better not to allow P

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-14 Thread Yubiao Feng
Hi Qiang ### System topic name '__system__' I think this format is clearer. Now the system automatically creates topics of type retry consumer letters and dead letters. These topics all end in uppercase letters, such as `-RETRY,` `-DLQ.` Is it better to define the system topic name in uppercase( '

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-14 Thread Asaf Mesika
On Tue, Feb 14, 2023 at 3:46 AM wrote: > Hi, Asaf > > Welcome to join this discussion. > > You mean that allows the *system* to use it when it's a partitioned > topic? > Sorry, I didn't get your point. What do you mean by *system*? > This sentence was a reply to: 2. Make the `-partition-` strin

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-13 Thread mattisonchao
Hi, Asaf Welcome to join this discussion. > You mean that allows the *system* to use it when it's a partitioned topic? Sorry, I didn't get your point. What do you mean by *system*? > Why postfix of `__`?Why uppercase ?Maybe `__system__`? Yes, That is a key point that I want to discuss in this thre

Re: [DISCUSS] PIP-242 Topic name restrictions

2023-02-13 Thread Asaf Mesika
On Sat, Feb 11, 2023 at 6:01 AM wrote: > Hi guys. > > This is the refined PIP-242 discussion thread. Because we have discussed > some parts of this in another thread[0][1], we can start the standard PIP > discussion to reach a detailed consensus. > > You can check here[2] to see the PIP. > > > Be

[DISCUSS] PIP-242 Topic name restrictions

2023-02-10 Thread mattisonchao
Hi guys. This is the refined PIP-242 discussion thread. Because we have discussed some parts of this in another thread[0][1], we can start the standard PIP discussion to reach a detailed consensus. You can check here[2] to see the PIP. Best, Mattison [0] https://lists.apache.org/thread/5s4ko