>
> Additional:
> Since the disallowed topic names configuration needs more discussion about
> name pattern, type etc. I think we can wait for the demand to consider it.
>
Sounds good. We can discuss this in the future with a new proposal if there
has someone who needs it.
Yong
On Sat, 11 Feb 2023 at 11:23, <mattisonc...@gmail.com> wrote:
> Hi guys.
>
> Thanks for your discussion in this thread. Since we have reached the
> discussion deadline.
> > I will keep this discussion for a week. If there are no more new types
> of restrictions, I will refine the previous PIP-242[0] to explain more
> details.
>
> I would like to refine the PIP-242, which includes four parts.
>
> • Introduce `enableStrictTopicName.` configuration.
> • Add NamedEntity validation for the topic name.
> • Add `-partition-` keyword.
> • Introduce topic name's structure.（we can keep the original topic names
> and introduce new system topic name structure `__SYS__<name>__`. ）
>
>
> Additional:
> Since the disallowed topic names configuration needs more discussion about
> name pattern, type etc. I think we can wait for the demand to consider it.
>
> Best,
> Mattison
>
> On Feb 6, 2023, 23:10 +0800, mattisonc...@gmail.com, wrote:
> > Hi, Asaf
> > > I don't understand the idea suggested of making the validation
> rulesconfigurable.If understand correctly:* "-partition" is not something
> you want configurable - it should always bevalidated* System topics - once
> we agree on a naming convention going forward, itshould always be validated.
> > We need to ensure compatibility so that users can choose. as Michael
> mentioned. the configurable restriction can easily help users to avoid
> breaking.
> > > In the context of PIP 242, we're introducing a config to
> optionallyenforce strict topic names. As such, we could rely on the config
> toeither use the "cheap" check to see if the topic starts with __ or
> wecould use the more expensive check to determine if the topic name isone
> of many possible system topic names. Because we want to maintainbackwards
> compatibility, we cannot completely get rid of the oldlogic.
> > Best,
> > Mattison
> > On Feb 5, 2023, 19:24 +0800, Asaf Mesika <asaf.mes...@gmail.com>, wrote:
> > > Thanks Mattison.
> > >
> > > I don't understand the idea suggested of making the validation rules
> > > configurable.
> > > If understand correctly:
> > > * "-partition" is not something you want configurable - it should
> always be
> > > validated
> > > * System topics - once we agree on a naming convention going forward,
> it
> > > should always be validated.
> > >
> > > I'm ok with adding configuration for the user so they can validate
> rules of
> > > their own, maybe even per tenant.
> > >
> > >
> > > On Fri, Feb 3, 2023 at 11:44 AM <mattisonc...@gmail.com> wrote:
> > >
> > > > Hi, Asaf
> > > >
> > > > We are using the regular expression to check the name.
> > > > "^[-=:.\\w]*$"
> > > > The \w means [A-Za-z0-9_] , which includes underscores.
> > > >
> > > > Best,
> > > > Mattison
> > > > On Feb 2, 2023, 23:01 +0800, Asaf Mesika <asaf.mes...@gmail.com>,
> wrote:
> > > > > > NamedEntity is not allowing underscores - does it make sense?
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Feb 2, 2023 at 8:35 AM Michael Marshall <
> mmarsh...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > > Thanks for starting this thread, Mattison.
> > > > > > > >
> > > > > > > > > > > > The topic name character validation is already done
> by
> > > > > > > > > > > > `NamedEntity#checkName`.
> > > > > > > >
> > > > > > > > Based on my reading of the code, only the tenant and the
> namespace
> > > > > > > > names are validated using that method. There is a call [0]
> that looks
> > > > > > > > like it validates topic names, but that method is only
> called by
> > > > > > > > tests.
> > > > > > > >
> > > > > > > > > > > > But I have a concern that whether we should
> > > > > > > > > > > > treat all topics that start with the long underscore
> ("__") as
> > > > system
> > > > > > > > > > > > topics?
> > > > > > > >
> > > > > > > > This is a reasonable concern, and my primary motivation in
> proposing
> > > > > > > > this change is to make it easier for the broker to handle
> system
> > > > > > > > topics, which often get unique treatment.
> > > > > > > >
> > > > > > > > I wrote on this topic in several replies on this thread from
> a year ago
> > > > > > > > [1].
> > > > > > > >
> > > > > > > > In the context of PIP 242, we're introducing a config to
> optionally
> > > > > > > > enforce strict topic names. As such, we could rely on the
> config to
> > > > > > > > either use the "cheap" check to see if the topic starts with
> __ or we
> > > > > > > > could use the more expensive check to determine if the topic
> name is
> > > > > > > > one of many possible system topic names. Because we want to
> maintain
> > > > > > > > backwards compatibility, we cannot completely get rid of the
> old
> > > > > > > > logic. I like self describing names because they are elegant
> and
> > > > > > > > efficient.
> > > > > > > >
> > > > > > > > > > > > If yes, how would you like to allow users to access
> the system
> > > > topics?
> > > > > > > >
> > > > > > > > I proposed some ideas at the end of that thread [1]. We
> should have a
> > > > > > > > clear definition of system topics and how they are or are
> not accessed
> > > > by
> > > > > > > > users. Ultimately, we continue to create new system topics
> without
> > > > > > > > reserving a designated naming structure and without defining
> how these
> > > > > > > > topics ought to be interacted with, as Yunze points out.
> Note that any
> > > > > > > > system topic we introduce could conflict with existing user
> topics, so
> > > > > > > > proactively reserving a set of names makes it easier for
> forwards
> > > > > > > > compatibility.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Michael
> > > > > > > >
> > > > > > > > [0]
> > > > > > > >
> > > >
> https://github.com/apache/pulsar/blob/b880b1d240ade864181935aa360bfca03a5aa67f/pulsar-common/src/main/java/org/apache/pulsar/common/naming/NamespaceName.java#L159
> > > > > > > > [1]
> https://lists.apache.org/thread/pj4n4wzm3do8nkc52l7g7obh0sktzm17
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Feb 1, 2023 at 11:28 PM r...@apache.org <
> > > > ranxiaolong...@gmail.com>
> > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Hi Mattison:
> > > > > > > > > > > >
> > > > > > > > > > > > Agree with Yong's idea. We can expose `disallowed
> topic` as a
> > > > > > > > configuration
> > > > > > > > > > > > to the user side, and a more flexible way is to
> expose it as a
> > > > > > > > > > > > namespace-level policy. This can ensure that there
> is no need to do
> > > > > > > > special
> > > > > > > > > > > > processing on customized keywords in the future, and
> the expected
> > > > effect
> > > > > > > > > > > > can be achieved by modifying the configuration.
> > > > > > > > > > > >
> > > > > > > > > > > > Think Yunze's concerns are justified for the system
> topic. Is it
> > > > okay if
> > > > > > > > we
> > > > > > > > > > > > use hard code? Because the identification of any
> keyword is likely
> > > > to be
> > > > > > > > > > > > hit by the user. The hard code method is used to
> filter out system
> > > > topics
> > > > > > > > > > > > and not allow users to operate during delete and
> create operations.
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > Thanks
> > > > > > > > > > > > Xiaolong Ran
> > > > > > > > > > > >
> > > > > > > > > > > > Dave Fisher <wave4d...@comcast.net> 于2023年2月2日周四
> 11:26写道：
> > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Sent from my iPhone
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > On Feb 1, 2023, at 6:52 PM, Yong
> Zhang <
> > > > zhangyong1025...@gmail.com>
> > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Mattison,
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > I agree with you about restricting
> the topic name.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > How about using a blacklist way to
> restrict it?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > If we do then please call it by another name
> like “disallowed”.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > We can have a blacklist on the topic
> name restriction and
> > > > make it
> > > > > > > > > > > > > > > > > > > > configurable. Add the keywords you
> mentioned in the default
> > > > > > > > > > > > > > > > configuration.
> > > > > > > > > > > > > > > > > > > > That would have a more general way
> to block a topic name
> > > > creation.
> > > > > > > > > > > > > > > > > > > > If we have more restrictions on the
> topic name in the
> > > > future, this
> > > > > > > > way
> > > > > > > > > > > > > > > > > > > > can make it easy to fit them without
> changing any code.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Is there anyone asking for this feature?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Best,
> > > > > > > > > > > > > > > > Dave
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > > > > > > > Yong
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > > >> On Thu, 2 Feb 2023 at 07:33, <
> mattisonc...@gmail.com>
> > > > wrote:
> > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > >> Hi, All
> > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > >> In the current
> implementation, pulsar didn't support
> > > > topic name
> > > > > > > > > > > > > > > > > > > > > > >> restriction. It's a good
> chance to discuss it.
> > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > >> I think this discussion aims
> to identify what types of
> > > > topic names
> > > > > > > > we
> > > > > > > > > > > > > > > > all
> > > > > > > > > > > > > > > > > > > > > > >> need to restrict.
> > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > >> I know three topic names that
> need to be restricted at
> > > > the moment.
> > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > >> 1. The `-partition-` keyword.
> > > > > > > > > > > > > > > > > > > > > > >> 2. Topic name characters
> validation.
> > > > > > > > > > > > > > > > > > > > > > >> 3. System topic prefix `__`.
> > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > >> Please feel free to leave
> your comments.
> > > > > > > > > > > > > > > > > > > > > > >> I will keep this discussion
> for a week. If there are no
> > > > more new
> > > > > > > > types
> > > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > > > >> restrictions, I will refine
> the previous PIP-242[0] to
> > > > explain more
> > > > > > > > > > > > > > > > details.
> > > > > > > > > > > > > > > > > > > > > > > > > >>> If we have other
> restrictions behind this
> > > > discussion. We can draft
> > > > > > > > a
> > > > > > > > > > > > > > > > new
> > > > > > > > > > > > > > > > > > > > > > >> PIP to add it directly.
> > > > > > > > > > > > > > > > > > > > > > >> Thanks to Michael's
> opinion[1], we can expand the
> > > > PIP-242 scopes to
> > > > > > > > help
> > > > > > > > > > > > > > > > > > > > > > >> pulsar have a good topic name
> restriction.
> > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > >> Best,
> > > > > > > > > > > > > > > > > > > > > > >> Mattison
> > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > > > >> [0]
> https://github.com/apache/pulsar/issues/19239
> > > > > > > > > > > > > > > > > > > > > > >> [1]
> > > > > > > >
> https://lists.apache.org/thread/dd1kxhodjvovtb8yyojkk209st4o0ft2
> > > > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > >
> > > >
>
