Re: [DISCUSS][PIP-226] Add JWKS support for AuthenticationProviderToken
Hi Michael, You are right, I already leave https://github.com/apache/pulsar/issues/8152. Thanks, Zixuan Michael Marshall 于2023年2月15日周三 13:27写道： > I added a review to https://github.com/apache/pulsar/pull/18336. > > I do not think it will be a problem to accept this PIP and to add OIDC > support. However, I think we should leave > https://github.com/apache/pulsar/issues/8152 open because this PIP > will not address the primary features requested in that issue. > > Thanks, > Michael > > On Mon, Feb 13, 2023 at 9:14 PM Zixuan Liu wrote: > > > > Ping Michael. > > > > > > Zixuan Liu 于2022年12月26日周一 11:07写道： > > > > > Hi Michael, > > > > > > Thank you for your suggestion! > > > > > > OIDC is very popular everywhere and makes great sense for Apache > Pulsar. > > > Your OIDC plugin is excellent, it follows the OIDC standard and > includes my > > > PIP idea. I'm looking forward to your OIDC plugin. > > > > > > However, some users didn't use the OIDC service, and it may be too > > > complicated to use your OIDC plugin, so I want to move forward with my > PIP. > > > > > > Thanks, > > > Zixuan > > > > > > Michael Marshall 于2022年12月24日周六 05:34写道： > > > > > >> I support adding JWKS retrieval to our Token Auth Provider, and thank > > >> you for your many security related contributions, Zixuan. > > >> > > >> I see your PIP implementation here [0]. I would like to discuss a > > >> competing implementation of a similar feature before we move forward > > >> with this PIP. I wrote an AuthenticationProviderPlugin to add OpenID > > >> Connect support here [1], and I would like to discuss contributing it > > >> to Apache Pulsar. My plugin supports retrieving JWKS from an identity > > >> provider, as your PIP proposes. > > >> > > >> In addition to the proposed additions in this PIP, my OIDC plugin: > > >> > > >> * supports multiple token issuers, known as trusted issuers in the > plugin > > >> > > >> * retrieves the JWKS uri for each issuer from the token issuer's > > >> /.well-known/openid-configuration endpoint > > >> > > >> * retrieves the JKWS when a client attempts to connect using a token > > >> issued by one of the trusted issuers > > >> > > >> * refreshes the JWKS after a configured amount of time, which allows > > >> for seamless key rotation without needing to restart the > > >> proxy/broker/function worker. (Restarts are still needed to mitigate > > >> problems like leaked private keys.) > > >> > > >> I think these extra features would be very valuable in Apache Pulsar. > > >> I am not sure how to proceed in this case where there are two > > >> implementations, but I think that if we add an OIDC auth provider or > > >> merge my OIDC plugin into the TokenAuthProvider, we might not need > > >> this PIP in its current form. > > >> > > >> What is your perspective on my alternative solution? > > >> > > >> On a practical licensing note, contributing the OIDC plugin to Apache > > >> Pulsar should not face any issues because Lari Hotari and I are the > > >> sole contributors, and I have permission from Lari and DataStax to > > >> contribute it. It has Apache License 2.0. > > >> > > >> Thanks, > > >> Michael > > >> > > >> [0] https://github.com/apache/pulsar/pull/18336 > > >> [1] https://github.com/datastax/pulsar-openid-connect-plugin > > >> > > >> On Wed, Dec 7, 2022 at 4:59 AM Zixuan Liu wrote: > > >> > > > >> > Hi all, > > >> > > > >> > I made a PIP to discuss: > https://github.com/apache/pulsar/issues/18798. > > >> > > > >> > Thanks, > > >> > Zixuan > > >> > > > >
Re: [VOTE][PIP-226] Add JWKS support for AuthenticationProviderToken
Closed by 3 (binding), and 1(non-binding). Thanks, Zixuan Michael Marshall 于2023年2月15日周三 13:31写道： > +1 (binding) > > I have some comments on the PR that I'd like to discuss before it is > merged, but I consider those minor implementation details. > > Thanks, > Michael > > On Tue, Feb 14, 2023 at 9:02 PM PengHui Li > wrote: > > > > +1 > > > > Penghui > > > > > On Feb 15, 2023, at 10:16, guo jiwei wrote: > > > > > > +1 (binding) > > > > > > > > > Regards > > > Jiwei Guo (Tboy) > > > > > > On Wed, Feb 15, 2023 at 10:11 AM wrote: > > >> > > >> +1, (non-binding) > > >> > > >> Best, > > >> Mattison > > >> On Dec 22, 2022, 11:24 +0800, Zixuan Liu , wrote: > > >>> Hi all, > > >>> > > >>> Voting for https://github.com/apache/pulsar/issues/18798. > > >>> > > >>> Thanks, > > >>> Zixuan > > >
Re: [DISCUSS] PIP-242 Topic name restrictions
Hi, All First of all, I want to list all of the system topics as follows. That Yunze has mentioned before. Namespace level: • pulsar/system • transaction_coordinator_assign • __transaction_log_ • resource-usage • pulsar/ • healthcheck Topic level: • __change_events • __transaction_buffer_snapshot_segment • ... We can check the details here. [0] Secondly. I think we need focus on the system topic name prefix. we have some options as follows: • __SYSTEM__ • __system__ Both them make sense for me. As there are already two people prefer `__SYSTEM`. if no other people have concern, I will chose it. Best, Mattison -[0] https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/java/org/apache/pulsar/common/naming/SystemTopicNames.java On Feb 15, 2023, 11:36 +0800, Michael Marshall , wrote: > I support this PIP, thank you for driving it forward Mattison. > > I support using the prefix __SYSTEM__ for system topics, and I agree > that DLQ and RETRY topics do not qualify as system topics because they > are produced to and consumed by regular clients. > > I think we might benefit from a generic definition for a system topic. > I provided a loose definition for system topics in this email [0]. I > copy it here: > > A system topic is a topic that is completely internal to Pulsar > components. Internally, it is a normal topic. It requires elevated > permission to produce/consume when authorization is enabled, even if > topic level policies are not enabled. Generic calls like > `clearNamespaceBacklog` should not affect system topics. Deleting a > namespace or tenant should delete the system topics within it. > > Thanks, > Michael > > [0] https://lists.apache.org/thread/sr01hvqmdrnk4lxwfwzcpg7y21psj6tt > > > On Tue, Feb 14, 2023 at 9:55 AM wrote: > > > > > > Could you list all existing system topic names that are used so we > > > > canunderstand the rule better? > > Yes, sure. I will list it later. > > > > And I saw that this proposal only forbids the creation of thesetopics. > > > > What about writing messages to them? I think it's better notto allow > > > > Pulsar clients to write messages to them. The Pulsar clientshould > > > > configure something to get the permission to write messages tothem. To > > > > keep the compatibility, maybe we can make use of the protocolversion. > > Sure, I agree with your point, because the current proposal just want to > > have topic name restriction. maybe we can revise the system topic > > permission and etc in the next proposal. (relate to system topic) > > > > Best, > > Mattison > > On Feb 14, 2023, 21:22 +0800, Yunze Xu , > > wrote: > > > > Could you list all existing system topic names that are used so we can > > > > understand the rule better? Such as the "-RETRY" and "-DLQ" topics > > > > mentioned above. > > > > > > > > And I saw that this proposal only forbids the creation of these > > > > topics. What about writing messages to them? I think it's better not > > > > to allow Pulsar clients to write messages to them. The Pulsar client > > > > should configure something to get the permission to write messages to > > > > them. To keep the compatibility, maybe we can make use of the protocol > > > > version. > > > > > > > > Thanks > > > > Yunze > > > > > > > > On Tue, Feb 14, 2023 at 5:38 PM Yubiao Feng > > > > wrote: > > > > > > > > > > > > Hi Qiang > > > > > > > > > > > > ### System topic name > > > > > > '__system__' I think this format is clearer. > > > > > > Now the system automatically creates topics of type retry consumer > > > > > > letters > > > > > > and dead letters. > > > > > > These topics all end in uppercase letters, such as `-RETRY,` `-DLQ.` > > > > > > Is it better to define the system topic name in uppercase( > > > > > > '__SYSTEM__' )? > > > > > > > > > > > > ### Another comment > > > > > > If you now redesign the topic name restrictions, should we make > > > > > > `-RETRY` > > > > > > and `-DLQ` keywords? > > > > > > > > > > > > Thanks > > > > > > Yubiao Feng > > > > > > > > > > > > On Tue, Feb 14, 2023 at 5:06 PM Asaf Mesika > > > > > > wrote: > > > > > > > > > > > > > > > > On Tue, Feb 14, 2023 at 3:46 AM > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > Hi, Asaf > > > > > > > > > > > > > > > > > > > > > > > > > > > > Welcome to join this discussion. > > > > > > > > > > > > > > > > > > You mean that allows the *system* to use it > > > > > > > > > > > > > > > > > > when it's a partitioned > > > > > > > > > > > > > > topic? > > > > > > > > > > > > > > Sorry, I didn't get your point. What do you mean by > > > > > > > > > > > > > > *system*? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This sentence was a reply to: > > > > > > > > > > > > > > > > > > > > 2. Make the `-partition-` string the keyword. That allows > > > > > > > > > > the user to use > > > > > > > > > > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [DISCUSS] PIP-242 Topic name restrictions
Hi Asaf There is a link to introduce the dynamic configuration. https://pulsar.apache.org/docs/2.10.x/admin-api-brokers/#dynamic-broker-configuration Best, Mattison On Feb 14, 2023, 17:06 +0800, Asaf Mesika , wrote: > On Tue, Feb 14, 2023 at 3:46 AM wrote: > > > Hi, Asaf > > > > Welcome to join this discussion. > > > > You mean that allows the *system* to use it when it's a partitioned > > topic? > > Sorry, I didn't get your point. What do you mean by *system*? > > > > This sentence was a reply to: > > 2. Make the `-partition-` string the keyword. That allows the user to use > > it when it's a partitioned topic. > > > > I wanted to say that this sentence should be: > Make the `-partition-` string the keyword, that allows the *system* to use > it when it's a partitioned topic. > > > > > > > Why postfix of `__`?Why uppercase ?Maybe `__system__`? > > Yes, That is a key point that I want to discuss in this > > thread. `__system__` is good for me. > > > > Can you please elaborate what it means to make it dynamic exactly? > > Sorry, I will refine it. it means we can update this configuration > > dynamically. (using rest api or sth) > > > > I'm unfamiliar with how Pulsar supports dynamic configuration. I would > love it if you can share a link or explain it briefly, thus explaining what > exactly you are going to change to support dynamic configuration. > > > General question: In the last thread you said something about > > configurablerules, etc? You decided not to use this idea? > > IMO, That idea is an advanced feature. we may need more time to discuss > > the details and for the topic name restriction, maybe we don't have strong > > reason to use that. > > > > We can introduce this advanced feature when we have a need for it. > > > > WDYT? > > > > I agree. Future PIP and discussion. > > > > > > Best, > > Mattison > > On Feb 13, 2023, 22:21 +0800, Asaf Mesika , wrote: > > > > You mean that allows the *system* to use it when it's a partitioned > > topic? > >
Re: [DISCUSS] PIP-242 Topic name restrictions
I have one question (apologies for the top posting). The Broker (and the other Pulsar components) use the regular Pulsar client to connect to "system topics" and in general they use the Pulsar wire protocol. The question is "how do you distinguish an internal component from a user component ?" How can you say that the broker is able to connect to a system topic and any other client cannot do it ? Enrico Il giorno mer 15 feb 2023 alle ore 15:38 ha scritto: > > Hi Asaf > > There is a link to introduce the dynamic configuration. > https://pulsar.apache.org/docs/2.10.x/admin-api-brokers/#dynamic-broker-configuration > > Best, > Mattison > On Feb 14, 2023, 17:06 +0800, Asaf Mesika , wrote: > > On Tue, Feb 14, 2023 at 3:46 AM wrote: > > > > > Hi, Asaf > > > > > > Welcome to join this discussion. > > > > > You mean that allows the *system* to use it when it's a partitioned > > > topic? > > > Sorry, I didn't get your point. What do you mean by *system*? > > > > > > > This sentence was a reply to: > > > > 2. Make the `-partition-` string the keyword. That allows the user to use > > > it when it's a partitioned topic. > > > > > > > I wanted to say that this sentence should be: > > Make the `-partition-` string the keyword, that allows the *system* to use > > it when it's a partitioned topic. > > > > > > > > > > > Why postfix of `__`?Why uppercase ?Maybe `__system__`? > > > Yes, That is a key point that I want to discuss in this > > > thread. `__system__` is good for me. > > > > > Can you please elaborate what it means to make it dynamic exactly? > > > Sorry, I will refine it. it means we can update this configuration > > > dynamically. (using rest api or sth) > > > > > > > I'm unfamiliar with how Pulsar supports dynamic configuration. I would > > love it if you can share a link or explain it briefly, thus explaining what > > exactly you are going to change to support dynamic configuration. > > > > > General question: In the last thread you said something about > > > configurablerules, etc? You decided not to use this idea? > > > IMO, That idea is an advanced feature. we may need more time to discuss > > > the details and for the topic name restriction, maybe we don't have strong > > > reason to use that. > > > > > > We can introduce this advanced feature when we have a need for it. > > > > > > WDYT? > > > > > > > I agree. Future PIP and discussion. > > > > > > > > > > Best, > > > Mattison > > > On Feb 13, 2023, 22:21 +0800, Asaf Mesika , wrote: > > > > > You mean that allows the *system* to use it when it's a partitioned > > > topic? > > >
Re: [DISCUSS] PIP-242 Topic name restrictions
Hi Enrico I think it's a good question. We can introduce a new method in the BrokerService to help brokers create the topic internally first(maybe just metadata is enough), and then to use pulsar client connect to it. WDYT? Best, Mattison On Feb 16, 2023, 00:01 +0800, Enrico Olivelli , wrote: > I have one question (apologies for the top posting). > > The Broker (and the other Pulsar components) use the regular Pulsar > client to connect to "system topics" > and in general they use the Pulsar wire protocol. > > The question is "how do you distinguish an internal component from a > user component ?" > How can you say that the broker is able to connect to a system topic > and any other client cannot do it ? > > Enrico > > Il giorno mer 15 feb 2023 alle ore 15:38 ha scritto: > > > > Hi Asaf > > > > There is a link to introduce the dynamic configuration. > > https://pulsar.apache.org/docs/2.10.x/admin-api-brokers/#dynamic-broker-configuration > > > > Best, > > Mattison > > On Feb 14, 2023, 17:06 +0800, Asaf Mesika , wrote: > > > > On Tue, Feb 14, 2023 at 3:46 AM wrote: > > > > > > > > > > Hi, Asaf > > > > > > > > > > > > Welcome to join this discussion. > > > > > > > > > > You mean that allows the *system* to use it when it's a > > > > > > > > > > partitioned > > > > > > topic? > > > > > > Sorry, I didn't get your point. What do you mean by *system*? > > > > > > > > > > > > > > This sentence was a reply to: > > > > > > > > 2. Make the `-partition-` string the keyword. That allows the user to > > > > use > > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > I wanted to say that this sentence should be: > > > > Make the `-partition-` string the keyword, that allows the *system* to > > > > use > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > > > > > > Why postfix of `__`?Why uppercase ?Maybe `__system__`? > > > > > > Yes, That is a key point that I want to discuss in this > > > > > > thread. `__system__` is good for me. > > > > > > > > > > Can you please elaborate what it means to make it dynamic > > > > > > > > > > exactly? > > > > > > Sorry, I will refine it. it means we can update this configuration > > > > > > dynamically. (using rest api or sth) > > > > > > > > > > > > > > I'm unfamiliar with how Pulsar supports dynamic configuration. I would > > > > love it if you can share a link or explain it briefly, thus explaining > > > > what > > > > exactly you are going to change to support dynamic configuration. > > > > > > > > > > General question: In the last thread you said something about > > > > > > configurablerules, etc? You decided not to use this idea? > > > > > > IMO, That idea is an advanced feature. we may need more time to > > > > > > discuss > > > > > > the details and for the topic name restriction, maybe we don't have > > > > > > strong > > > > > > reason to use that. > > > > > > > > > > > > We can introduce this advanced feature when we have a need for it. > > > > > > > > > > > > WDYT? > > > > > > > > > > > > > > I agree. Future PIP and discussion. > > > > > > > > > > > > > > > > > > > > Best, > > > > > > Mattison > > > > > > On Feb 13, 2023, 22:21 +0800, Asaf Mesika , > > > > > > wrote: > > > > > > > > > > You mean that allows the *system* to use it when it's a > > > > > > > > > > partitioned > > > > > > topic? > > > > > >
Re: [DISCUSS] PIP-242 Topic name restrictions
Il giorno mer 15 feb 2023 alle ore 17:07 ha scritto: > > Hi Enrico > > I think it's a good question. We can introduce a new method in the > BrokerService to help brokers create the topic internally first(maybe just > metadata is enough), and then to use a pulsar client to connect to it. I am sorry but I am not sure that this is enough to prevent reads/writes from unallowed clients. We should elaborate more on this topic on the PIP Enrico > > WDYT? > > > Best, > Mattison > On Feb 16, 2023, 00:01 +0800, Enrico Olivelli , wrote: > > I have one question (apologies for the top posting). > > > > The Broker (and the other Pulsar components) use the regular Pulsar > > client to connect to "system topics" > > and in general they use the Pulsar wire protocol. > > > > The question is "how do you distinguish an internal component from a > > user component ?" > > How can you say that the broker is able to connect to a system topic > > and any other client cannot do it ? > > > > Enrico > > > > Il giorno mer 15 feb 2023 alle ore 15:38 ha > > scritto: > > > > > > Hi Asaf > > > > > > There is a link to introduce the dynamic configuration. > > > https://pulsar.apache.org/docs/2.10.x/admin-api-brokers/#dynamic-broker-configuration > > > > > > Best, > > > Mattison > > > On Feb 14, 2023, 17:06 +0800, Asaf Mesika , wrote: > > > > > On Tue, Feb 14, 2023 at 3:46 AM wrote: > > > > > > > > > > > > Hi, Asaf > > > > > > > > > > > > > > Welcome to join this discussion. > > > > > > > > > > > You mean that allows the *system* to use it when it's a > > > > > > > > > > > partitioned > > > > > > > topic? > > > > > > > Sorry, I didn't get your point. What do you mean by *system*? > > > > > > > > > > > > > > > > > This sentence was a reply to: > > > > > > > > > > 2. Make the `-partition-` string the keyword. That allows the user to > > > > > use > > > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > I wanted to say that this sentence should be: > > > > > Make the `-partition-` string the keyword, that allows the *system* > > > > > to use > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > > > > > > > > > > Why postfix of `__`?Why uppercase ?Maybe > > > > > > > > > > > `__system__`? > > > > > > > Yes, That is a key point that I want to discuss in this > > > > > > > thread. `__system__` is good for me. > > > > > > > > > > > Can you please elaborate what it means to make it dynamic > > > > > > > > > > > exactly? > > > > > > > Sorry, I will refine it. it means we can update this configuration > > > > > > > dynamically. (using rest api or sth) > > > > > > > > > > > > > > > > > I'm unfamiliar with how Pulsar supports dynamic configuration. I would > > > > > love it if you can share a link or explain it briefly, thus > > > > > explaining what > > > > > exactly you are going to change to support dynamic configuration. > > > > > > > > > > > > General question: In the last thread you said something about > > > > > > > configurablerules, etc? You decided not to use this idea? > > > > > > > IMO, That idea is an advanced feature. we may need more time to > > > > > > > discuss > > > > > > > the details and for the topic name restriction, maybe we don't > > > > > > > have strong > > > > > > > reason to use that. > > > > > > > > > > > > > > We can introduce this advanced feature when we have a need for it. > > > > > > > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > I agree. Future PIP and discussion. > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > Mattison > > > > > > > On Feb 13, 2023, 22:21 +0800, Asaf Mesika > > > > > > > , wrote: > > > > > > > > > > > You mean that allows the *system* to use it when it's a > > > > > > > > > > > partitioned > > > > > > > topic? > > > > > > >
Force redirect questions from Slack to GitHub Discussions or StackOverflow?
Hi everyone! Enrico Olivelli asked me to repost it to the mailing list. --- Me I’m very worried that good answers from David Kjerrumgaard and others won’t be googleable because it’s in Slack. To make any search you need to pay a monthly subscription. In my opinion, it would be wiser to make StackOverflow, Discuss, or GitHub discussions a place for questions. And strictly redirect people who ask any question in Slack to the right place. In GitHub discussions, you also can manage categories, as well as you can manage channels in Slack. Subscription to a specific category is coming soon. https://github.com/community/community/discussions/3951#discussioncomment-3879939 The most important that it’s searchable. Examples: https://github.com/community/community/discussions https://github.com/apache/superset/discussions --- Later me to David Kjerrumgaard What do you think about the following workflow: 1. The user asks a new question in Slack. 2. You click the three dots button on his message -> “Convert to a GitHub discussion”. 3. The bot creates a new GitHub discussion in the “Q&A” category with the original message’s content. 4. The bot converts the original message to a thread and posts a message like: > In order to keep history and make your question searchable by other Apache > Pulsar users, it has been converted to a GitHub discussion. > Further conversation is available by this link: > https://github.com/apache/pulsar/discussions/18766. 5. You answer the question on GitHub. How it may work: https://api.slack.com/best-practices/blueprints/actions-and-dialogs I can try to implement it makes any sense for you. (edited) Full original thread: https://apache-pulsar.slack.com/archives/C5Z4T36F7/p1676449603034309
Re: [DISCUSS] PIP-242 Topic name restrictions
> I am sorry but I am not sure that this is enough to preventreads/writes from > unallowed clients. IMO, We can consider the authorisation part in another PIP because We are just focusing on adding the topic name constraint of topic creation. Maybe we can use another PIP to clearify all of system topic's behaviour, like authorisation something. e.g. we just allow superusers to read/write the data to that system topic. > We should elaborate more on this topic on the PIP I will add the internal system topic creation logic in the PIP. Best, Mattison On Feb 16, 2023, 00:41 +0800, Enrico Olivelli , wrote: > Il giorno mer 15 feb 2023 alle ore 17:07 ha scritto: > > > > Hi Enrico > > > > I think it's a good question. We can introduce a new method in the > > BrokerService to help brokers create the topic internally first(maybe just > > metadata is enough), and then to use a pulsar client to connect to it. > > I am sorry but I am not sure that this is enough to prevent > reads/writes from unallowed clients. > We should elaborate more on this topic on the PIP > > Enrico > > > > > WDYT? > > > > > > Best, > > Mattison > > On Feb 16, 2023, 00:01 +0800, Enrico Olivelli , wrote: > > > > I have one question (apologies for the top posting). > > > > > > > > The Broker (and the other Pulsar components) use the regular Pulsar > > > > client to connect to "system topics" > > > > and in general they use the Pulsar wire protocol. > > > > > > > > The question is "how do you distinguish an internal component from a > > > > user component ?" > > > > How can you say that the broker is able to connect to a system topic > > > > and any other client cannot do it ? > > > > > > > > Enrico > > > > > > > > Il giorno mer 15 feb 2023 alle ore 15:38 ha > > > > scritto: > > > > > > > > > > > > Hi Asaf > > > > > > > > > > > > There is a link to introduce the dynamic configuration. > > > > > > https://pulsar.apache.org/docs/2.10.x/admin-api-brokers/#dynamic-broker-configuration > > > > > > > > > > > > Best, > > > > > > Mattison > > > > > > On Feb 14, 2023, 17:06 +0800, Asaf Mesika , > > > > > > wrote: > > > > > > > > > > On Tue, Feb 14, 2023 at 3:46 AM > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > Hi, Asaf > > > > > > > > > > > > > > > > > > > > > > > > > > > > Welcome to join this discussion. > > > > > > > > > > > > > > > > > > > > > > You mean that allows the *system* > > > > > > > > > > > > > > > > > > > > > > to use it when it's a partitioned > > > > > > > > > > > > > > topic? > > > > > > > > > > > > > > Sorry, I didn't get your point. What do you mean by > > > > > > > > > > > > > > *system*? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This sentence was a reply to: > > > > > > > > > > > > > > > > > > > > 2. Make the `-partition-` string the keyword. That allows > > > > > > > > > > the user to use > > > > > > > > > > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I wanted to say that this sentence should be: > > > > > > > > > > Make the `-partition-` string the keyword, that allows the > > > > > > > > > > *system* to use > > > > > > > > > > it when it's a partitioned topic. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Why postfix of `__`?Why uppercase > > > > > > > > > > > > > > > > > > > > > > ?Maybe `__system__`? > > > > > > > > > > > > > > Yes, That is a key point that I want to discuss in > > > > > > > > > > > > > > this > > > > > > > > > > > > > > thread. `__system__` is good for me. > > > > > > > > > > > > > > > > > > > > > > Can you please elaborate what it > > > > > > > > > > > > > > > > > > > > > > means to make it dynamic exactly? > > > > > > > > > > > > > > Sorry, I will refine it. it means we can update > > > > > > > > > > > > > > this configuration > > > > > > > > > > > > > > dynamically. (using rest api or sth) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'm unfamiliar with how Pulsar supports dynamic > > > > > > > > > > configuration. I would > > > > > > > > > > love it if you can share a link or explain it briefly, thus > > > > > > > > > > explaining what > > > > > > > > > > exactly you are going to change to support dynamic > > > > > > > > > > configuration. > > > > > > > > > > > > > > > > > > > > > > > > General question: In the last thread you said > > > > > > > > > > > > > > something about > > > > > > > > > > > > > > configurablerules, etc? You decided not to use this > > > > > > > > > > > > > > idea? > > > > > > > > > > > > > > IMO, That idea is an advanced feature. we may need > > > > > > > > > > > > > > more time to discuss > > > > > > > > > > > > > > the details and for the topic name restriction, > > > > > > > > > > > > > > maybe we don't have strong > > > > > > > > > > > > > > reason to use that. > > > > > > > > > > > > > > > > > > > > > > > > > > > > We can introduce this advanced feature when
Re: Force redirect questions from Slack to GitHub Discussions or StackOverflow?
+100 Also note that for good or bad reasons, the number of questions on StaOverflow is often used as a metric for the popularity of a project. Le mer. 15 févr. 2023 à 13:12, Kiryl Valkovich a écrit : > Hi everyone! Enrico Olivelli asked me to repost it to the mailing list. > > --- Me > I’m very worried that good answers from David Kjerrumgaard and others > won’t be googleable because it’s in Slack. To make any search you need to > pay a monthly subscription. > > In my opinion, it would be wiser to make StackOverflow, Discuss, or GitHub > discussions a place for questions. And strictly redirect people who ask any > question in Slack to the right place. > In GitHub discussions, you also can manage categories, as well as you can > manage channels in Slack. > > Subscription to a specific category is coming soon. > https://github.com/community/community/discussions/3951#discussioncomment-3879939 > The most important that it’s searchable. > > Examples: > https://github.com/community/community/discussions > https://github.com/apache/superset/discussions > > --- Later me to David Kjerrumgaard > What do you think about the following workflow: > 1. The user asks a new question in Slack. > 2. You click the three dots button on his message -> “Convert to a GitHub > discussion”. > 3. The bot creates a new GitHub discussion in the “Q&A” category with the > original message’s content. > 4. The bot converts the original message to a thread and posts a message > like: > > In order to keep history and make your question searchable by other > Apache Pulsar users, it has been converted to a GitHub discussion. > > Further conversation is available by this link: > https://github.com/apache/pulsar/discussions/18766. > 5. You answer the question on GitHub. > > How it may work: > https://api.slack.com/best-practices/blueprints/actions-and-dialogs > I can try to implement it makes any sense for you. (edited) > > Full original thread: > https://apache-pulsar.slack.com/archives/C5Z4T36F7/p1676449603034309 > >
Re: Force redirect questions from Slack to GitHub Discussions or StackOverflow?
+1 Since web pages are more easily and public. Best Mattison On Feb 16, 2023, 07:58 +0800, Christophe Bornet , wrote: > +100 > Also note that for good or bad reasons, the number of questions on > StaOverflow is often used as a metric for the popularity of a project. > > Le mer. 15 févr. 2023 à 13:12, Kiryl Valkovich > a écrit : > > > Hi everyone! Enrico Olivelli asked me to repost it to the mailing list. > > > > --- Me > > I’m very worried that good answers from David Kjerrumgaard and others > > won’t be googleable because it’s in Slack. To make any search you need to > > pay a monthly subscription. > > > > In my opinion, it would be wiser to make StackOverflow, Discuss, or GitHub > > discussions a place for questions. And strictly redirect people who ask any > > question in Slack to the right place. > > In GitHub discussions, you also can manage categories, as well as you can > > manage channels in Slack. > > > > Subscription to a specific category is coming soon. > > https://github.com/community/community/discussions/3951#discussioncomment-3879939 > > The most important that it’s searchable. > > > > Examples: > > https://github.com/community/community/discussions > > https://github.com/apache/superset/discussions > > > > --- Later me to David Kjerrumgaard > > What do you think about the following workflow: > > 1. The user asks a new question in Slack. > > 2. You click the three dots button on his message -> “Convert to a GitHub > > discussion”. > > 3. The bot creates a new GitHub discussion in the “Q&A” category with the > > original message’s content. > > 4. The bot converts the original message to a thread and posts a message > > like: > > > > In order to keep history and make your question searchable by other > > Apache Pulsar users, it has been converted to a GitHub discussion. > > > > Further conversation is available by this link: > > https://github.com/apache/pulsar/discussions/18766. > > 5. You answer the question on GitHub. > > > > How it may work: > > https://api.slack.com/best-practices/blueprints/actions-and-dialogs > > I can try to implement it makes any sense for you. (edited) > > > > Full original thread: > > https://apache-pulsar.slack.com/archives/C5Z4T36F7/p1676449603034309 > > > >
Re: [VOTE] Pulsar Client C++ Release 3.1.2 Candidate 1
Close this vote with 3 binding +1s votes: * Matteo * Bo * Penghui Thanks, Yunze On Wed, Feb 15, 2023 at 12:31 PM PengHui Li wrote: > > +1 binding > > The tests failed with curl 7.84. upgrading to 7.87 fixed the issue. > > Thanks, > Penghui > > On Wed, Feb 15, 2023 at 10:36 AM 丛搏 wrote: > > > +1 (binding) > > > > - Build from source (macos x86_64 12.6) > > - Run pulsar-tests with apachepulsar/pulsar:2.11.0 docker images > > > > https://github.com/apache/pulsar-client-cpp/issues/192 I found in my env. > > > > Thanks, > > Bo > > > > Zike Yang 于2023年2月14日周二 15:35写道： > > > > > > +1 (non-binding) > > > > > > - Build from source (macos x86_64 12.6) > > > - Run example producer/consumer > > > - Run perf producer/consumer. > > > - Run unit tests > > > > > > I have found an issue when running the unit test [0]. But it seems not > > > a regression bug and only affects the test code. So I am +1. > > > > > > [0] https://github.com/apache/pulsar-client-cpp/issues/193 > > > > > > Thanks, > > > Zike Yang > > > > > > On Mon, Feb 13, 2023 at 9:48 AM Yunze Xu > > wrote: > > > > > > > > Hi Bo, > > > > > > > > This test takes about 1 minute to finish, did you wait long enough? I > > > > ran this test in my local env (Ubuntu 20.04) against a Pulsar 2.11.0 > > > > standalone and it succeeded. > > > > > > > > ``` > > > > $ ./tests/pulsar-tests > > > > --gtest_filter='BasicEndToEndTest.testMessageListenerPause' > > > > Note: Google Test filter = BasicEndToEndTest.testMessageListenerPause > > > > [==] Running 1 test from 1 test suite. > > > > [--] Global test environment set-up. > > > > [--] 1 test from BasicEndToEndTest > > > > [ RUN ] BasicEndToEndTest.testMessageListenerPause > > > > 2023-02-13 09:28:30.822 INFO [139733722046144] BasicEndToEndTest:922 > > > > | res = 204 > > > > 2023-02-13 09:28:30.822 INFO [139733722046144] ClientConnection:190 | > > > > [ -> pulsar://localhost:6650] Create ClientConnection, > > > > timeout=1 > > > > 2023-02-13 09:28:30.822 INFO [139733722046144] ConnectionPool:97 | > > > > Created connection for pulsar://localhost:6650 > > > > 2023-02-13 09:28:30.823 INFO [139733718923008] ClientConnection:388 | > > > > [127.0.0.1:37066 -> 127.0.0.1:6650] Connected to broker > > > > 2023-02-13 09:28:30.870 INFO [139733718923008] HandlerBase:72 | > > > > > > [persistent://public/default/partition-testMessageListenerPause-partition-0, > > > > ] Getting connection from pool > > > > 2023-02-13 09:28:31.116 INFO [139733718923008] ProducerImpl:202 | > > > > > > [persistent://public/default/partition-testMessageListenerPause-partition-0, > > > > ] Created producer on broker [127.0.0.1:37066 -> 127.0.0.1:6650] > > > > 2023-02-13 09:28:31.116 INFO [139733722046144] Client:87 | > > > > Subscribing on Topic :partition-testMessageListenerPause > > > > 2023-02-13 09:28:31.119 INFO [139733718923008] HandlerBase:72 | > > > > > > [persistent://public/default/partition-testMessageListenerPause-partition-0, > > > > subscription-name, 0] Getting connection from pool > > > > 2023-02-13 09:28:31.165 INFO [139733718923008] ConsumerImpl:238 | > > > > > > [persistent://public/default/partition-testMessageListenerPause-partition-0, > > > > subscription-name, 0] Created consumer on broker [127.0.0.1:37066 -> > > > > 127.0.0.1:6650] > > > > 2023-02-13 09:28:31.165 INFO [139733718923008] > > > > MultiTopicsConsumerImpl:274 | Successfully Subscribed to a single > > > > partition of topic in TopicsConsumer. Partitions need to create : 0 > > > > 2023-02-13 09:28:31.165 INFO [139733718923008] > > > > MultiTopicsConsumerImpl:137 | Successfully Subscribed to Topics > > > > 2023-02-13 09:28:31.165 INFO [139733722046144] > > > > MultiTopicsConsumerImpl:286 | [ Topics Consumer > > > > > > persistent://public/default/partition-testMessageListenerPause,subscription-name] > > > > Unsubscribing > > > > 2023-02-13 09:28:31.165 INFO [139733722046144] ConsumerImpl:289 | > > > > > > [persistent://public/default/partition-testMessageListenerPause-partition-0, > > > > subscription-name, 0] Unsubscribing > > > > 2023-02-13 09:28:31.187 INFO [139733718923008] ConsumerImpl:294 | > > > > > > [persistent://public/default/partition-testMessageListenerPause-partition-0, > > > > subscription-name, 0] Unsubscribed successfully > > > > 2023-02-13 09:28:31.187 INFO [139733718923008] > > > > MultiTopicsConsumerImpl:291 | [Muti Topics Consumer: TopicName - > > > > persistent://public/default/partition-testMessageListenerPause - > > > > Subscription - subscription-name]Unsubscribed successfully > > > > 2023-02-13 09:28:31.187 INFO [139733722046144] Client:87 | > > > > Subscribing on Topic :partition-testMessageListenerPause > > > > 2023-02-13 09:28:31.188 INFO [139733718923008] HandlerBase:72 | > > > > > > [persistent://public/default/partition-testMessageListenerPause-partition-0, > > > > subscription-name, 1] Getting connection from pool > > > > 2023-02-13 09:28:31.195 INFO [139733718923008] ConsumerImpl:238 |
Re: [DISCUSS] Switch the client complier to JDK 11 or 17 from JDK 8
-1 Upgrading the client is harder than upgrading the broker. Pulsar usually acts as a service to serve many lines of business and only a single team is responsible to maintain the server side. It would be hard to push all the client sides to upgrade the JDK. Instead of upgrading the minimum required JDK of the client, it's better to open a separate project if you think the client with JDK 17 could bring significant performance and security improvements. Thanks, Yunze On Wed, Feb 15, 2023 at 1:08 AM Zixuan Liu wrote: > > Hi all, > > We are using JDK 17 to compile all the components. For the main component, > the broker and proxy require JDK 17, and the client requires JDK 8. > > In Pulsar 3.x, we should keep up with modern JDKS for all components, and > over time, I believe many users have adopted JDK 11/17, which provides > significant performance and security improvements, see > https://learn.microsoft.com/en-us/java/openjdk/reasons-to-move-to-java-11 > > Switching to JDK17 is a bit radical, and we still need to consider this. > > Thanks, > Zixuan
