[ANNOUNCE] Apache Pulsar 2.9.3 released

2022-07-20 Thread mattison chao
The Apache Pulsar team is proud to announce Apache Pulsar version 2.9.3. Pulsar is a highly scalable, low latency messaging platform running on commodity hardware. It provides simple pub-sub semantics over topics, guaranteed at-least-once delivery of messages, automatic cursor management for subsc

[GitHub] [pulsar-helm-chart] Anmol057 commented on pull request #274: Bump Apache Pulsar 2.10.1

2022-07-20 Thread GitBox
Anmol057 commented on PR #274: URL: https://github.com/apache/pulsar-helm-chart/pull/274#issuecomment-1190154668 Hi, when can we expect 2.10.1 to be released? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [pulsar-helm-chart] Anmol057 commented on issue #267: helm chart for pulsar version 2.10.0

2022-07-20 Thread GitBox
Anmol057 commented on issue #267: URL: https://github.com/apache/pulsar-helm-chart/issues/267#issuecomment-1190155738 Hi, When can we expect the 2.10.0 version to be released? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [pulsar-site] SignorMercurio commented on pull request #147: Generate config docs from source code

2022-07-20 Thread GitBox
SignorMercurio commented on PR #147: URL: https://github.com/apache/pulsar-site/pull/147#issuecomment-1190258286 @Anonymitaet PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [pulsar-helm-chart] MBcom commented on pull request #233: feat(certs): use actual v1 spec for certs

2022-07-20 Thread GitBox
MBcom commented on PR #233: URL: https://github.com/apache/pulsar-helm-chart/pull/233#issuecomment-1190300067 +1 We were able to successfully upgrade to v2.9 after making these changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: PIP-187 Add API to analyse a subscription backlog and provide a accurate value

2022-07-20 Thread Asaf Mesika
I'm not sure I understand the context exactly: You say today we can only know the number of entries, hence we'll have a wrong number of backlog for subscription since: 1. One entry contains multiple messages (batch message) 2. Subscription may contain a filter, which requires you to read the entir

Re: [VOTE] PIP-187 Add API to analyze a subscription backlog and provide a accurate value

2022-07-20 Thread Asaf Mesika
Sorry to barge in the vote - I forgot to send my reply on the discussion 2 days ago :) On Tue, Jul 19, 2022 at 11:22 PM Nicolò Boschi wrote: > +1, thanks > > Nicolò Boschi > > Il Mar 19 Lug 2022, 22:16 Christophe Bornet ha > scritto: > > > +1 > > > > Le mar. 19 juil. 2022 à 20:01, Andrey Yegor

[GitHub] [pulsar-helm-chart] MBcom opened a new issue, #278: Make included Prometheus server configurable

2022-07-20 Thread GitBox
MBcom opened a new issue, #278: URL: https://github.com/apache/pulsar-helm-chart/issues/278 **The prometheus configuration is currently hardcoded. Extending it is not possible - e.g. connecting it with an external alert manager.** We want to connect the included Prometheus server to an ex

[GitHub] [pulsar-helm-chart] MBcom opened a new issue, #279: Make included prometheus server configurable

2022-07-20 Thread GitBox
MBcom opened a new issue, #279: URL: https://github.com/apache/pulsar-helm-chart/issues/279 **The prometheus configuration is currently hardcoded. Extending it is not possible - e.g. connecting it with an external alert manager.** We want to connect the included Prometheus server to an ex

[GitHub] [pulsar-helm-chart] MBcom closed issue #279: Make included prometheus server configurable

2022-07-20 Thread GitBox
MBcom closed issue #279: Make included prometheus server configurable URL: https://github.com/apache/pulsar-helm-chart/issues/279 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [pulsar-helm-chart] MBcom opened a new pull request, #280: Make included prometheus server configurable

2022-07-20 Thread GitBox
MBcom opened a new pull request, #280: URL: https://github.com/apache/pulsar-helm-chart/pull/280 Fixes #278 ### Motivation We want to connect the included Prometheus server to an external Alert Manager which is currently not possible. ### Modifications We have mov

[GitHub] [pulsar-helm-chart] MBcom opened a new issue, #281: Allow setting nodePort Option for proxy service

2022-07-20 Thread GitBox
MBcom opened a new issue, #281: URL: https://github.com/apache/pulsar-helm-chart/issues/281 **Is your feature request related to a problem? Please describe.** We can not set a specific nodePort for the proxy service. We want to persist a specific nodePort to make the Chart redeployable in

[GitHub] [pulsar-helm-chart] MBcom opened a new pull request, #282: Allow setting nodePort Option for proxy service

2022-07-20 Thread GitBox
MBcom opened a new pull request, #282: URL: https://github.com/apache/pulsar-helm-chart/pull/282 Fixes #281 ### Motivation We can not set a specific nodePort for the proxy service. We want to persist a specific nodePort to make the Chart redeployable in our infrastructure.

Re: PIP-187 Add API to analyse a subscription backlog and provide a accurate value

2022-07-20 Thread Enrico Olivelli
Asaf, Il giorno mer 20 lug 2022 alle ore 15:40 Asaf Mesika ha scritto: > > I'm not sure I understand the context exactly: > > You say today we can only know the number of entries, hence we'll have a > wrong number of backlog for subscription since: > 1. One entry contains multiple messages (batch

Re: PIP-187 Add API to analyse a subscription backlog and provide a accurate value

2022-07-20 Thread Asaf Mesika
On Wed, Jul 20, 2022 at 5:46 PM Enrico Olivelli wrote: > Asaf, > > Il giorno mer 20 lug 2022 alle ore 15:40 Asaf Mesika > ha scritto: > > > > I'm not sure I understand the context exactly: > > > > You say today we can only know the number of entries, hence we'll have a > > wrong number of backlo

[DISCUSS] Alternatives to changing public protocol

2022-07-20 Thread Asaf Mesika
Hi, We started discussing in PIP-180, which Penghui recommended I move to a dedicated thread. Pulsar has a public API in its binary protocol, which the clients use to communicate with it. Nonetheless, it is its public API to the server. I believe the public API should not be changed for internal

Re: [DISCUSS] PIP-186: Introduce two phase deletion protocol based on system topic

2022-07-20 Thread Yan Zhao
Hi, Enrico. If we bind the the topic with per-tenant, when tenant be deleted or the tenant not be load anymore, we data in the tennat system topic can't be consumed before the tenant next load. On 2022/07/14 15:35:16 Enrico Olivelli wrote: > This is very interesting. > > I have only one concern

[GitHub] [pulsar-site] merlimat merged pull request #151: Add Pulsar Summit San Francisco

2022-07-20 Thread GitBox
merlimat merged PR #151: URL: https://github.com/apache/pulsar-site/pull/151 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apac

Re: [DISCUSS] PIP-192 New Pulsar Broker Load Balancer

2022-07-20 Thread Rajan Dhabalia
Hi, I have gone through PIP but I don't see some basic information as part of PIP: 1. current issue in current load balancer strategy 2. Are there any performance and feature gaps in the current load balancer? Please provide data or metrics to show the impact. 3. what exactly we are solving in new

Re: [DISCUSS] PIP-186: Introduce two phase deletion protocol based on system topic

2022-07-20 Thread Enrico Olivelli
Il Mer 20 Lug 2022, 18:05 Yan Zhao ha scritto: > Hi, Enrico. If we bind the the topic with per-tenant, when tenant be > deleted or the tenant not be load anymore, we data in the tennat system > topic can't be consumed before the tenant next load. > This is a good point. So let's go with the sys

Re: [DISCUSS] PIP-192 New Pulsar Broker Load Balancer

2022-07-20 Thread Heesung Sohn
Hi Rajan, Please find my answers inline. Regards, Heesung On Wed, Jul 20, 2022 at 1:37 PM Rajan Dhabalia wrote: > Hi, > > I have gone through PIP but I don't see some basic information as part of > PIP: > 1. current issue in current load balancer strategy 2. Are there any performance and feat

Re: [DISCUSS] PIP-192 New Pulsar Broker Load Balancer

2022-07-20 Thread Matteo Merli
On Wed, Jul 20, 2022 at 1:37 PM Rajan Dhabalia wrote: > Also, here we have to clearly document that it will not impact behavior of > existing load-balancers now or in future. Pulsar is used by many companies > and orgs so, deprecating and not maintaining existing components is not > acceptable in

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

2022-07-20 Thread PengHui Li
Hi Haiting, One question about the schema. How can the consumer get the schema from the shadow topic during consumption? We should add this part in the proposal. Thanks, Penghui On Mon, Jul 11, 2022 at 9:09 PM Asaf Mesika wrote: > On Thu, Jun 23, 2022 at 6:26 AM Haiting Jiang > wrote: > > > H

Re: [DISCUSS] Alternatives to changing public protocol

2022-07-20 Thread PengHui Li
Thanks for starting this proposal Asaf. I'm trying to think more about this part today. Currently, the public API protocol is defined in PulsarApi.proto [1] And we have internal proto files such as MLDataFormats.proto [2], SchemaRegistryFormat.proto [3], ResourceUsage.proto [4]. It looks like th

Re: PIP-187 Add API to analyse a subscription backlog and provide a accurate value

2022-07-20 Thread PengHui Li
> What if the topic owner creates an internal subscription, consumes the messages, and updates a count per filter. I agree with this approach. If we need to scan all the backlogs to calculate the accurate backlogs for each operation, it's so expensive and difficult to apply to the production envir

Re: [DISCUSS] Apache Pulsar 2.11.0 Release

2022-07-20 Thread PengHui Li
Thanks for volunteering Nicolò. > So a plan could be to try to merge the work in progress targeted for 2.11 by the mid of August and then start the code freezing as described in the PIP. So the target release date will be early September. One point is Pulsar Summit San Francisco will start on Aug