Re: [DISCUSS] PIP-177: Add the classLoader field for SchemaDefinition

2022-06-14 Thread Enrico Olivelli
Hello, how will it work with non JSON Schemas ? Enrico Il giorno mar 14 giu 2022 alle ore 17:57 Cong Zhao ha scritto: > > Hello Pulsar Community, > > > Here is a PIP to add the classLoader field for SchemaDefinition. I look > forward to your feedback. > > > PIP: https://github.com/apache/pulsar/

Re: [VOTE] PIP-177: Add the classLoader field for SchemaDefinition

2022-06-19 Thread Enrico Olivelli
+1 (binding) Enrico Il giorno ven 17 giu 2022 alle ore 03:21 Cong Zhao ha scritto: > > Hi Pulsar Community, > > > I would like to start a VOTE on "Add the classLoader field for > SchemaDefinition" (PIP-177). > > > The proposal can be read at https://github.com/apache/pulsar/issues/16058 > > and

Re: Pulsar logging framework design

2022-06-21 Thread Enrico Olivelli
Tison, Il giorno mar 21 giu 2022 alle ore 12:44 tison ha scritto: > > Hi devs, > > I learn that PulsarAdminImpl has a static block requires classpath contains > exact either of > > * slf4j-jdk14 > * jul-to-slf4j > > I'm curious what logging framework Pulsar choose among the codebase. AKAIF we ar

Re: [VOTE] PIP-172: Introduce the HEALTH_CHECK command in the binary protocol

2022-06-22 Thread Enrico Olivelli
Hi, IIUC nobody participated in the discussion. it is too early to start a VOTE -1 (temporary) Enrico Il giorno mer 22 giu 2022 alle ore 05:26 PengHui Li ha scritto: > > +1 > > Penghui > > On Wed, Jun 22, 2022 at 11:16 AM Cong Zhao wrote: > > > Hi Pulsar Community, > > > > I would like to sta

Re: [DISCUSS] PIP-172: Introduce the HEALTH_CHECK command in the binary protocol

2022-06-22 Thread Enrico Olivelli
I believe that this proposal is too broad. the PIP reads about: - adding HEALTHCHECK to the binary protocol - add a HEALTHCHECK cache on the broker Also, I would like to understand in which usecase you can use the binary endpoint and not the HTTP endpoint. Health Check is good for scripts and fo

Re: [DISCUSS] PIP-172: Introduce the HEALTH_CHECK command in the binary protocol

2022-06-22 Thread Enrico Olivelli
ery X seconds and usually not concurrently if you have multiple (tens? hundreds?) of Pulsar clients that require the HC, this will be a big problem, is this the reason why you want to add some cache to the response of the HC ? Enrico > > > On 2022/06/22 10:19:52 Enrico Olivelli wrote:

Re: Required and recommended JavaVersion for Pulsar broker

2022-06-22 Thread Enrico Olivelli
ranch-2.10` is compatible with jdk8. > > Thanks > ZhangJian He > > tison 于2022年6月22日周三 23:37写道: > > > Hi devs. > > > > During a review of PR-16163[1] I notice a comment from @Enrico Olivelli > > on PR-14953 said: > > > > > We are curre

Re: [DISCUSS] Enable GitHub Discussions?

2022-06-25 Thread Enrico Olivelli
Tison, Il Sab 25 Giu 2022, 21:32 tison ha scritto: > Hi, > > There is a previous discussion at > https://lists.apache.org/thread/83pst643h9cqcryo3zsjd240jmqzvn73. > > I'd like to bump this thread since INFRA has implemented forward GitHub > Discussions activities to mailing list[1][2][3]. > > It

Re: [VOTE] Pulsar Release 2.10.1 Candidate 1

2022-06-28 Thread Enrico Olivelli
Hang, Il giorno lun 27 giu 2022 alle ore 03:55 Hang Chen ha scritto: > > Hi Penghui, > Thanks for your great work! > > I found two problems with this candidate, I'm not sure whether it will > block this release. > - There are some CVEs in this candidate. Please refer to: > https://gith

Re: [VOTE] Pulsar Release 2.9.3 Candidate 1

2022-06-28 Thread Enrico Olivelli
Hang, Il giorno lun 27 giu 2022 alle ore 04:35 Hang Chen ha scritto: > > Hi Mattison, > Thanks for your great work! > > I found one problem with this candidate. > - When I import `pulsar-broker` and `managed-ledger` into my > project, I found the project build failed with the following >

Re: [VOTE] [PIP-179] Support the admin API to check unknown request parameters

2022-06-30 Thread Enrico Olivelli
+1 (binding) Enrico Il giorno gio 30 giu 2022 alle ore 08:57 Zike Yang ha scritto: > > +1 > > Zike Yang > > On Thu, Jun 30, 2022 at 12:38 PM mattison chao > wrote: > > > +1(non-binding) > > > > Best, > > Mattison > > > > On Thu, 30 Jun 2022 at 12:35, Michael Marshall > > wrote: > > > > > +1 >

Re: [DISCUSS] Move all generated documentation to apache/pulsar-site repo

2022-06-30 Thread Enrico Olivelli
+1 I like this idea. We should not commit generated content to the code git repo Enrico Il giorno gio 30 giu 2022 alle ore 06:59 Michael Marshall ha scritto: > > Hi Pulsar Community, > > I would like to discuss changing where our generated docs are stored. > > Currently, we keep some generated d

Re: [DISCUSS] Reject partitioned topic creation when the topic name contains the `partition` keyword.

2022-06-30 Thread Enrico Olivelli
Il giorno mer 29 giu 2022 alle ore 15:58 Dave Maughan ha scritto: > > Hi Mattison, > > 1. Should we reject the partitioned topic metadata creation when the topic > > name contains the `-partition-` keywords? > > > I'm not familiar with all the details but my naive opinion would be that we > should

Re: [VOTE] Pulsar Release 2.9.3 Candidate 1

2022-07-04 Thread Enrico Olivelli
ase. Right? > > > > Best, > > Mattison > > > > On Tue, 28 Jun 2022 at 20:21, mattison chao > > wrote: > > > >> @mattison It's better to upload the image under your org first so that we > >>> can verify the image > >> &g

Re: [VOTE] Pulsar Release 2.9.3 Candidate 1

2022-07-04 Thread Enrico Olivelli
; > On Tue, 28 Jun 2022 at 20:21, mattison chao > > > wrote: > > > > > >> @mattison It's better to upload the image under your org first so that we > > >>> can verify the image > > >> > > >> > > >> Sure, I will do

Re: [VOTE] PIP-181: Pulsar Shell

2022-07-04 Thread Enrico Olivelli
+1 (binding) Enrico Il giorno mar 5 lug 2022 alle ore 02:35 mattison chao ha scritto: > > +1(non-binding) > > > On Jul 4, 2022, at 10:45 PM, Nicolò Boschi wrote: > > > > Dear Community, > > > > I would like to start a VOTE on "PIP-181: Pulsar Shell" > > > > The proposal can be read at [0] and t

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

2022-07-05 Thread Enrico Olivelli
I have a couple of additional questions. 1. Security What about security permissions about the shadow topic ? We are reading from another topic. I think we must clarify the decisions in the PIP 2. Truncation and deletion What happens when you truncate or delete the source topic ? please add a par

Re: [VOTE] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership

2022-07-05 Thread Enrico Olivelli
-1 (binding) I left other comments on the discussion. I believe that this feature is very powerful and I generally support it. But we must clarify how it integrates with many other Pulsar features Thanks for driving this improvement Enrico Il giorno mar 5 lug 2022 alle ore 08:29 Haiting Jiang ha

Re: [DISCUSS] Remove Pulsar Broker Javadocs from Website

2022-07-06 Thread Enrico Olivelli
Micheal, I fully agree we need only the javadocs for: - the client - pulsar functions and pulsar IO - shared modules (pulsar-common...) Enrico Il giorno mer 6 lug 2022 alle ore 05:36 Michael Marshall ha scritto: > > Hi Pulsar Community, > > We currently host the generated Pulsar Broker Javadocs

Re: [DISSCUSS]PIP-181: Reduce unnecessary REST call in broker

2022-07-07 Thread Enrico Olivelli
Xiaoyu Hou, I support this PIP Enrico Il giorno gio 7 lug 2022 alle ore 05:37 Michael Marshall ha scritto: > > Hi Xiaoyu Hou, > > Thank you for your proposal. I agree that this proposal could help > reduce unnecessary overhead by calling the internal broker's methods > directly instead of initia

Re: [VOTE] Pulsar Release 2.9.3 Candidate 2

2022-07-08 Thread Enrico Olivelli
+1 (binding) - built from sources, with JDK11, run a couple of smoke tests - verified RAT - verified checksums and signatures - run some JMS tests using Transactions (the JMS library bundles 2.10.x Java client) Thanks for driving the release! Enrico Enrico Il giorno mer 6 lug 2022 alle ore 10:2

Re: [DISCUSS] PIP-184: Topic specific consumer priorityLevel

2022-07-08 Thread Enrico Olivelli
Dave, Why can't you create multiple independent Consumers ? They will share the same resources (memory pools, thread pools) Enrico Il giorno ven 8 lug 2022 alle ore 15:00 Dave Maughan ha scritto: > > Hi Pulsar community, > > I've opened a PIP to discuss a new Java client option to allow setti

Re: CI - remove cmd line test retries?

2022-07-12 Thread Enrico Olivelli
+1 Enrico Il giorno mar 12 lug 2022 alle ore 09:24 PengHui Li ha scritto: > > +1 > > Penghui > > On Tue, Jul 12, 2022 at 1:47 PM Michael Marshall > wrote: > > > +1 I support removing the retries for the many reasons already mentioned. > > > > - Michael > > > > On Tue, Jul 12, 2022 at 12:35 AM M

Batch Messages with only 1 message

2022-07-12 Thread Enrico Olivelli
Hello, I think that we could implement a small but effective enhancement to batching. It may happen that even if you enable batching you come to create entries with 1 only message. Processing batch messages requires a good amount of resources, both on the broker and on the client side. Especiall

Tool to calculate accurate backlog of a subscription (considering batching and entry filters)

2022-07-12 Thread Enrico Olivelli
Hello folks, there is a long standing issue about calculating an accurate backlog in presence of batch messages [1]. Now that we have PIP-105, broker side filters, the problem becomes bigger. I have sent a patch [2] to provide a feature to "analise" a Subscription and scan the non dispatched messa

Re: [VOTE] Enable GitHub Discussion for user-facing discussion

2022-07-12 Thread Enrico Olivelli
+1 Enrico Il giorno mar 12 lug 2022 alle ore 13:05 Yu ha scritto: > > Hi tison, thanks for creating this vote! > I thought we reached a lay consensus since there was no objection since the > last discussion [1] > > +1 for this proposal. > I've explained the reasons and showed the benefits previ

Re: Batch Messages with only 1 message

2022-07-12 Thread Enrico Olivelli
t; > Qiang Huang 于2022年7月12日周二 17:34写道: > > > > > > > +1 > > > > Good idea! It will greatly reduce the resource consumption of small > > > > batches. > > > > > > > > Yubiao Feng 于2022年7月12日周二 > > 16:00写道: >

Re: Batch Messages with only 1 message

2022-07-12 Thread Enrico Olivelli
ks Enrico > > Thanks, > Xiaoyu Hou > > Enrico Olivelli 于2022年7月12日周二 22:28写道: > > > Thank you all ! > > > > I am going to create a issue/feature request > > > > Is there anyone interested in working on that ? > > > > We need a PIP. &g

Re: Tool to calculate accurate backlog of a subscription (considering batching and entry filters)

2022-07-12 Thread Enrico Olivelli
delivered to a subscription Enrico > > Penghui > > On Tue, Jul 12, 2022 at 8:58 PM Enrico Olivelli wrote: > > > Hello folks, > > there is a long standing issue about calculating an accurate backlog > > in presence of batch messages [1]. > > Now that we have PIP-1

Re: [DISCUSS] PIP-184: Cluster migration or Blue-Green cluster deployment support in Pulsar

2022-07-13 Thread Enrico Olivelli
Very interesting. Overall I support this proposal A couple of questions: - What about monitoring the status of the migration? - Should we block all maintenance operations on the "blue" cluster ? like deleting/creating stuff - Should we stop ledger trimming and offloading ? - What about authenticat

Re: Tool to calculate accurate backlog of a subscription (considering batching and entry filters)

2022-07-13 Thread Enrico Olivelli
sation to understand if there is interest. Let me prepare an official PIP Thanks for your feedback Enrico > > Thanks, > Penghui > > On Tue, Jul 12, 2022 at 10:42 PM Enrico Olivelli > wrote: > > > Il giorno mar 12 lug 2022 alle ore 15:04 PengHui Li > > ha scritt

PIP-187 Add API to analyse a subscription backlog and provide a accurate value

2022-07-14 Thread Enrico Olivelli
Hello, this is a PIP to implement a tool to analyse the subscription backlog Link: https://github.com/apache/pulsar/issues/16597 Prototype: https://github.com/apache/pulsar/pull/16545 Below you can find the proposal (I will amend the GH issue while we discuss, as usual) Enrico Motivation Curre

Re: [DISCUSS] PIP-186: Introduce two phase deletion protocol based on system topic

2022-07-14 Thread Enrico Olivelli
This is very interesting. I have only one concern. I think that we should at least use a per-tenant system topic, or, better, per-namespace. There is no need to create the deletion topic if there is nothing to delete. I am used to dealing with Pulsar clusters in which Tenants are strictly isolate

Re: [DISCUSS]PIP-189: No batching if only one message in batch

2022-07-15 Thread Enrico Olivelli
Thanks for driving this initiative. I think that it is better to not add a configuration flag for this feature. Because: - it is always a good idea to not create a batch message - everybody will have to turn this feature explicitly on, otherwise there is no benefit - it is very hard to explain wh

Re: [DISCUSS] Apache Pulsar 2.11.0 Release

2022-07-18 Thread Enrico Olivelli
Nicolò, Il Lun 18 Lug 2022, 10:00 Nicolò Boschi ha scritto: > Thanks Penghui for the reminder. > I'd like to also include PIP: 181 Pulsar shell if the time permits. > > I believe that is a good idea to start testing the code freeze proposed by > PIP-175 (https://github.com/apache/pulsar/issues/1

Re: PIP-187 Add API to analyse a subscription backlog and provide a accurate value

2022-07-18 Thread Enrico Olivelli
Any comments ? FYI I have updated the PIP after addressing some feedback on the PR: - no more need to create a dummy Dispatcher - now we are reading entries in batches I would like to see this in 2.11 please Enrico Il giorno gio 14 lug 2022 alle ore 09:34 Enrico Olivelli ha scritto: > >

Re: PIP-187 Add API to analyse a subscription backlog and provide a accurate value

2022-07-19 Thread Enrico Olivelli
helpful to monitor real backlog in production > > > environments. > > > Currently we are struck at "entries" count but doesn't describe well > how > > > much real message lag we have. > > > > > > +1 excited to see it work on our production env

[VOTE] PIP-187 Add API to analyze a subscription backlog and provide a accurate value

2022-07-19 Thread Enrico Olivelli
This is the VOTE thread for PIP-187 This is the GH issue: https://github.com/apache/pulsar/issues/16597 This is the PR: https://github.com/apache/pulsar/pull/16545 The vote is open for at least 48 hours Below you can find a copy of the text of the PIP Best regards Enrico Motivation Currently

Re: [VOTE] PIP-100: Pulsar pluggable topic factory

2022-07-19 Thread Enrico Olivelli
Any other VOTEs ? Please anyone take a look Enrico Il giorno ven 8 ott 2021 alle ore 08:24 Enrico Olivelli ha scritto: > > +1 (binding) > > Enrico > > Il Ven 8 Ott 2021, 00:15 Rajan Dhabalia ha scritto: >> >> I would like to start voting on the PIP. >>

Re: PIP-187 Add API to analyse a subscription backlog and provide a accurate value

2022-07-20 Thread Enrico Olivelli
get a period size of the subscription. > Both solutions are expensive as opposed to nothing of course. Both has to > be a well documented conscious choice. > WDYT? Enrico > > Asaf > > > On Thu, Jul 14, 2022 at 10:34 AM Enrico Olivelli > wrote: > > > Hello, >

Re: [DISCUSS] PIP-186: Introduce two phase deletion protocol based on system topic

2022-07-20 Thread Enrico Olivelli
let's go with the system topic. Thanks Are we adding a configuratio flag? Enrico > On 2022/07/14 15:35:16 Enrico Olivelli wrote: > > This is very interesting. > > > > I have only one concern. > > I think that we should at least use a per-tenant system topic, or, &

Re: PIP-187 Add API to analyse a subscription backlog and provide a accurate value

2022-07-21 Thread Enrico Olivelli
are in the case of general purpose filtering (actually we allow Java code to be plugged into the browser), so pre-calculating the counters won't work well. Enrico > > Thanks, > Penghui > > On Wed, Jul 20, 2022 at 11:23 PM Asaf Mesika wrote: > > > On Wed, Jul 20, 2022 at

Re: [DISCUSS] PIP-191: Support batched message using entry filter

2022-07-21 Thread Enrico Olivelli
Thank you for this proposal ! I understand the problem, and I have already thought about it, because I am the author of some filters (especially the JMS Selectors Filter) but we have to clarify more about this PIP. 1) It is not clear to me if you want to push all the messages metadata in the main

Re: [DISCUSS] Apache Pulsar 2.8.4 release

2022-07-21 Thread Enrico Olivelli
Thank you Yunze I agree that it is time to cut a release of 2.8. We should also one last release out of 2.7 branch. I hope that someone will volunteer Enrico Il Gio 21 Lug 2022, 18:10 Dave Fisher ha scritto: > Thank you for volunteering! > > > On Jul 21, 2022, at 12:57 AM, Yunze Xu > wrote:

Re: PIP-187 Add API to analyse a subscription backlog and provide a accurate value

2022-07-22 Thread Enrico Olivelli
s subscription is 8. Makes sense to me. I will add an optional starting position, if it is not provided we will scan from the lastDeletedMarkPosition. The result of the scan will return the first and the last positions Let me update the PIP Enrico > > Thanks, > Penghui > On Thu, J

Re: [DISCUSS] Apache Pulsar 2.11.0 Release

2022-07-22 Thread Enrico Olivelli
oint is Pulsar > > > Summit > > > San Francisco will start on August 18, 2022. I think maybe we can start > > to > > > test > > > the master branch for now and continue the in-progress tasks. If we can > > > have a > > > major release before Pu

Re: [VOTE] PIP-187 Add API to analyze a subscription backlog and provide a accurate value

2022-07-26 Thread Enrico Olivelli
> > > > > > > > +1 > > > > > > > > > > > > > > On Tue, Jul 19, 2022 at 10:51 AM Dave Fisher > > > > wrote: > > > > > > > > > > > > > > > +1 (binding) > > > > > > > > >

Re: [DISCUSS] Restart Release Pulsar 2.7.5

2022-07-26 Thread Enrico Olivelli
+1 thank you Enrico Il giorno mar 26 lug 2022 alle ore 13:19 Haiting Jiang ha scritto: > > Hello Pulsar Community, > > Previously in early February, we had a discussion about 2.7.5 release in [1]. > But unfortunately, there seems to be a test regression in Candidate 1 and the > release process

Re: [VOTE]PIP-189: No batching if only one message in batch.

2022-07-26 Thread Enrico Olivelli
+1 (binding) This is a small improvement but a great impact !! Enrico Il giorno mar 26 lug 2022 alle ore 12:48 Anon Hxy ha scritto: > > Hi Zike > > Thanks for your reminding. I just forgot to update the issue and have > updated it now. I am not going to add the configuration > `batchingSingle

[ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar

2022-07-26 Thread Enrico Olivelli
I am glad to announce that the Apache Pulsar PMC invited Micheal to join the PMC and he accepted. Micheal is doing a great job in stewarding our community Please join me and celebrate ! Enrico Olivelli

Re: 回复: [Discuss] PIP-190: Simplify Pulsar documentation release and maintenance strategy

2022-07-26 Thread Enrico Olivelli
Yu Il Mer 27 Lug 2022, 05:33 Liu Yu ha scritto: > +1 for this proposal since it improves the efficiency of managing docs. > > We'll take it as we reach a lazy consensus and implement this plan if > there is no objection until tomorrow. > This is a PIP so we must wait for a explicit approval by

Re: [DISCUSS] PIP-195: New bucket based delayed message tracker

2022-07-26 Thread Enrico Olivelli
Good proposal. How do you enable this feature? A flag in broker.conf? Is it possible to enable the feature by doing a rolling upgrade of the brokers? What happens if you have brokers that are still running the old tracker? Is it possible to rollback seamlessly? Enrico Il Mer 27 Lug 2022, 05:39

Re: [VOTE] PIP-193 : Sink preprocessing Function

2022-07-28 Thread Enrico Olivelli
+1 (binding) Thank you I am sure that this feature will have a huge impact on Pulsar IO users Enrico Il giorno gio 28 lug 2022 alle ore 12:39 Christophe Bornet ha scritto: > > Hi, Pulsar community, > > I'd like to start a vote on PIP-193 : Sink preprocessing Function > > You can find the propo

Re: [VOTE] PIP-184: Topic specific consumer priorityLevel

2022-07-29 Thread Enrico Olivelli
+1 (binding) Enrico Il Ven 29 Lug 2022, 07:13 PengHui Li ha scritto: > +1 (binding) > > Penghui > On Jul 25, 2022, 13:42 +0800, mattison chao , > wrote: > > +1(non-binding) > > > > Best, > > Mattison > > > > On Mon, 25 Jul 2022 at 13:19, guo jiwei wrote: > > > > > > +1 > > > > > > Regards > >

Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.

2022-07-29 Thread Enrico Olivelli
e don't need to add that field anymore because we are going to use optional MessageIdData message_id = 9; is this correct ? Enrico > > Thanks, > Haiting > > On 2022/07/07 11:30:59 Haiting Jiang wrote: > > Hi Enrico, > > > > Thanks for your feedback. > >

Re: [VOTE] PIP-190: Simplify Pulsar documentation release and maintenance strategy

2022-07-29 Thread Enrico Olivelli
+1 (binding) Enrico Il giorno ven 29 lug 2022 alle ore 07:02 PengHui Li ha scritto: > > +1 (binding) > > Penghui > On Jul 28, 2022, 16:40 +0800, Zike Yang , wrote: > > +1 (non-binding) > > This will greatly reduce the amount of work that contributors have to > > do to update and maintain the doc

Re: [DISCUSS] Does stale bot make value for you?

2022-07-30 Thread Enrico Olivelli
Il Sab 30 Lug 2022, 17:53 tison ha scritto: > For example, even if we close (and lock?) the issue or pull request after a > certain interval, the stale bot helps on transforming issues state with a > clear rule (although a human action could be more friendly). > > Instead, we leave a comment and

Re: [DISCUSS] PIP 197: Add Schema hash and equals to public API

2022-08-11 Thread Enrico Olivelli
A Schema instance is stateful (and contians mutable state) and in the end it contains a reference to the PulsarClient instance that is using it. Look at how AutoConsume schema works for instance. I don't think it is possible to support such API in the general case. If you want to use it as a key

Re: [DISCUSS] PIP-186: Introduce two phase deletion protocol based on system topic

2022-08-11 Thread Enrico Olivelli
Il Sab 23 Lug 2022, 03:26 Yan Zhao ha scritto: > Yes, wo define `twoPhaseDeletionEnabled` in the Service Configuration. > Thanks. I suggest to include 'topic' in the flag, we have too many entities in Pulsar Enrico >

Re: [VOTE] PIP-191: Support batched message using entry filter

2022-08-16 Thread Enrico Olivelli
-1 I am sorry but I don't think that this PIP brings some improvements. I was out and I wasn't able to participate in the discussion. The final proposal is to change the way the Producer creates batches but: 1) Messages will be still "batched" so the EntryFilter won't see benefits (the filter cann

Re: [DISCUSS] Skip unnecessary tests when there are only cpp/python related changes

2022-08-16 Thread Enrico Olivelli
+1 Enrico Il giorno mar 16 ago 2022 alle ore 11:32 Nicolò Boschi ha scritto: > > A big +1 for this! > > Nicolò Boschi > > > Il giorno dom 14 ago 2022 alle ore 06:20 Qiang Huang < > qiang.huang1...@gmail.com> ha scritto: > > > +1. It can help reduce a lot of useless duplication of test cases. > >

Re: [DISCUSS] PIP-200 Package Pulsar Trino distro and config in a dedicated folder

2022-08-17 Thread Enrico Olivelli
I generally agree with the PIP Can you please explain the interactions with the Pulsar Helm chart ? also we have to draw a migration path, because users that will upgrade Pulsar will have to move the configuration files in another location Enrico Il giorno mer 17 ago 2022 alle ore 11:15 tison h

Re: [DISCUSS] Apache Pulsar 2.11.0 Release

2022-08-17 Thread Enrico Olivelli
ei Guo (Tboy) > > > > > > > > > On Wed, Jul 27, 2022 at 10:59 AM Zixuan Liu wrote: > > > > > >> +1 > > >> > > >> Thanks, > > >> Zixuan > > >> > > >> Yunze Xu 于2022年7月26日周二 23:34写道: > >

Re: [DISCUSS] Apache Pulsar 2.11.0 Release

2022-08-17 Thread Enrico Olivelli
ago 2022 alle ore 14:40 Nicolò Boschi ha scritto: > > This is another blocker for 2.11.0 for a regression about streaming > dispatch that I found in the 2.11 branch > https://github.com/apache/pulsar/pull/17143 > > Nicolò Boschi > > > Il giorno mer 17 ago 2022 alle ore 11:45 E

Re: [DISCUSS] Add an auth data const for refresh the original auth data

2022-08-17 Thread Enrico Olivelli
So the problem is that the Proxy is not requesting a refresh ? Enrico Il giorno mer 17 ago 2022 alle ore 16:26 Zixuan Liu ha scritto: > > Thank @Qiang! > > Update the diagram: > https://drive.google.com/file/d/1E6z0dzXzvW5ZxG6d6YUghL9OikA8j4UC/view?usp=sharing > > Thanks, > Zixuan > > Qiang Huan

[DISCUSS] [PIP-201] Extensions mechanism for Pulsar Admin CLI tools

2022-08-18 Thread Enrico Olivelli
Hello, I have drafted a PIP around this proposal. PIP link: https://github.com/apache/pulsar/issues/17155 I am preparing an official PR, I already have a working prototype. Copy of the contents of the GH issue is attached for discussion here on the Mailing list. Motivation There are many proje

Re: [ANNOUNCE] Jiwei Guo as a new PMC member in Pulsar

2022-08-18 Thread Enrico Olivelli
Congratulations ! Enrico Il giorno gio 18 ago 2022 alle ore 17:10 Cong Zhao ha scritto: > > Congratulations! > > Thanks, > Cong Zhao > > On 2022/08/18 11:24:01 PengHui Li wrote: > > Hi, all > > > > I'm glad to announce that the Apache Pulsar PMC invited Jiwei Guo to join > > the > > PMC and he

Re: [DISCUSS] Enable message deduplication for replicator by default

2022-08-18 Thread Enrico Olivelli
I agree Enrico Il Gio 18 Ago 2022, 18:23 PengHui Li ha scritto: > Hi all, > > When I tried to fix a problem related to replicator > https://github.com/apache/pulsar/pull/17154 > It surprised me that the message deduplication will not work by default > with the replicator. > I always thought it

Re: [DISCUSS] [PIP-201] Extensions mechanism for Pulsar Admin CLI tools

2022-08-22 Thread Enrico Olivelli
/github.com/datastax/pulsar-jms/pull/53/files#diff-9afaac9c7dc4b3d674e0623cd3d76348b01537c6095e9b5b8e804f59a481cceeR31 it is only a mock command at the moment, but it is good to showcase the feature Enrico > > Thanks, > Yunze > > > > > > 2022年8月18日 16:23,Enrico

Re: [DISCUSS] [PIP-201] Extensions mechanism for Pulsar Admin CLI tools

2022-08-22 Thread Enrico Olivelli
should be > complete and more clear. > I will leave that to the implementation PR Enrico > Thanks, > Yunze > > > > > > 2022年8月22日 18:37,Enrico Olivelli 写道: > > > > Yunze, > > > > Il giorno lun 22 ago 2022 alle ore 08:06 Yunze Xu > > ha scri

Re: [DISCUSS] [PIP-201] Extensions mechanism for Pulsar Admin CLI tools

2022-08-22 Thread Enrico Olivelli
Yunze, Il Lun 22 Ago 2022, 20:20 Enrico Olivelli ha scritto: > > > Il Lun 22 Ago 2022, 17:07 Yunze Xu ha > scritto: > >> I will take a look. But I think we should also add a trivial example >> (or a test) in the Apache repo > > I will add integration tests f

Re: [VOTE] [PIP-169] Support split bundle by flow or qps

2022-08-24 Thread Enrico Olivelli
+1 (binding) Enrico Il giorno mer 24 ago 2022 alle ore 07:13 Aloys Zhang ha scritto: > > +1 > > Haiting Jiang 于2022年8月24日周三 12:35写道: > > > +1 (non) > > > > Thanks, > > Haiting > > > > On Wed, Aug 24, 2022 at 10:19 AM guo jiwei wrote: > > > > > +1 (binding) > > > > > > > > > Regards > > > Jiwe

Re: [DISCUSS] [PIP-201] Extensions mechanism for Pulsar Admin CLI tools

2022-08-24 Thread Enrico Olivelli
he pulsar admin tool. I like that the design avoids an unnecessary > dependency on JCommander. > > Thanks, > Michael > > > On Mon, Aug 22, 2022 at 1:22 PM Enrico Olivelli wrote: > > > > Yunze, > > > > > > > > Il Lun 22 Ago 2022, 20:20 Enrico Oli

[VOTE] [PIP-201] Extensions mechanism for Pulsar Admin CLI tools

2022-08-24 Thread Enrico Olivelli
the subject of the VOTE. Best regards Enrico Olivelli

Re: [DISCUSS] Apache Pulsar 2.10.2 release

2022-08-24 Thread Enrico Olivelli
+1 Enrico Il giorno mer 24 ago 2022 alle ore 14:49 Qiang Huang ha scritto: > > +1 > > PengHui Li 于2022年8月23日周二 23:42写道: > > > +1 > > > > And, I have added the release/2.10.2 label, Lan Liang. > > > > Thanks, > > Penghui > > > > On Tue, Aug 23, 2022 at 11:02 PM Lan Liang > > wrote: > > > > > H

Re: [DISCUSS] Use Github project to track the multi clients development

2022-08-24 Thread Enrico Olivelli
I think that the main problem is that in our community we have plenty of Java developers and there are a few committers that review the patches on non-Java client. The same problem is also for other projects like the Helm Chart. Also, the Java client is used internally by the broker so it is usuall

[DISCUSS] Catch up reads with many subscriptions - saving reads from BK

2022-08-24 Thread Enrico Olivelli
Hello folks, These days I am working on a use case in which I have many subscriptions on the same topic (~100). If you get to a situation with a big backlog, the consumers are no more able to leverage the broker cache then the broker starts to read from BookKeeper. In 2.11 we have a first mechanis

Re: [DISCUSS] Catch up reads with many subscriptions - saving reads from BK

2022-08-24 Thread Enrico Olivelli
duplicated in the bookie's netty direct memory > >> pool. With this design, an entry will be materialized once with many > >> different ByteBuffer's objects pointing to it. > >> > >> Thanks, > >> Michael > >> > >> On Wed, Aug

Re: [DISCUSS] Catch up reads with many subscriptions - saving reads from BK

2022-08-25 Thread Enrico Olivelli
After doing more testing I have updated the patch with a small change but very effective https://github.com/apache/pulsar/pull/17241 With the latest commit now we are able to attach to pending requests for ranges that are "larger" than the requested range. So if you have 100...200 in pending state

Re: [Discuss] [PIP-204] Extensions for BrokerInterceptor

2022-08-25 Thread Enrico Olivelli
Makes sense +1 Enrico Il giorno ven 26 ago 2022 alle ore 05:17 Aloys Zhang ha scritto: > > Thanks for your suggestion. > `onMessagePublish` is more precise for understanding. > > PengHui Li 于2022年8月26日周五 10:30写道: > > > Looks good to me. > > I just have one suggestion. > > > > Maybe using `onMes

Re: [DISCUSS] PIP-204: Reactive Java client for Apache Pulsar

2022-08-30 Thread Enrico Olivelli
I support the initiative, especially because I guess we will need someday to build a new native Java client and do not wrap the existing one. Also, Pulsar must be on that ecosystem with a strong presence. Enrico Il Lun 29 Ago 2022, 15:11 Lari Hotari ha scritto: > I updated it to be PIP-205 sinc

Re: [VOTE] [PIP-204] Extensions for BrokerInterceptor

2022-08-30 Thread Enrico Olivelli
+1 (binding) Enrico Il giorno mar 30 ago 2022 alle ore 13:24 lin chen ha scritto: > > +1 > > Aloys Zhang 于2022年8月30日周二 19:02写道: > > > Hi devs, > > > > This is the VOTE thread for PIP-204 Extensions for BrokerInterceptor. > > > > PIP issue: https://github.com/apache/pulsar/issues/17267 > > > >

Re: [VOTE] [PIP-201] Extensions mechanism for Pulsar Admin CLI tools

2022-08-31 Thread Enrico Olivelli
ve > > > > > > Sent from my iPhone > > > > > > > On Aug 24, 2022, at 12:43 AM, Yunze Xu > > > wrote: > > > > > > > > +1 (non binding) > > > > > > > > Thanks, > > > > Yunze > > > > &

Re: [VOTE] PIP-200 Package Pulsar Trino distro and config in a dedicated folder

2022-08-31 Thread Enrico Olivelli
+1 (binding) Enrico Il giorno mer 31 ago 2022 alle ore 13:47 PengHui Li ha scritto: > > +1 (binding) > > Penghui > > On Mon, Aug 29, 2022 at 10:18 PM Nicolò Boschi wrote: > > > +1 (non binding) > > Thanks for taking care of this > > > > Nicolò Boschi > > > > > > Il giorno lun 29 ago 2022 alle o

Re: [VOTE] PIP-205: Reactive Java client for Apache Pulsar

2022-08-31 Thread Enrico Olivelli
+1 (binding) Enrico Il giorno mer 31 ago 2022 alle ore 16:06 Lari Hotari ha scritto: > > Hello, > > This is the official VOTE thread for PIP-205: Reactive Java client for Apache > Pulsar. > PIP link: https://github.com/apache/pulsar/issues/17335 > discussion: https://lists.apache.org/thread/xkf

Re: [VOTE] Pulsar Release 2.7.5 Candidate 3

2022-09-01 Thread Enrico Olivelli
+1 (binding) Built from sources and run some tests. I wasn't able to run all the tests from pulsar-broker, it takes too much time and there are flaky tests. Btw we know the status of the branch, now for newer branches we are in better shape. There is no need to hold the release thank you Haiting

Re: [DISCUSS] General question regarding PIP comments

2022-09-01 Thread Enrico Olivelli
Asaf, good question! Il giorno gio 1 set 2022 alle ore 15:01 Asaf Mesika ha scritto: > > Hi, > > I'm fairly new to this project and I have seen 2 options to comment to a > PIP: > 1. In the GitHub issue. > 2. In the mailing list. > > I was wondering what is the best practice. In Apache projects

Re: Pulsar CI congested, master branch build broken

2022-09-02 Thread Enrico Olivelli
Lari, Il giorno mar 30 ago 2022 alle ore 14:39 Lari Hotari ha scritto: > > Pulsar CI continues to be congested, and the build queue is long. > > I would strongly advice everyone to use "personal CI" to mitigate the issue > of the long delay of CI feedback. You can simply open a PR to your own >

Re: [VOTE] PIP-195: New bucket based delayed message tracker

2022-09-05 Thread Enrico Olivelli
+1 (binding) Enrico Il giorno lun 5 set 2022 alle ore 11:54 lordcheng10 ha scritto: > > +1 > > guo jiwei 于2022年9月1日周四 11:00写道: > > > +1 Great work! > > > > > > Regards > > Jiwei Guo (Tboy) > > > > > > On Wed, Aug 31, 2022 at 6:19 PM Ran Gao wrote: > > > > > +1 Great work! > > > > > > Thanks, >

Re: [DISCUSS] Apache Pulsar 2.11.0 Release

2022-09-05 Thread Enrico Olivelli
scribeAsync(org.apache.pulsar.client.impl.MultiTopicsConsumerImplTest) > >> > > Time elapsed: 5.05 s <<< FAILURE! > >> > > org.testng.internal.thread.ThreadTimeoutException: Method > >> > > > >> > > > >> > > &g

[DISCUSS] Pulsar 3.0 - dreaming of the ability to rename objects

2022-09-05 Thread Enrico Olivelli
Hello, Now that we have started to think about Pulsar 3.0, let me share another long standing feature that can be implemented mostly only with such big changes that would deserve a major version bump. Currently every object managed by Pulsar is directly handled and referred using a logical, human

Re: CloudEvents binding requires standard for Pulsar

2022-09-07 Thread Enrico Olivelli
Devin, thanks for bringing up this discussion. I have one high level question: what is the goal that we want to achieve ? something like: 1) Use CloudEvents format natively in Pulsar Schema registry, so that Pulsar clients can register their schema using that format 2) Publish on some HTTP endpoin

Re: [DISCUSS] Pulsar 3.0 - dreaming of the ability to rename objects

2022-09-07 Thread Enrico Olivelli
evel object that is a "container of tenants" and we don't even have the concept of "user". In order to support this use case we have to add a new object, like a "space", and when you connect to Pulsar you set the "space", then every reference to a tenan

Re: [Pulsar Test]Some updates about Pulsar Jenkins test

2019-12-30 Thread Enrico Olivelli
Il mar 31 dic 2019, 04:08 Guangning E ha scritto: > At present, I noticed that some people in pulsar's warehouse could not > trigger the test, so I create a Jira > https://issues.apache.org/jira/browse/INFRA-19626?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel > to > track t

Re: Flaky tests [VOTE] Pulsar Release 2.5.0 Candidate 2

2020-01-13 Thread Enrico Olivelli
(out of the VOTE thread) I have been trying to run the tests with Mac + jdk8 but without success. it is expected that we have lots of flaky tests ? I have never succeeded to run the full suite without a JVM crash btw I am continuing my tests Regards Enrico Il giorno lun 13 gen 2020 alle ore 10:

Re: [VOTE]Github action migration

2020-01-14 Thread Enrico Olivelli
+1 (non binding) I hope we can do the same in Apache Bookkeeper Enrico Il mer 15 gen 2020, 04:34 xiaolong ran ha scritto: > LGTM +1 > > -- > Thanks > Xiaolong Ran > > > 在 2020年1月15日,上午10:10,Guangning E 写道: > > > > Now pulsar's pull request is blocked on the test. We hope to migrate the > > t

New proposal on BookKeeper service discovery

2020-01-16 Thread Enrico Olivelli
Hi Pulsar community I am just cc'ing you regarding a proposal on BookKeeper, that will enable BookKeeper to abstract from using the network address of the bookie in ledger metadata. It is BP-38 Publish Bookie Service Info on Metadata Service You can find the design document here https://github.

Re: [VOTE] Adopt using release labels for minor releases

2020-01-18 Thread Enrico Olivelli
+1 ( non binding) Enrico Il sab 18 gen 2020, 22:43 Sijie Guo ha scritto: > Hi all, > > This is the official vote thread for adopting using release labels for > minor release as proposed in > > https://lists.apache.org/thread.html/ra057f9bb3385f62a9216a202260805074c3f1af77c144907fa1b50cd%40%3Cde

Re: pulsarbot

2020-02-05 Thread Enrico Olivelli
Very interesting. I left a comment on a merged PR (#3). Scripts do not have ASF license header. Not a big deal because we will never "release" them. It would be awesome to backport to BookKeeper once we are stable here in Pulsar Thanks Enrico Il giorno mer 5 feb 2020 alle ore 16:06 Devin Bost

Re: [VOTE] Pulsar Release 2.5.1 Candidate 3

2020-04-13 Thread Enrico Olivelli
Guagning it looks like your GPG key is not here https://dist.apache.org/repos/dist/release/pulsar/KEYS please add your key to the file Enrico Il giorno lun 13 apr 2020 alle ore 09:12 Guangning E ha scritto: > This is the third release candidate for Apache Pulsar, version 2.5.1. > > It fixes th

<    1   2   3   4   5   6   7   8   9   10   >