Re: [questions] Which jakartaEE version we plan for pulsar 3.0.0?

2023-02-27 Thread Zixuan Liu
Only the `canal` module requires the spring, so don't worry. If I wrong, please let me know. Thanks, Zixuan ZhangJian He 于2023年2月27日周一 14:04写道: > Hi @Enrico Olivelli @Zixuan Liu > > I'm concerned that this change may create some complex compatibility issues > between our client and Springboot

Re: [ANNOUNCE] New committer: Yuri Mizushima

2023-02-27 Thread Zixuan Liu
Congrats! Yuri Best, Zixuan Horizon <1060026...@qq.com.invalid> 于2023年2月27日周一 15:57写道: > Congrats! > >

Re: [questions] Which jakartaEE version we plan for pulsar 3.0.0?

2023-02-27 Thread Enrico Olivelli
Are we talking about the Server side or the Pulsar Admin Client (that uses HTTP) ? On the server side we don't care about "SpringBoot", we are not using it Can you please elaborate more ? Enrico Il giorno lun 27 feb 2023 alle ore 09:02 Zixuan Liu ha scritto: > > Only the `canal` module requires

Re: [questions] Which jakartaEE version we plan for pulsar 3.0.0?

2023-02-27 Thread ZhangJian He
Sorry, what I want to talk is `pulsar-admin-client`, it uses jersey client, which depends on `jakarta.ws.rs-api` Thanks ZhangJian He On Mon, 27 Feb 2023 at 16:27, Enrico Olivelli wrote: > Are we talking about the Server side or the Pulsar Admin Client (that > uses HTTP) ? > On the server side

Re: [ANNOUNCE] New committer: Yuri Mizushima

2023-02-27 Thread Zhengxin Cai
Congrats! Yuri Zixuan Liu 于2023年2月27日周一 16:04写道: > Congrats! Yuri > > Best, > Zixuan > > Horizon <1060026...@qq.com.invalid> 于2023年2月27日周一 15:57写道: > > > Congrats! > > > > >

Re: [ANNOUNCE] New committer: Yuri Mizushima

2023-02-27 Thread ZhangJian He
Congratulations! Thanks ZhangJian He On Mon, 27 Feb 2023 at 17:02, Zhengxin Cai wrote: > Congrats! Yuri > > Zixuan Liu 于2023年2月27日周一 16:04写道: > > > Congrats! Yuri > > > > Best, > > Zixuan > > > > Horizon <1060026...@qq.com.invalid> 于2023年2月27日周一 15:57写道: > > > > > Congrats! > > > > > > > > >

Re: [DISCUSS] Using bouncycastle fips instead bouncycastle non-fips

2023-02-27 Thread Asaf Mesika
So if I understand you correctly, once you switch to the FIPS version of Bouncy Castle, users might get exceptions if they don't use specific algorithms or encryption schemes? Potentially a breaking change? You can't switch it off via config? On Wed, Feb 22, 2023 at 3:56 PM Zixuan Liu wrote: > >

Re: [ANNOUNCE] New committer: Yuri Mizushima

2023-02-27 Thread Zike Yang
Congratulations! Yuri BR, Zike Yang On Mon, Feb 27, 2023 at 5:15 PM ZhangJian He wrote: > > Congratulations! > > Thanks > ZhangJian He > > > On Mon, 27 Feb 2023 at 17:02, Zhengxin Cai wrote: > > > Congrats! Yuri > > > > Zixuan Liu 于2023年2月27日周一 16:04写道: > > > > > Congrats! Yuri > > > > > > Bes

Re: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema compatibility checks without using avro-protobuf

2023-02-27 Thread Asaf Mesika
> > And you can see the difference between ProtoBuf and ProtoBufNative: > > https://pulsar.apache.org/docs/next/schema-get-started/#protobufnative > > https://pulsar.apache.org/docs/next/schema-get-started/#protobuf > I read it and they look identical. What's the difference between them? Current

Re: [DISCUSS] Change PIP template

2023-02-27 Thread Asaf Mesika
On Sun, Feb 26, 2023 at 2:49 PM Girish Sharma wrote: > Good proposal Asaf. > I've also wondered why the PIP creation and discussion process is so > separated. The PIP discussion and voting starts off as a GitHub issue, but > all of its discussion happens here on the mailing list. Is there scope o

Re: [DISCUSS] Change PIP template

2023-02-27 Thread Asaf Mesika
On Sun, Feb 26, 2023 at 5:02 PM Dave Fisher wrote: > Excellent proposal. > > Inline below. > > Sent from my iPhone > > > On Feb 26, 2023, at 3:19 AM, Asaf Mesika wrote: > > > > Hi, > > > > I would like to suggest two changes I'd like to make to the PIP design > > template: > > 1. Remove the for

Re: [DISCUSS] Change PIP template

2023-02-27 Thread Asaf Mesika
Can someone direct me to the places I need to change exactly, so I won't make mistakes in the PR? On Mon, Feb 27, 2023 at 9:56 AM Enrico Olivelli wrote: > +1 great suggestions > > Enrico > > Il giorno lun 27 feb 2023 alle ore 02:54 ha > scritto: > > > > +1 > > > > Best, > > Mattison > > On Feb

Re: [DISCUSS] PIP-249: Pulsar website redesign

2023-02-27 Thread Enrico Olivelli
I am supportive of this idea about the website. I won't enter the details, but I have read the PIP and overall the idea seems good to me Enrico Il giorno ven 24 feb 2023 alle ore 17:37 Asaf Mesika ha scritto: > > You are correct - we did start with what content we wanted to show, then > designed

Re: [DISCUSS] Change PIP template

2023-02-27 Thread Girish Sharma
Hi Asaf, I was referring to the PIP process, as a whole, as explained in https://github.com/apache/pulsar/blob/master/wiki/proposals/PIP.md Someone looking at GitHub ticket would find and almost empty PIP GH issue while the same PIP has had many discussions over here in the ML. There is scope of i

Re: [DISCUSS] Change PIP template

2023-02-27 Thread Asaf Mesika
Mails don't support things like markdown diagrams or images and are generally less easy to read. My proposal includes a required section called Links in which you need to fill in the discussion thread in DEV mailing list and vote thread. On Mon, Feb 27, 2023 at 3:08 PM Girish Sharma wrote: > H

Re: [DISCUSS] PIP-246: Improved PROTOBUF_NATIVE schema compatibility checks without using avro-protobuf

2023-02-27 Thread SiNan Liu
> > I read it and they look identical. What's the difference between them? Current avro,json, and protobuf schemas are all implemented based on AVRO. > What do you mean, they are all implemented based on Avro? You mean the > protobuf schema is converted into an Avro Schema, and then you use Avro >

Re: [DISCUSS] Using bouncycastle fips instead bouncycastle non-fips

2023-02-27 Thread Zixuan Liu
> users might get exceptions if they don't use specific algorithms or encryption schemes? Could you share more info about this? Asaf Mesika 于2023年2月27日周一 18:01写道: > So if I understand you correctly, once you switch to the FIPS version of > Bouncy Castle, users might get exceptions if they don't

Re: [ANNOUNCE] New committer: Yuri Mizushima

2023-02-27 Thread Max Xu
Congratulations! Yuri Best, Max Xu On Mon, Feb 27, 2023 at 6:55 PM Zike Yang wrote: > Congratulations! Yuri > > BR, > Zike Yang > > On Mon, Feb 27, 2023 at 5:15 PM ZhangJian He wrote: > > > > Congratulations! > > > > Thanks > > ZhangJian He > > > > > > On Mon, 27 Feb 2023 at 17:02, Zhengxin C

Re: [ANNOUNCE] New committer: Yuri Mizushima

2023-02-27 Thread Xiangying Meng
Congratulations! Yuri Sincerely, Xiangying On Mon, Feb 27, 2023 at 10:36 PM Max Xu wrote: > Congratulations! Yuri > > Best, > Max Xu > > > On Mon, Feb 27, 2023 at 6:55 PM Zike Yang wrote: > > > Congratulations! Yuri > > > > BR, > > Zike Yang > > > > On Mon, Feb 27, 2023 at 5:15 PM ZhangJian He

Re: [ANNOUNCE] New committer: Yuri Mizushima

2023-02-27 Thread 太上玄元道君
congrats! Thanks, Tao Jiuming Nozomi Kurihara 于2023年2月27日周一 15:47写道: > Hi everyone, > > The Project Management Committee (PMC) for Apache Pulsar > has invited Yuri Mizushima (https://github.com/equanz) to become a > committer and we are pleased > to announce that he has accepted. > > Yuri has b

Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread Enrico Olivelli
Hello Committers, I believe that we should stop cherry-picking breaking changes like [1] to released branches. Really, this is something that we cannot do. When you decide to cherry-pick a commit to a "stable branch", currently branch-2.8, branch-2.9, branch-2.10 and branch-2.11 you always have to

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread mattisonchao
+1000, We should pay attention on two places: 1. Why add release/xxx label to the breaking change PR ? 2. We should recheck the context when committer cherry-pick PRs. Best, Mattison On Feb 27, 2023, 23:27 +0800, Enrico Olivelli , wrote: > Hello Committers, > I believe that we should stop cherr

Re: [ANNOUNCE] New committer: Yuri Mizushima

2023-02-27 Thread Christophe Bornet
Congrats Yuri! Le lun. 27 févr. 2023 à 15:54, 太上玄元道君 a écrit : > congrats! > > Thanks, > Tao Jiuming > > Nozomi Kurihara 于2023年2月27日周一 15:47写道: > > > Hi everyone, > > > > The Project Management Committee (PMC) for Apache Pulsar > > has invited Yuri Mizushima (https://github.com/equanz) to becom

[DISCUSS] Support processingGuarantees "EFFECTIVELY_ONCE" in python function

2023-02-27 Thread laminar
Hi all, I would like to discuss supporting the processingGuarantess `EFFECTIVELY_ONCE` in python function runtime. In this PR's(https://github.com/apache/pulsar/pull/18929) discussion, we conclude that to achieve the exactly processing guarantees for `EFFECTIVELY_ONCE`, the user needs to ensur

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread Niclas Hedhman
Linus Thorvalds has spoken about this more than once, and his position is that the ultimate failure of a project is to worsen the user experience, and changing behavior is "worse experience". He claim to go as far as revert bug fixes if there are people depending on the buggy behavior and ca

Re: [ANNOUNCE] New committer: Yuri Mizushima

2023-02-27 Thread Baodi Shi
Congratulations! Yuri Thanks, Baodi Shi 在 2023年2月28日 05:04:58 上,Christophe Bornet 写道: > Congrats Yuri! > > Le lun. 27 févr. 2023 à 15:54, 太上玄元道君 a écrit : > > congrats! > > > Thanks, > > Tao Jiuming > > > Nozomi Kurihara 于2023年2月27日周一 15:47写道: > > > > Hi everyone, > > > > > > The Project Ma

[DISCUSS] Support custom compressionType for pulsar functions

2023-02-27 Thread Pengcheng Jiang
Hello, community: ### Motivation Currently, pulsar functions are using `LZ4` as the compression type, and users cannot change it, yet some users may want to custom this behavior. ### Modifications Add a `CompressionType` field(which is an enum) to the `ProducerSpec` in the `Function.proto`; thi

Re: [ANNOUNCE] New committer: Yuri Mizushima

2023-02-27 Thread Yubiao Feng
Congratulations! Thanks Yubiao Feng On Mon, Feb 27, 2023 at 3:47 PM Nozomi Kurihara wrote: > Hi everyone, > > The Project Management Committee (PMC) for Apache Pulsar > has invited Yuri Mizushima (https://github.com/equanz) to become a > committer and we are pleased > to announce that he has acc

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread Yubiao Feng
Hi Enrico Olivelli Thank you for helping me correct my mistake Yubiao Feng On Mon, Feb 27, 2023 at 11:27 PM Enrico Olivelli wrote: > Hello Committers, > I believe that we should stop cherry-picking breaking changes like [1] > to released branches. > Really, this is something that we cannot do

[DISCUSS] Why not split `memoryLimit` into `consumerMemoryLimit ` and `producerMemoryLimit `

2023-02-27 Thread Jiaqi Shen
Context. - PIP-74 https://github.com/apache/pulsar/wiki/PIP-74%3A-Pulsar-client-memory-limits - PR-8965 https://github.com/apache/pulsar/pull/8965 - PR-15216 https://github.com/apache/pulsar/pull/15216 Hello, community: There are some questions about PIP-74 I want to figure out. PIP-74 and its i

Re: [ANNOUNCE] New committer: Yuri Mizushima

2023-02-27 Thread houxiaoyu
Congrats ! Yubiao Feng 于2023年2月28日周二 09:49写道: > Congratulations! > Thanks > Yubiao Feng > > On Mon, Feb 27, 2023 at 3:47 PM Nozomi Kurihara > wrote: > > > Hi everyone, > > > > The Project Management Committee (PMC) for Apache Pulsar > > has invited Yuri Mizushima (https://github.com/equanz) to

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread Xiangying Meng
Hi Enrico Olivelli, Totally agree, we should be careful when cherry-picking PRs. And we can't trust our own judgment too much. For an uncertain PR, we must submit a PR and wait for everyone to review it together. For example, for the PR [1] mentioned above, the measure I took was to push a PR to c

Re: [VOTE] Pulsar Client Python Release 3.1.0 Candidate 3

2023-02-27 Thread PengHui Li
+1 (binding) - install on macos - start a standalone (latest master) - run the example https://github.com/apache/pulsar-client-python#running-examples Regards, Penghui On Thu, Feb 23, 2023 at 11:00 PM Enrico Olivelli wrote: > Thank you Yunze for double checking. > > I don't have time to test t

Re: [Discuss] PIP-248: Add backlog eviction metric

2023-02-27 Thread PengHui Li
Maybe we can use `pulsar_subscription_backlog_quota_percentage`? Add _subscription here because this metric is only for subscriptions. Aggregated by topic/namespace level is meaningless. We should also mention the new metrics only be exposed if the subscription level metrics is enabled. And we sho

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread PengHui Li
Hi enrico, +1 for your point. Do you know the details of the breaking change? I can't find any discussions under the mailing list about the breaking change. I have added the `release/important-notice ` label to the PR, and we should also discuss first, better to have a proposal if we are making

Re: [DISCUSS]Add an internal class to `TransactionBufferStats` to record the snapshot status uniformly.

2023-02-27 Thread PengHui Li
Hi Xiangying, If the change applies to users, we should start with a proposal to provide the background, the goal and the related API changs. Thanks, Penghui On Sun, Feb 26, 2023 at 11:38 PM Asaf Mesika wrote: > On Fri, Feb 24, 2023 at 5:33 AM Xiangying Meng > wrote: > > > Hi Asaf, > > > How

[DISCUSS] The GET_TOPICS_OF_NAMESPACE command should filter out system topics

2023-02-27 Thread Baodi Shi
Hi, In v2.11.0, the system topic enables by default [1]. This causes the `GET_TOPICS_OF_NAMESPACE ` command to redundantly return the system topic. This causes some incompatibility issues: If consumers use pattern subscribe and the pattern is `/tenant/namespace/.*`, it will subscribe `__change_ev

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread tison
> The release manager is unable to review all PRs before releasing it. At least the RM is responsible for PRs cherry-picked he/she made. As we take compatibility in a high priority, if it's unclear a fix (patch) without breaking changes, the RM can ask for confirmation. Best, tison. PengHui Li

Re: [DISCUSS] Change PIP template

2023-02-27 Thread PengHui Li
+1 Penghui On Mon, Feb 27, 2023 at 9:24 PM Asaf Mesika wrote: > Mails don't support things like markdown diagrams or images and are > generally less easy to read. > My proposal includes a required section called Links in which you need to > fill in the discussion thread in DEV mailing list and

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread tison
For "what is breaking changes", at least the PULL_REQUEST_TEMPLATE gives some hints: https://github.com/apache/pulsar/blob/bbb543d8ed2b03361807c852da1a31cfb92939f3/.github/PULL_REQUEST_TEMPLATE.md?plain=1#L56-L72 For the issued PR, Hang commented at https://github.com/apache/pulsar/pull/15914#issu

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread tison
I agree that the `release/*` tags are misleading hints. Remove all the tags for the issued PR. Best, tison. tison 于2023年2月28日周二 13:13写道: > For "what is breaking changes", at least the PULL_REQUEST_TEMPLATE gives > some hints: > https://github.com/apache/pulsar/blob/bbb543d8ed2b03361807c852da1a

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread PengHui Li
> At least the RM is responsible for PRs cherry-picked he/she made. As we take compatibility in a high priority, if it's unclear a fix (patch) without breaking changes, the RM can ask for confirmation. Yes, but it's not easy without going through all the comments, review the PR changes. If we don'

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread Dave Fisher
Sent from my iPhone > On Feb 27, 2023, at 9:08 PM, tison wrote: > >  >> >> The release manager is unable to review all PRs before releasing it. > > At least the RM is responsible for PRs cherry-picked he/she made. As we > take compatibility in a high priority, if it's unclear a fix (patch)

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread PengHui Li
> For the issued PR, Hang commented at https://github.com/apache/pulsar/pull/15914#issuecomment-1146942281 that it changes the default behavior. Yes, but it looks like it has been ignored by the reviewers and the author. The PR still gets merged, finally. We should give a `Change Request` if the P

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread tison
> Yes, but.. For this case, I agree that the RM trust Jason who tagged the PR needs to be picked to 2.10. In this case Jason was supposed to do the check. > guidance for release managers to evaluate PR cherry-pick labels carefully For the current workflow, the RM can trust the labels since only

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread Dave Fisher
Sent from my iPhone > On Feb 27, 2023, at 9:28 PM, tison wrote: > >  >> >> Yes, but.. > > For this case, I agree that the RM trust Jason who tagged the PR needs to > be picked to 2.10. In this case Jason was supposed to do the check. > >> guidance for release managers to evaluate PR cherr

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread Niclas Hedhman
Enrico, I find it interesting that the main reaction is to focus on the symptom, rather than the disease. From my experience, the open source project should drop the support of multiple release branches (especially since they are claimed to be backward compatible), only have one released ve

Re: Please stop cherry-picking (breaking) changes to the released branches

2023-02-27 Thread PengHui Li
I found another example https://github.com/apache/pulsar/pull/19425 Which has cherry-picked and then reverted If the PR's author knows the PR should be cherry-pick to some branches, it's better to contains this part in the PR description and explain why the fix should be cherry-picked. The reviewe

Re: [Discuss] PIP-248: Add backlog eviction metric

2023-02-27 Thread 太上玄元道君
Hi Asaf, I've updated the PIP, PTAL Thank, Tao Jiuming Asaf Mesika 于2023年2月26日周日 23:03写道: > Hi, > > Pulsar has 2 configurations for the backlog eviction: > > backlogQuotaDefaultLimitBytes and backlogQuotaDefaultLimitSecond, if > > topic backlog reaches the threshold of any item, backlog evictio

Re: [Discuss] PIP-248: Add backlog eviction metric

2023-02-27 Thread 太上玄元道君
> Maybe we can use `pulsar_subscription_backlog_quota_percentage`? It makes sense, Thanks > Add _subscription here because this metric is only for subscriptions. > Aggregated by topic/namespace level is meaningless. We should also mention > the new metrics only be exposed if the subscription