Fwd: [Discuss] Pulsar Release Notes Architecture

2022-01-11 Thread Yu
Hi Pulsarers, To move [PIP 112: Generate Release Notes Automatically] forward, we need to define the Pulsar Release Notes Architecture, which only includes major changes (important features/enhancements/bug fixes) in list form rather than a raw dump of PRs. Here [2] is the draft, feel free to com

Re: [DISCUSSION] PIP-124: Create init subscription before sending message to DLQ

2022-01-11 Thread Matteo Merli
> If we want to hold that the DLQ is not a normal topic, then I can see > why we would have a DLQ specific feature here. I think that, good or bad, the impression that users have that the DLQ is not a "normal" topic comes from 2 factors: 1. The experience with traditional messaging systems JMS an

Re: [DISCUSSION] PIP-124: Create init subscription before sending message to DLQ

2022-01-11 Thread Michael Marshall
> It looks like a feature that supports retaining data while no subscriptions. Yes, that is my proposed feature. How we handle messages on a topic with an empty set of subscriptions is a design decision. Note that when there are no subscriptions for a topic, the following two statements are both

Re: [VOTE] PIP-122: Change loadBalancer default loadSheddingStrategy to ThresholdShedder

2022-01-11 Thread Michael Marshall
+1 - assuming we ensure that the `ThresholdShedder` has unit test coverage. Thanks, Michael On Tue, Jan 11, 2022 at 9:53 PM r...@apache.org wrote: > > +1 (non-binding) > > -- > > Thanks > Xiaolong Ran > > Haiting Jiang 于2022年1月12日周三 09:52写道: > > > +1 > > > > On 2022/01/10 06:47:44 Hang Chen w

Re: [VOTE] PIP-122: Change loadBalancer default loadSheddingStrategy to ThresholdShedder

2022-01-11 Thread r...@apache.org
+1 (non-binding) -- Thanks Xiaolong Ran Haiting Jiang 于2022年1月12日周三 09:52写道: > +1 > > On 2022/01/10 06:47:44 Hang Chen wrote: > > This is the voting thread for PIP-122. It will stay open for at least 48 > > hours. > > > > https://github.com/apache/pulsar/issues/13340 > > > > Pasted below for q

Re: [VOTE] PIP-121: Pulsar cluster level auto failover on client side

2022-01-11 Thread r...@apache.org
+1 (non-binding) -- Thanks Xiaolong Ran Zike Yang 于2022年1月12日周三 09:58写道: > +1 (non-binding) > > On Wed, Jan 12, 2022 at 9:52 AM Haiting Jiang > wrote: > > > > +1 > > > > On 2022/01/12 00:09:26 Matteo Merli wrote: > > > +1 > > > -- > > > Matteo Merli > > > > > > > > > On Tue, Jan 11, 2022 at 1

Re: [VOTE] PIP-117: Change Pulsar standalone defaults

2022-01-11 Thread r...@apache.org
+1 (non-binding) -- Thanks Xiaolong Ran Neng Lu 于2022年1月12日周三 04:45写道: > +1 (non-binding) > > On Wed, Jan 5, 2022 at 7:19 AM Lan Liang wrote: > > > +1 > > > > > > > > > > > > > > Best Regards, > > Lan Liang > > On 12/23/2021 19:21,Haiting Jiang wrote: > > +1 > > > > Thanks, > > Haiting > > > >

Re: [VOTE] PIP-132: Include message header size when check maxMessageSize of non-batch message on the client side.

2022-01-11 Thread r...@apache.org
+1 (non-binding) -- Thanks Xiaolong Ran mattison chao 于2022年1月12日周三 10:15写道: > +1 (non-binding) > > On Wed, 12 Jan 2022 at 09:59, Zike Yang > wrote: > > > +1 (non-binding) > > > > On Wed, Jan 12, 2022 at 9:58 AM Haiting Jiang > > wrote: > > > > > > This is the voting thread for PIP-132. It w

[Discuss] Create new issues to SDKs in different languages

2022-01-11 Thread r...@apache.org
Hello everyone: At present, all our PIP and related function changes are mainly in the Java language, and all new functions will be merged into the Java SDK first, but for SDKs in other languages, this is completely a black box, they don't know what changes or optimizations have been made on the J

Re: [DISCUSSION] PIP-129: Introduce intermediate state for ledger deletion

2022-01-11 Thread Hang Chen
+1 Good Job. Looking forward to this feature. Best, Hang PengHui Li 于2022年1月12日周三 10:54写道: > > +1 It's a nice approach for making sure the ledger can be deleted correctly. > > Regards, > Penghui > > On Wed, Jan 12, 2022 at 10:23 AM Zhanpeng Wu > wrote: > > > https://github.com/apache/pulsar/iss

Re: [DISCUSSION] PIP-129: Introduce intermediate state for ledger deletion

2022-01-11 Thread PengHui Li
+1 It's a nice approach for making sure the ledger can be deleted correctly. Regards, Penghui On Wed, Jan 12, 2022 at 10:23 AM Zhanpeng Wu wrote: > https://github.com/apache/pulsar/issues/13526 > > > > ## Motivation > > Under the current ledger-trimming design in > `org.apache.bookkeeper.m

[DISCUSSION] PIP-129: Introduce intermediate state for ledger deletion

2022-01-11 Thread Zhanpeng Wu
https://github.com/apache/pulsar/issues/13526 ## Motivation Under the current ledger-trimming design in `org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl#internalTrimLedgers`, we need to collect those ledgers that need to be deleted first, and then perform the asynchronous deletion of t

Re: [DISCUSSION] PIP-135: Include MetadataStore backend for Etcd

2022-01-11 Thread 陳智弘
+1 Haiting Jiang 於 2022年1月12日 週三 09:50 寫道: > +1 > > On 2022/01/12 01:44:21 PengHui Li wrote: > > +1 > > > > Penghui > > > > On Wed, Jan 12, 2022 at 8:39 AM mattison chao > > wrote: > > > > > +1 > > > > > > On Wed, 12 Jan 2022 at 08:09, Matteo Merli wrote: > > > > > > > https://github.com/apach

Re: [VOTE] PIP-132: Include message header size when check maxMessageSize of non-batch message on the client side.

2022-01-11 Thread mattison chao
+1 (non-binding) On Wed, 12 Jan 2022 at 09:59, Zike Yang wrote: > +1 (non-binding) > > On Wed, Jan 12, 2022 at 9:58 AM Haiting Jiang > wrote: > > > > This is the voting thread for PIP-132. It will stay open for at least 48 > hours. > > > > https://github.com/apache/pulsar/issues/13591 > > > >

Re: [VOTE] PIP-132: Include message header size when check maxMessageSize of non-batch message on the client side.

2022-01-11 Thread Hang Chen
+1 (binding) Best, Hang Zike Yang 于2022年1月12日周三 09:59写道: > > +1 (non-binding) > > On Wed, Jan 12, 2022 at 9:58 AM Haiting Jiang wrote: > > > > This is the voting thread for PIP-132. It will stay open for at least 48 > > hours. > > > > https://github.com/apache/pulsar/issues/13591 > > > > Paste

Re: [VOTE] PIP-132: Include message header size when check maxMessageSize of non-batch message on the client side.

2022-01-11 Thread Zike Yang
+1 (non-binding) On Wed, Jan 12, 2022 at 9:58 AM Haiting Jiang wrote: > > This is the voting thread for PIP-132. It will stay open for at least 48 > hours. > > https://github.com/apache/pulsar/issues/13591 > > Pasted below for quoting convenience. > > > > ## Motivation > > Currently, Pulsar

Re: [VOTE] PIP-121: Pulsar cluster level auto failover on client side

2022-01-11 Thread Zike Yang
+1 (non-binding) On Wed, Jan 12, 2022 at 9:52 AM Haiting Jiang wrote: > > +1 > > On 2022/01/12 00:09:26 Matteo Merli wrote: > > +1 > > -- > > Matteo Merli > > > > > > On Tue, Jan 11, 2022 at 12:07 PM Neng Lu wrote: > > > > > > +1 (non-binding) > > > > > > On Mon, Jan 10, 2022 at 12:40 AM PengHu

[VOTE] PIP-132: Include message header size when check maxMessageSize of non-batch message on the client side.

2022-01-11 Thread Haiting Jiang
This is the voting thread for PIP-132. It will stay open for at least 48 hours. https://github.com/apache/pulsar/issues/13591 Pasted below for quoting convenience. ## Motivation Currently, Pulsar client (Java) only checks payload size for max message size validation. Client throws Timeo

Re: [VOTE] PIP-121: Pulsar cluster level auto failover on client side

2022-01-11 Thread mattison chao
+1 On Wed, 12 Jan 2022 at 09:52, Haiting Jiang wrote: > +1 > > On 2022/01/12 00:09:26 Matteo Merli wrote: > > +1 > > -- > > Matteo Merli > > > > > > On Tue, Jan 11, 2022 at 12:07 PM Neng Lu wrote: > > > > > > +1 (non-binding) > > > > > > On Mon, Jan 10, 2022 at 12:40 AM PengHui Li > wrote: >

Re: [VOTE] PIP-121: Pulsar cluster level auto failover on client side

2022-01-11 Thread Haiting Jiang
+1 On 2022/01/12 00:09:26 Matteo Merli wrote: > +1 > -- > Matteo Merli > > > On Tue, Jan 11, 2022 at 12:07 PM Neng Lu wrote: > > > > +1 (non-binding) > > > > On Mon, Jan 10, 2022 at 12:40 AM PengHui Li wrote: > > > > > +1 (binding) > > > > > > Penghui > > > > > > On Mon, Jan 10, 2022 at 4:38 P

Re: [VOTE] PIP-122: Change loadBalancer default loadSheddingStrategy to ThresholdShedder

2022-01-11 Thread Haiting Jiang
+1 On 2022/01/10 06:47:44 Hang Chen wrote: > This is the voting thread for PIP-122. It will stay open for at least 48 > hours. > > https://github.com/apache/pulsar/issues/13340 > > Pasted below for quoting convenience. > > > > ### Motivation > The ThresholdShedder load balance policy sinc

Re: [DISCUSSION] PIP-135: Include MetadataStore backend for Etcd

2022-01-11 Thread Haiting Jiang
+1 On 2022/01/12 01:44:21 PengHui Li wrote: > +1 > > Penghui > > On Wed, Jan 12, 2022 at 8:39 AM mattison chao > wrote: > > > +1 > > > > On Wed, 12 Jan 2022 at 08:09, Matteo Merli wrote: > > > > > https://github.com/apache/pulsar/issues/13717 > > > > > > - > > > > > > ## Motivation > > >

Re: [DISCUSSION] PIP-135: Include MetadataStore backend for Etcd

2022-01-11 Thread PengHui Li
+1 Penghui On Wed, Jan 12, 2022 at 8:39 AM mattison chao wrote: > +1 > > On Wed, 12 Jan 2022 at 08:09, Matteo Merli wrote: > > > https://github.com/apache/pulsar/issues/13717 > > > > - > > > > ## Motivation > > > > Since all the pieces that composed the proposal in PIP-45 were finally > >

Re: [DISCUSSION] PIP-135: Include MetadataStore backend for Etcd

2022-01-11 Thread mattison chao
+1 On Wed, 12 Jan 2022 at 08:09, Matteo Merli wrote: > https://github.com/apache/pulsar/issues/13717 > > - > > ## Motivation > > Since all the pieces that composed the proposal in PIP-45 were finally > merged > and are currently ready for 2.10 release, it is now possible to add other > metad

Re: [VOTE] PIP-121: Pulsar cluster level auto failover on client side

2022-01-11 Thread Matteo Merli
+1 -- Matteo Merli On Tue, Jan 11, 2022 at 12:07 PM Neng Lu wrote: > > +1 (non-binding) > > On Mon, Jan 10, 2022 at 12:40 AM PengHui Li wrote: > > > +1 (binding) > > > > Penghui > > > > On Mon, Jan 10, 2022 at 4:38 PM Enrico Olivelli > > wrote: > > > > > +1 (binding) > > > > > > Enrico > > > >

Re: [VOTE] PIP-122: Change loadBalancer default loadSheddingStrategy to ThresholdShedder

2022-01-11 Thread Matteo Merli
+1 -- Matteo Merli On Tue, Jan 11, 2022 at 12:05 PM Neng Lu wrote: > > +1 (non-binding) > > On Mon, Jan 10, 2022 at 12:39 AM Enrico Olivelli > wrote: > > > +1 (binding) > > > > Enrico > > > > Il giorno lun 10 gen 2022 alle ore 07:47 Hang Chen > > ha scritto: > > > > > > > > This is the voting

[DISCUSSION] PIP-135: Include MetadataStore backend for Etcd

2022-01-11 Thread Matteo Merli
https://github.com/apache/pulsar/issues/13717 - ## Motivation Since all the pieces that composed the proposal in PIP-45 were finally merged and are currently ready for 2.10 release, it is now possible to add other metadata backends that can be used to support a BookKeeper + Pulsar cluster.

Re: [DISCUSSION] PIP-133 Pulsar Functions Add API For Accessing Other Function States

2022-01-11 Thread Enrico Olivelli
Thank you for posting your PIP ! I am sharing some of Neng's concerns. We should define clearly how security works. Also, currently the function defines some "namespace" for the state storage, and we recently added support for custom state storage implementation. With this change each function wi

Re: [VOTE] PIP-117: Change Pulsar standalone defaults

2022-01-11 Thread Neng Lu
+1 (non-binding) On Wed, Jan 5, 2022 at 7:19 AM Lan Liang wrote: > +1 > > > > > > > Best Regards, > Lan Liang > On 12/23/2021 19:21,Haiting Jiang wrote: > +1 > > Thanks, > Haiting > > On 2021/12/23 05:35:03 Michael Marshall wrote: > +1 > > - Michael > > On Wed, Dec 22, 2021 at 6:18 PM Sijie Guo

Re: [DISCUSSION] PIP-133 Pulsar Functions Add API For Accessing Other Function States

2022-01-11 Thread Neng Lu
Before we advance further, we first need to get on the same page of the pros and cons of allowing this feature. If functions can access (especially the write access) other functions' state, the data ownership will be a mess, isolation is broken and data security might be compromised. On Wed,

Re: [VOTE] PIP-121: Pulsar cluster level auto failover on client side

2022-01-11 Thread Neng Lu
+1 (non-binding) On Mon, Jan 10, 2022 at 12:40 AM PengHui Li wrote: > +1 (binding) > > Penghui > > On Mon, Jan 10, 2022 at 4:38 PM Enrico Olivelli > wrote: > > > +1 (binding) > > > > Enrico > > > > Il giorno lun 10 gen 2022 alle ore 07:45 Hang Chen > > ha scritto: > > > > > > This is the votin

Re: [VOTE] PIP-122: Change loadBalancer default loadSheddingStrategy to ThresholdShedder

2022-01-11 Thread Neng Lu
+1 (non-binding) On Mon, Jan 10, 2022 at 12:39 AM Enrico Olivelli wrote: > +1 (binding) > > Enrico > > Il giorno lun 10 gen 2022 alle ore 07:47 Hang Chen > ha scritto: > > > > > This is the voting thread for PIP-122. It will stay open for at least 48 > > hours. > > > > https://github.com/apache