Hi all, Since there are no objections on this PIP and this vote has been staying open for 48 hours. Close the vote with 3 (+1) bindings and 2 (+1) non-bindings. Thanks for your participation.
Thanks, Zike On Thu, Jan 27, 2022 at 10:09 AM Zike Yang <zky...@streamnative.io> wrote: > > Hi Michael, > > Sorry, I forgot about it. Sorry for my mistake. > Thanks for your correction. I will stay this vote open. > > Thanks, > Zike > > > > > > On Thu, Jan 27, 2022 at 12:31 AM Michael Marshall <mmarsh...@apache.org> > wrote: > > > > Hi Zike, > > > > Based on our current voting rules, I believe we need to leave the vote > > open for at least 17 more hours. > > > > There has been discussion on the dev list about changing this rule, > > but I don't believe we have reached any conclusions yet. > > > > Thanks, > > Michael > > > > On Wed, Jan 26, 2022 at 6:04 AM Zike Yang > > <zky...@streamnative.io.invalid> wrote: > > > > > > Thanks for your participation and your great efforts to move this forward! > > > Close the vote with 3 (+1) bindings and 2 (+1) non-bindings. > > > > > > Thanks, > > > Zike > > > > > > On Wed, Jan 26, 2022 at 3:54 PM PengHui Li <peng...@apache.org> wrote: > > > > > > > > +1 (binding) > > > > > > > > Regards, > > > > Penghui > > > > > > > > On Wed, Jan 26, 2022 at 12:17 PM Michael Marshall <mmarsh...@apache.org> > > > > wrote: > > > > > > > > > +1 (non binding) - this proposal looks great! Thank you for a good > > > > > discussion of this feature! > > > > > > > > > > Thanks, > > > > > Michael > > > > > > > > > > On Tue, Jan 25, 2022 at 8:20 PM Hang Chen <chenh...@apache.org> wrote: > > > > > > > > > > > > +1 (binding) > > > > > > > > > > > > Thanks, > > > > > > Hang > > > > > > > > > > > > Jia Zhai <zhai...@apache.org> 于2022年1月26日周三 10:17写道： > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jan 25, 2022 at 8:28 PM mattison chao > > > > > > > <mattisonc...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > > > > > > On Jan 25, 2022, at 5:34 PM, Zike Yang <zky...@streamnative.io > > > > > .INVALID> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > Hi Pulsar Community, > > > > > > > > > > > > > > > > > > This is the voting thread for PIP-124. It will stay open for > > > > > > > > > at > > > > > least > > > > > > > > > 48 hours. Pasted below for quoting convenience. > > > > > > > > > > > > > > > > > > Here is the issue for this PIP: > > > > > > > > https://github.com/apache/pulsar/issues/13408 > > > > > > > > > > > > > > > > > > ---- > > > > > > > > > > > > > > > > > > ## Motivation > > > > > > > > > > > > > > > > > > If we enable the DLQ when consuming messages. For some > > > > > > > > > messages > > > > > that > > > > > > > > > can't be processed successfully, the messages will be moved > > > > > > > > > to the > > > > > > > > > DLQ, but if we do not specify the data retention for the > > > > > > > > > namespace > > > > > or > > > > > > > > > create a subscription for the DLQ to retain the data, the > > > > > > > > > data of > > > > > the > > > > > > > > > DLQ will be removed automatically. Therefore, we need to > > > > > > > > > create the > > > > > > > > > initial subscription before sending messages to the DLQ. > > > > > > > > > > > > > > > > > > ## Goal > > > > > > > > > > > > > > > > > > Users can set the initial subscription name in the > > > > > DeadLetterPolicy. > > > > > > > > > The consumer will create the initial subscription before > > > > > > > > > sending > > > > > > > > > messages to the DLQ. At this point, subsequent messages > > > > > > > > > produced to > > > > > > > > > the DLQ are not automatically deleted unexpectedly. If > > > > > > > > > `allowAutoSubscriptionCreation` in `broker.conf` is false, the > > > > > initial > > > > > > > > > subscription won't be created automatically. Otherwise, it > > > > > > > > > will > > > > > > > > > confuse the user. Users can explicitly create that > > > > > > > > > subscription to > > > > > > > > > handle this case. > > > > > > > > > > > > > > > > > > This PIP needs to be compatible with the previous behavior. > > > > > > > > > The > > > > > > > > > initial subscription name in the DeadLetterPolicy is > > > > > > > > > optional. The > > > > > > > > > default behavior is not to create the initial subscription > > > > > > > > > which is > > > > > > > > > consistent with the original behavior. > > > > > > > > > > > > > > > > > > ## API Changes > > > > > > > > > > > > > > > > > > * Add `initSubscriptionName` to the `DeadLetterPolicy` > > > > > > > > > > > > > > > > > > ```java > > > > > > > > > /** > > > > > > > > > * Name of the initial subscription name of the dead letter > > > > > > > > > topic. > > > > > > > > > * If this field is not set, the initial subscription will not > > > > > > > > > be > > > > > created. > > > > > > > > > */ > > > > > > > > > private String initSubscriptionName; > > > > > > > > > ``` > > > > > > > > > > > > > > > > > > * Add a new field to the `CommandProducer`. The broker will > > > > > > > > > use > > > > > this > > > > > > > > > field to create the initial subscription. > > > > > > > > > > > > > > > > > > ```proto > > > > > > > > > optional string initial_subscription_name > > > > > > > > > ``` > > > > > > > > > > > > > > > > > > * Add a new config to the Producer Configuration. This allows > > > > > > > > > the > > > > > > > > > consumer to specify the initial subscription when creating the > > > > > > > > > deadLetterProducer. > > > > > > > > > > > > > > > > > > ```java > > > > > > > > > /** > > > > > > > > > * Use this config to automatically create an initial > > > > > > > > > subscription > > > > > > > > > when creating the topic. > > > > > > > > > * If this field is not set, the initial subscription will not > > > > > > > > > be > > > > > created. > > > > > > > > > * > > > > > > > > > * @param initialSubscriptionName > > > > > > > > > * Name of the initial subscription of the topic. > > > > > > > > > * @return the producer builder instance > > > > > > > > > */ > > > > > > > > > ProducerBuilder<T> initialSubscriptionName(String > > > > > > > > initialSubscriptionName); > > > > > > > > > ``` > > > > > > > > > > > > > > > > > > > > > > > > > > > ## Implementation > > > > > > > > > > > > > > > > > > When the deadLetterProducer is initialized, the consumer will > > > > > > > > > set > > > > > the > > > > > > > > > initial subscription of the deadLetterProducer according to > > > > > > > > > the > > > > > > > > > DeadLetterPolicy. > > > > > > > > > > > > > > > > > > When the broker creates a producer(after receiving the > > > > > > > > > CommandProducer), if it finds that the topic does not exist, > > > > > > > > > it > > > > > will > > > > > > > > > not only create the topic(if the topic automatically creation > > > > > > > > > is > > > > > > > > > enabled) but also create the initial subscription(if the > > > > > > > > > initial > > > > > > > > > subscription name is set). When creating an initial > > > > > > > > > subscription, > > > > > the > > > > > > > > > user needs to have the `canConsume` permission and the > > > > > > > > > `allowAutoSubscriptionCreation` needs to be enabled. > > > > > > > > > > > > > > > > > > ## Reject Alternatives > > > > > > > > > > > > > > > > > > ### Create the initial subscription using the consumer > > > > > > > > > > > > > > > > > > Before the deadLetterProducer is initialized, the consumer > > > > > > > > > first > > > > > tries > > > > > > > > > to create a deadLetterConsumer using the initial subscription > > > > > > > > > name > > > > > in > > > > > > > > > the DeadLetterPolicy. When this subscription already exists, > > > > > > > > > the > > > > > > > > > ConsumerBusy exception will occur. In this case, we can > > > > > > > > > ignore that > > > > > > > > > exception and create the deadLetterProducer. > > > > > > > > > > > > > > > > > > This is the original solution for this PIP. But this > > > > > > > > > introduces too > > > > > > > > > much workload for the client. In the subsequent discussion, > > > > > > > > > we try > > > > > to > > > > > > > > > add a new command `CommandCreateSubscription` to avoid > > > > > > > > > creating and > > > > > > > > > closing the deadLetterConsumer, but again, this introduces > > > > > workload as > > > > > > > > > well as greater complexity. > > > > > > > > > > > > > > > > > > ### Use the retention policy to retent the data of the DLQ > > > > > > > > > > > > > > > > > > Before creating the deadLetterProducer, an admin can create a > > > > > > > > > retention policy for the topic or namespace. Then, consumers > > > > > > > > > of the > > > > > > > > > topic have the duration of the retention policy to discover > > > > > > > > > the > > > > > topic > > > > > > > > > and create a subscription before messages are lost. > > > > > > > > > > > > > > > > > > But users are not easy to set a different data retention > > > > > > > > > policy or > > > > > > > > > create a new subscription for a lazy created DLQ topic. > > > > > > > > > > > > > > > > > > ### Add a new policy to retain data when the topic has no > > > > > subscriptions > > > > > > > > > > > > > > > > > > Adding a new policy for pulsar topics: a namespace or a topic > > > > > > > > > level > > > > > > > > > policy that makes it possible to retain messages indefinitely > > > > > > > > > when > > > > > a > > > > > > > > > topic has no subscriptions. > > > > > > > > > > > > > > > > > > For the auto-created topic, the subscription can only be > > > > > determined at > > > > > > > > > the time of creation. It may or may not create. If users are > > > > > > > > > able > > > > > to > > > > > > > > > determine which consumers are, and these consumers need to > > > > > > > > > receive > > > > > any > > > > > > > > > message sent by the producer, they should create the topic and > > > > > > > > > subscription manually or use the consumer to trigger the topic > > > > > > > > > auto-creation, not the producer. > > > > > > > > > > > > > > > > > > ### Add the initial subscription name field to the metadata > > > > > > > > > of the > > > > > > > > > CommandProducer > > > > > > > > > > > > > > > > > > Use producer metadata to carry the init subscription name so > > > > > > > > > that > > > > > we > > > > > > > > > don't need to introduce a new field in CommandProducer, and > > > > > > > > > the new > > > > > > > > > field looks a little confusing. > > > > > > > > > > > > > > > > > > While this does not change protobuf, it still implicitly > > > > > > > > > changes > > > > > the > > > > > > > > > protocol. This solution has different advantages and > > > > > > > > > disadvantages > > > > > > > > > than the current one. Finally, we decided to go with the > > > > > > > > > current > > > > > > > > > solution. > > > > > > > > > > > > > > > > > > In addition, there is a discussion about whether we need to > > > > > > > > > create > > > > > the > > > > > > > > > initial subscription on the retryLetterTopic. The consumer has > > > > > already > > > > > > > > > subscribed to the retryLetterTopic and created the > > > > > > > > > subscription > > > > > before > > > > > > > > > creating the retryLetterProducer, so we don't need to create > > > > > > > > > the > > > > > > > > > initial subscription for it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Prototype implementation PR: > > > > > https://github.com/apache/pulsar/pull/13355 > > > > > > > > > > > > > > > > > > ---- > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > Zike Yang > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Zike Yang > > > > -- > Zike Yang -- Zike Yang
