+1 (non-binding)
> On Jan 25, 2022, at 5:34 PM, Zike Yang <zky...@streamnative.io.INVALID> wrote:
>
> Hi Pulsar Community,
>
> This is the voting thread for PIP-124. It will stay open for at least
> 48 hours. Pasted below for quoting convenience.
>
> Here is the issue for this PIP: https://github.com/apache/pulsar/issues/13408
>
> ----
>
> ## Motivation
>
> If we enable the DLQ when consuming messages. For some messages that
> can't be processed successfully, the messages will be moved to the
> DLQ, but if we do not specify the data retention for the namespace or
> create a subscription for the DLQ to retain the data, the data of the
> DLQ will be removed automatically. Therefore, we need to create the
> initial subscription before sending messages to the DLQ.
>
> ## Goal
>
> Users can set the initial subscription name in the DeadLetterPolicy.
> The consumer will create the initial subscription before sending
> messages to the DLQ. At this point, subsequent messages produced to
> the DLQ are not automatically deleted unexpectedly. If
> `allowAutoSubscriptionCreation` in `broker.conf` is false, the initial
> subscription won't be created automatically. Otherwise, it will
> confuse the user. Users can explicitly create that subscription to
> handle this case.
>
> This PIP needs to be compatible with the previous behavior. The
> initial subscription name in the DeadLetterPolicy is optional. The
> default behavior is not to create the initial subscription which is
> consistent with the original behavior.
>
> ## API Changes
>
> * Add `initSubscriptionName` to the `DeadLetterPolicy`
>
> ```java
> /**
> * Name of the initial subscription name of the dead letter topic.
> * If this field is not set, the initial subscription will not be created.
> */
> private String initSubscriptionName;
> ```
>
> * Add a new field to the `CommandProducer`. The broker will use this
> field to create the initial subscription.
>
> ```proto
> optional string initial_subscription_name
> ```
>
> * Add a new config to the Producer Configuration. This allows the
> consumer to specify the initial subscription when creating the
> deadLetterProducer.
>
> ```java
> /**
> * Use this config to automatically create an initial subscription
> when creating the topic.
> * If this field is not set, the initial subscription will not be created.
> *
> * @param initialSubscriptionName
> * Name of the initial subscription of the topic.
> * @return the producer builder instance
> */
> ProducerBuilder<T> initialSubscriptionName(String initialSubscriptionName);
> ```
>
>
> ## Implementation
>
> When the deadLetterProducer is initialized, the consumer will set the
> initial subscription of the deadLetterProducer according to the
> DeadLetterPolicy.
>
> When the broker creates a producer(after receiving the
> CommandProducer), if it finds that the topic does not exist, it will
> not only create the topic(if the topic automatically creation is
> enabled) but also create the initial subscription(if the initial
> subscription name is set). When creating an initial subscription, the
> user needs to have the `canConsume` permission and the
> `allowAutoSubscriptionCreation` needs to be enabled.
>
> ## Reject Alternatives
>
> ### Create the initial subscription using the consumer
>
> Before the deadLetterProducer is initialized, the consumer first tries
> to create a deadLetterConsumer using the initial subscription name in
> the DeadLetterPolicy. When this subscription already exists, the
> ConsumerBusy exception will occur. In this case, we can ignore that
> exception and create the deadLetterProducer.
>
> This is the original solution for this PIP. But this introduces too
> much workload for the client. In the subsequent discussion, we try to
> add a new command `CommandCreateSubscription` to avoid creating and
> closing the deadLetterConsumer, but again, this introduces workload as
> well as greater complexity.
>
> ### Use the retention policy to retent the data of the DLQ
>
> Before creating the deadLetterProducer, an admin can create a
> retention policy for the topic or namespace. Then, consumers of the
> topic have the duration of the retention policy to discover the topic
> and create a subscription before messages are lost.
>
> But users are not easy to set a different data retention policy or
> create a new subscription for a lazy created DLQ topic.
>
> ### Add a new policy to retain data when the topic has no subscriptions
>
> Adding a new policy for pulsar topics: a namespace or a topic level
> policy that makes it possible to retain messages indefinitely when a
> topic has no subscriptions.
>
> For the auto-created topic, the subscription can only be determined at
> the time of creation. It may or may not create. If users are able to
> determine which consumers are, and these consumers need to receive any
> message sent by the producer, they should create the topic and
> subscription manually or use the consumer to trigger the topic
> auto-creation, not the producer.
>
> ### Add the initial subscription name field to the metadata of the
> CommandProducer
>
> Use producer metadata to carry the init subscription name so that we
> don't need to introduce a new field in CommandProducer, and the new
> field looks a little confusing.
>
> While this does not change protobuf, it still implicitly changes the
> protocol. This solution has different advantages and disadvantages
> than the current one. Finally, we decided to go with the current
> solution.
>
> In addition, there is a discussion about whether we need to create the
> initial subscription on the retryLetterTopic. The consumer has already
> subscribed to the retryLetterTopic and created the subscription before
> creating the retryLetterProducer, so we don't need to create the
> initial subscription for it.
>
>
>
> Prototype implementation PR: https://github.com/apache/pulsar/pull/13355
>
> ----
>
> Thanks
> Zike Yang
