It makes sense
Thanks,
Tao Jiuming
> 2023年2月9日 15:44，Baodi Shi <ba...@apache.org> 写道：
>
> Agree, and we can add verification on the client side. When the topic is
> non-persistent and uses durable to subscribe, print warns logs:
> “non-persistent not support durable subscribe will use non-durable to
> subscribe.”
>
>
>
> Thanks,
> Baodi Shi
>
> 在 2023年2月9日 15:10:12 上，Jiuming Tao <jm...@streamnative.io.invalid> 写道：
>
>> Agree. But we need to take care of the compatibility. How do we handle
>>
>> this compatibility?
>>
>>
>> How about set `isDurable = false` when create `NonPersistentSubscription`?
>>
>> In `NonPersistentSubscription`, we can change the constructor from
>> ```
>> public NonPersistentSubscription(NonPersistentTopic topic, String
>> subscriptionName, boolean isDurable,
>> Map<String, String> properties) {
>> this.topic = topic;
>> this.topicName = topic.getName();
>> this.subName = subscriptionName;
>> this.fullName = MoreObjects.toStringHelper(this).add("topic",
>> topicName).add("name", subName).toString();
>> IS_FENCED_UPDATER.set(this, FALSE);
>> this.lastActive = System.currentTimeMillis();
>> this.isDurable = isDurable;
>> this.subscriptionProperties = properties != null
>> ? Collections.unmodifiableMap(properties) :
>> Collections.emptyMap();
>> }
>>
>> ```
>>
>> to:
>>
>> ```
>> public NonPersistentSubscription(NonPersistentTopic topic, String
>> subscriptionName,
>> Map<String, String> properties) {
>> this.topic = topic;
>> this.topicName = topic.getName();
>> this.subName = subscriptionName;
>> this.fullName = MoreObjects.toStringHelper(this).add("topic",
>> topicName).add("name", subName).toString();
>> IS_FENCED_UPDATER.set(this, FALSE);
>> this.lastActive = System.currentTimeMillis();
>> this.isDurable = false;
>> this.subscriptionProperties = properties != null
>> ? Collections.unmodifiableMap(properties) :
>> Collections.emptyMap();
>> }
>>
>> ```
>>
>> In the broker side, for non-persistent topic, there is no durable
>> subscriptions.
>> In the client side, users don’t need to change their code.
>>
>> Thanks,
>> Tao Jiuming
>>
>> 2023年2月9日 15:01，Zike Yang <z...@apache.org> 写道：
>>
>>
>>> It makes sense, for non-persistent topic, it should not create a durable
>> subscription.
>>
>>
>> Agree. But we need to take care of the compatibility. How do we handle
>>
>> this compatibility?
>>
>>
>> BR,
>>
>> Zike Yang
>>
>>
>> On Thu, Feb 9, 2023 at 2:43 PM Jiuming Tao
>>
>> <jm...@streamnative.io.invalid> wrote:
>>
>>>
>>
>>>> We can add validation to inform the user that a durable subscription on
>> a
>>
>>>> non-persistent topic is invalid.
>>
>>>
>>
>>> It makes sense, for non-persistent topic, it should not create a durable
>> subscription.
>>
>>>
>>
>>> Thanks,
>>
>>> Tao Jiuming
>>
>>>
>>
>>>> 2023年2月8日 23:06，Baodi Shi <ba...@apache.org> 写道：
>>
>>>>
>>
>>>>>
>>
>>>>> But it seems that currently all Durable subscriptions on
>>
>>>>>
>>
>>>> non-persistent topics are NonDurable, and I wonder if that confuses
>>
>>>>> users.
>>
>>>>>
>>
>>>>
>>
>>>> We can add validation to inform the user that a durable subscription on
>> a
>>
>>>> non-persistent topic is invalid.
>>
>>>>
>>
>>>>
>>
>>>> Thanks,
>>
>>>> Baodi Shi
>>
>>>>
>>
>>>>
>>
>>>> 在 2023年2月8日 10:48:32 上，Zike Yang <z...@apache.org> 写道：
>>
>>>>
>>
>>>>> I agree to remove all subscriptions when there're no consumers on the
>>
>>>>> non-persistent topic. The configuration
>>
>>>>> `subscriptionExpirationTimeMinutes ` should not take effect on the
>>
>>>>> non-persistent topic.
>>
>>>>>
>>
>>>>> But it seems that currently all Durable subscriptions on
>>
>>>>> non-persistent topics are NonDurable, and I wonder if that confuses
>>
>>>>> users.
>>
>>>>>
>>
>>>>>> But for `NonPersistentDispatcherMultipleConsumers`,
>>
>>>>> `NonPersistentStickyKeyDispatcherMultipleConsumers`, I’m not sure that
>>
>>>>> if it matters when we auto delete the subscriptions which has no
>>
>>>>> consumers connected.
>>
>>>>>
>>
>>>>> It doesn't matter IMO.
>>
>>>>>
>>
>>>>> Thanks,
>>
>>>>> Zike Yang
>>
>>>>>
>>
>>>>>
>>
>>>>>
>>
>>>>> Zike Yang
>>
>>>>>
>>
>>>>> On Tue, Feb 7, 2023 at 9:22 PM Jiuming Tao
>>
>>>>> <jm...@streamnative.io.invalid> wrote:
>>
>>>>>
>>
>>>>>
>>
>>>>>> I think we are talking about the durable subscription of the
>>
>>>>> non-persistent
>>
>>>>>
>>
>>>>>> topic, right?
>>
>>>>>
>>
>>>>>
>>
>>>>> Yes, non-durable subscriptions created by Reader, if the reader
>>
>>>>> disconnected, the subscription will be deleted automatically.
>>
>>>>>
>>
>>>>>
>>
>>>>>
>>
>>>>> For `NonPersistentDispatcher`, it has the following implementations:
>>
>>>>>
>>
>>>>> 1. NonPersistentDispatcherMultipleConsumers
>>
>>>>>
>>
>>>>> 2. NonPersistentDispatcherSingleActiveConsumer
>>
>>>>>
>>
>>>>> 3. NonPersistentStickyKeyDispatcherMultipleConsumers
>>
>>>>>
>>
>>>>>
>>
>>>>> For `NonPersistentDispatcherSingleActiveConsumer`, auto delete the
>>
>>>>> subscription when there isn’t consumer connected affect nothing.
>>
>>>>>
>>
>>>>>
>>
>>>>> But for `NonPersistentDispatcherMultipleConsumers`,
>>
>>>>> `NonPersistentStickyKeyDispatcherMultipleConsumers`, I’m not sure that
>> if
>>
>>>>> it matters when we auto delete the subscriptions which has no
>> consumers
>>
>>>>> connected.
>>
>>>>>
>>
>>>>>
>>
>>>>> If it doesn’t matter, I agree with you.
>>
>>>>>
>>
>>>>>
>>
>>>>> Auto delete the durable subscriptions when there aren’t consumers
>>
>>>>> connected to the non-persistent topic.
>>
>>>>>
>>
>>>>>
>>
>>>>>
>>
>>>>> Thanks,
>>
>>>>>
>>
>>>>> Tao Jiuming
>>
>>>>>
>>
>>>>>
>>
>>>>>
>>
>>>>>> 2023年2月7日 19:58，PengHui Li <peng...@apache.org <mailto:
>>
>>>>> peng...@apache.org>> 写道：
>>
>>>>>
>>
>>>>>>
>>
>>>>>
>>
>>>>>> I think we are talking about the durable subscription of the
>>
>>>>> non-persistent
>>
>>>>>
>>
>>>>>> topic, right? For the Reader API (non-durable subscription), the
>>
>>>>>
>>
>>>>>> subscription will be
>>
>>>>>
>>
>>>>>> removed after the reader disconnects. If not, it should be BUG.
>>
>>>>>
>>
>>>>>>
>>
>>>>>
>>
>>>>>> IMO, we should remove the subscription automatically if all the
>> consumers
>>
>>>>>
>>
>>>>>> are disconnected for the non-persistent topic. The consumers can only
>> get
>>
>>>>>
>>
>>>>>> the new incoming messages after they connect to the non-persistent
>> topic.
>>
>>>>>
>>
>>>>>> It looks like even if we leave the subscription there; it will
>>
>>>>>
>>
>>>>>> help nothing.
>>
>>>>>
>>
>>>>>> The behavior should be the same as creating a new subscription with
>> the
>>
>>>>>
>>
>>>>>> same subscription name.
>>
>>>>>
>>
>>>>>>
>>
>>>>>
>>
>>>>>> For subscription management, users are not able to disable the
>>
>>>>> subscription
>>
>>>>>
>>
>>>>>> auto-creation of the non-persistent topic? We don't have any metadata
>>
>>>>>
>>
>>>>>> persist
>>
>>>>>
>>
>>>>>> in the metadata store. After the broker restarts, we will lose
>>
>>>>> everything.
>>
>>>>>
>>
>>>>>>
>>
>>>>>
>>
>>>>>> So I think we don't need to introduce a new configuration. Instead, we
>>
>>>>>
>>
>>>>>> should
>>
>>>>>
>>
>>>>>> figure out if there is a strong reason to leave the durable
>> subscription
>>
>>>>>
>>
>>>>>> after
>>
>>>>>
>>
>>>>>> all the consumers are disconnected. If there is no strong reason, I
>> tend
>>
>>>>> to
>>
>>>>>
>>
>>>>>> fix
>>
>>>>>
>>
>>>>>> incorrect behavior directly.
>>
>>>>>
>>
>>>>>>
>>
>>>>>
>>
>>>>>> Best,
>>
>>>>>
>>
>>>>>> Penghui
>>
>>>>>
>>
>>>>>>
>>
>>>>>
>>
>>>>>>
>>
>>>>>
>>
>>>>>>
>>
>>>>>
>>
>>>>>>
>>
>>>>>
>>
>>>>>> On Tue, Feb 7, 2023 at 4:14 PM Jiuming Tao
>> <jm...@streamnative.io.invalid
>>
>>>>> <mailto:jm...@streamnative.io.invalid>>
>>
>>>>>
>>
>>>>>> wrote:
>>
>>>>>
>>
>>>>>>
>>
>>>>>
>>
>>>>>>> Hi,
>>
>>>>>
>>
>>>>>>>
>>
>>>>>
>>
>>>>>>> I’ve opened a new PIP to discuss:
>>
>>>>>
>>
>>>>>>>
>>
>>>>>
>>
>>>>>>> Currently, in Pulsar, we have a configuration named
>>
>>>>>
>>
>>>>>>> `subscriptionExpirationTimeMinutes` to manage the subscription
>>
>>>>> expiration
>>
>>>>>
>>
>>>>>>> of `PersistentTopic` and `NonPersistentTopic`.
>>
>>>>>
>>
>>>>>>>
>>
>>>>>
>>
>>>>>>> When we set a value which is greater than 0 to
>>
>>>>>
>>
>>>>>>> `subscriptionExpirationTimeMinutes`, it will affect both
>>
>>>>> `PersistentTopic`
>>
>>>>>
>>
>>>>>>> and `NonPersistentTopic`. Their inactive subscriptions will get
>> expired
>>
>>>>> and
>>
>>>>>
>>
>>>>>>> will clean automatically.
>>
>>>>>
>>
>>>>>>>
>>
>>>>>
>>
>>>>>>> For `NonPersistentTopic`, its subscriptions can be clean because we
>>
>>>>> don't
>>
>>>>>
>>
>>>>>>> guarantee its data integrity. But for `PersistentTopic`, if we clean
>> its
>>
>>>>>
>>
>>>>>>> subscriptions automatically may lead to data loss.
>>
>>>>>
>>
>>>>>>>
>>
>>>>>
>>
>>>>>>> However, their subscription expiration is managed by the same
>>
>>>>>
>>
>>>>>>> configuration(`subscriptionExpirationTimeMinutes`), we can't manage
>>
>>>>> their
>>
>>>>>
>>
>>>>>>> subscription expiration independently.
>>
>>>>>
>>
>>>>>>>
>>
>>>>>
>>
>>>>>>> So I want to introduce a new configuration named
>>
>>>>>
>>
>>>>>>> `nonPersistentSubscriptionExpirationTimeMinutes` to manage
>>
>>>>>
>>
>>>>>>> `NonPersistentTopic`'s subscription expiration.
>>
>>>>>
>>
>>>>>>>
>>
>>>>>
>>
>>>>>>>
>>
>>>>>
>>
>>>>>>> Please feel free to leave your comments, the PIP link:
>>
>>>>>
>>
>>>>>>> https://github.com/apache/pulsar/issues/19448 <
>>
>>>>> https://github.com/apache/pulsar/issues/19448> <
>>
>>>>>
>>
>>>>>>> https://github.com/apache/pulsar/issues/19448 <
>>
>>>>> https://github.com/apache/pulsar/issues/19448>>
>>
>>>>>
>>
>>>>>>>
>>
>>>>>
>>
>>>>>>>
>>
>>>>>
>>
>>>>>>> Thanks,
>>
>>>>>
>>
>>>>>>> Tao Jiuming
>>
>>>>>
>>
>>>>>
>>
>>>>>
>>
>>>
>>
>>
>>
