[VOTE] PIP-348: Trigger offload on topic load stage

2024-05-06 Thread Hang Chen
Hi guys, I want to start voting for PIP-348. PIP: https://github.com/apache/pulsar/pull/22650 PR: https://github.com/apache/pulsar/pull/22652 DISCUSSION Thread: https://lists.apache.org/thread/2ndomp8v4wkcykzthhlyjqfmswor88kv Thanks, Hang

[VOTE] PIP-349: Add systemCursorNames ignore list for TTL check

2024-05-06 Thread Hang Chen
Hi guys, I want to start voting for PIP-349. PIP: https://github.com/apache/pulsar/pull/22651 PR: https://github.com/apache/pulsar/pull/22614 DISCUSSION Thread: https://lists.apache.org/thread/xgcworz4j8rjlqwr476s7sqn9do43f1t Thanks, Hang

Re: [DISCUSS] PIP-348: Trigger offload on topic load stage

2024-05-06 Thread Hang Chen
Hi guys, If there are no objections, I will send out the vote. Best, Hang Hang Chen 于2024年5月6日周一 22:23写道: > > >I see, but when a broker loads a bundle with many topics, you trigger > immediately offloading for them, even if they don't have much traffic. > This is kind of a "bad surprise" wh

Re: [DISCUSS] PIP-349: Add systemCursorNames ignore list for TTL check

2024-05-06 Thread Hang Chen
Hi guys, If there are no objections, I will send out the vote. Best, Hang Hang Chen 于2024年5月7日周二 11:17写道: > > >It seems that we only need to keep `additionalSystemCursorNames`. > > Yes, we just use the `additionalSystemCursorNames` > > Best, > Hang > > Zixuan Liu 于2024年5月6日周一 23:52写道: > >

Re: [DISCUSS] PIP-349: Add systemCursorNames ignore list for TTL check

2024-05-06 Thread Hang Chen
>It seems that we only need to keep `additionalSystemCursorNames`. Yes, we just use the `additionalSystemCursorNames` Best, Hang Zixuan Liu 于2024年5月6日周一 23:52写道: > > It seems that we only need to keep `additionalSystemCursorNames`. > > `systemCursorNames` is unnecessary, which should be built-i

Re: [DISCUSS] PIP-349: Add systemCursorNames ignore list for TTL check

2024-05-06 Thread Zixuan Liu
It seems that we only need to keep `additionalSystemCursorNames`. `systemCursorNames` is unnecessary, which should be built-in and immutable. Thanks, Zixuan Hang Chen 于2024年5月6日周一 22:22写道: > >We can make this "configurable", but the well known names should be always > handled automatically > >

Re: [DISCUSS] PIP-348: Trigger offload on topic load stage

2024-05-06 Thread Hang Chen
>I see, but when a broker loads a bundle with many topics, you trigger immediately offloading for them, even if they don't have much traffic. This is kind of a "bad surprise" when you update a cluster to the newer version and you don't expect load to spike unexpectedly. My concerns is only about th

Re: [DISCUSS] PIP-349: Add systemCursorNames ignore list for TTL check

2024-05-06 Thread Hang Chen
>We can make this "configurable", but the well known names should be always handled automatically I changed the configuration name to `additionalSystemCursorNames`. I think the user should be defined as a Pulsar administrator. For cursors that need to be ignored by TTL should be owned by Pulsar in

Re: [DISCUSSION] Event time based compaction

2024-05-06 Thread Lari Hotari
Hi Marek, I think that this is a good proposal. Thanks for driving this change. Please go ahead and create a PIP to document this change and bring it to voting. You can see in https://github.com/apache/pulsar/tree/master/pip how proposals are documented in PIP documents. There's a template avail

Re: [DISCUSS] PIP-348: Trigger offload on topic load stage

2024-05-06 Thread Enrico Olivelli
Il giorno lun 6 mag 2024 alle ore 04:15 Hang Chen ha scritto: > Hi Enrico, > I agree with Jiuming, we do not need a flag to control this > behavior. We will call `maybeOffloadInBackground` method in the > managedledger init stage. This method only checks the offload policies > to decide if we

Re: [DISCUSS] PIP-349: Add systemCursorNames ignore list for TTL check

2024-05-06 Thread Enrico Olivelli
Il giorno lun 6 mag 2024 alle ore 09:07 PengHui Li ha scritto: > For the backlog quota check, I think we also need the same capacity. > > For configurable. I think the reason is users can have their plugins > running > with the broker which relies on the Pulsar topic. They don't want the > applic

Re: [DISCUSS] PIP-349: Add systemCursorNames ignore list for TTL check

2024-05-06 Thread PengHui Li
For the backlog quota check, I think we also need the same capacity. For configurable. I think the reason is users can have their plugins running with the broker which relies on the Pulsar topic. They don't want the application to change the TTL or backlog policies for some subscription/cursor. Bu