Hi, It looks like we move the bundle from one broker to another broker but doesn't move the historical load data of this bundle. I suppose this is the main cause of the problem.
Thanks, Penghui On 2022/10/24 17:46:21 Heesung Sohn wrote: > Hi, > > Currently, the historical score is based on exponential smoothing. > As you described, the current default value is conservative (by default > 90%), meaning it weighs significantly more on the historical load than the > current load in the formula. > historicalPecentage is configurable, and one could make it smaller to > smooth the load data more radically. > > One thing we could tweak the current ThresholdShedder for the repeated > unloading problem(because the shedder algo smooths the old load data too > late after unloading) is to pre-update load data for the broker that just > ran the unload operation(in the brokerAvgResourceUsage object in > ThersholdShedder). > > For example, right after sending the unload command, the leader could > simply clear the load data of the target broker in brokerAvgResourceUsage, > or we could add a secondary load variable(last-unloaded-percentage) and > negate the load in the next load score formula. > > In the future, when we implement the bundle transfer unloading(described in > PIP-192), we can specifically pre-update the load data for the unload > destination broker as well(Currently, there is no way to pre-identify the > destination broker when making an unloading decision.) > > Regards, > Heesung > > On Mon, Oct 24, 2022 at 7:38 AM BIGO <fengwenzhi....@bigo.sg.invalid> wrote: > > > Hi pulsar-dev community, I raised a pip to discuss : PIP-217: > > LoadShedding Strategy Improment > > PIP link: PIP-217: LoadShedding Strategy Improment · Issue #18173 · > > apache/pulsar (github.com) <https://github.com/apache/pulsar/issues/18173> > > > > Regards, > > TheTumbled. >
