Hi, Currently, the historical score is based on exponential smoothing. As you described, the current default value is conservative (by default 90%), meaning it weighs significantly more on the historical load than the current load in the formula. historicalPecentage is configurable, and one could make it smaller to smooth the load data more radically.
One thing we could tweak the current ThresholdShedder for the repeated unloading problem(because the shedder algo smooths the old load data too late after unloading) is to pre-update load data for the broker that just ran the unload operation(in the brokerAvgResourceUsage object in ThersholdShedder). For example, right after sending the unload command, the leader could simply clear the load data of the target broker in brokerAvgResourceUsage, or we could add a secondary load variable(last-unloaded-percentage) and negate the load in the next load score formula. In the future, when we implement the bundle transfer unloading(described in PIP-192), we can specifically pre-update the load data for the unload destination broker as well(Currently, there is no way to pre-identify the destination broker when making an unloading decision.) Regards, Heesung On Mon, Oct 24, 2022 at 7:38 AM BIGO <fengwenzhi....@bigo.sg.invalid> wrote: > Hi pulsar-dev community, I raised a pip to discuss : PIP-217: > LoadShedding Strategy Improment > PIP link: PIP-217: LoadShedding Strategy Improment · Issue #18173 · > apache/pulsar (github.com) <https://github.com/apache/pulsar/issues/18173> > > Regards, > TheTumbled.
