I guess this is a mitigation that should be taken until we fix the memory usage metrics.
I hope some effort could be put in fixing the memory usage metrics for heap memory, direct memory and Netty buffer pool usage. For improving the heap memory metrics, there's an issue https://github.com/apache/pulsar/issues/21973 . It contains some possible solutions. Netty pools direct memory allocations and therefore relying on direct memory usage isn't meaningful. There's a lot of metrics for the buffer pool, https://netty.io/4.1/api/io/netty/buffer/PooledByteBufAllocator.html . There are multiple heap arenas and direct arenas which each have a set of metrics, https://netty.io/4.1/api/io/netty/buffer/PoolArenaMetric.html . Is anyone interested in contributing to fix the memory usage metrics used for Pulsar load balancing? -Lari On 2024/06/11 03:50:31 thetumbled wrote: > Hi, Pulsar Community. > I open a new PIP for making resource weight conf work for > OverloadShedder, LeastLongTermMessageRate, ModularLoadManagerImpl, so that we > can avoid the impact of memory usage and direct memory usage. > link: https://github.com/apache/pulsar/pull/22889 > > Thanks, > Wenzhi Feng(thetumbled)
