Re: Performance issue in Pulsar with large fan-outs since Pulsar 2.8.2

2022-08-25 Thread Lari Hotari
I have submitted a fix in https://github.com/apache/pulsar/pull/17273 . The main motivation of this PR is to fix and restore broker cache eviction of entries based on the earliest read position of active cursors (consumers) and also handle this in an efficient and performant way so that the origin

Re: Performance issue in Pulsar with large fan-outs since Pulsar 2.8.2

2022-08-21 Thread Michael Marshall
Thank you for starting this important discussion, Lari. It seems the issue is in the way the `heap` list in the ManagedCursorContainer class is ordered. The heap is ordered by mark delete position, while cache eviction is determined by the slowest read position [0]. The PR [1] essentially says tha