After doing more testing I have updated the patch with a small change but very effective https://github.com/apache/pulsar/pull/17241
With the latest commit now we are able to attach to pending requests for ranges that are "larger" than the requested range. So if you have 100...200 in pending state and it comes 101..200 the new read will be attached to the pending read. I have attached to the PR some graphs to show that the rate of "partial matches" is very high both in the tailing reads case and in catch up cases. In my test case I have 62 subscriptions on a partitioned topic with 6 partitions, on 6 brokers . I thought about try to make it more complicated, like: - splitting the reads to smaller chunks and aligned - try to use a pending read that overlaps with the requested range and start a second read to read the missing entries Both of the two strategies are very complex and the current form is already doing good results I did some investigation on the overhead on allocations, the GC is not suffering from this change in my tests Enrico
