After doing more testing I have updated the patch with a small change
but very effective
https://github.com/apache/pulsar/pull/17241
With the latest commit now we are able to attach to pending requests
for ranges that are "larger" than the requested range.
So if you have 100...200 in pending state
Thank you folks for your feedback
Il giorno gio 25 ago 2022 alle ore 05:56 Haiting Jiang
ha scritto:
>
> Here is my idea of handing the read position shift if we want to go a bit
> further with this solution.
>
> Instead of only checking the exact match of entry range match
> (PendingReadKey in t
Here is my idea of handing the read position shift if we want to go a bit
further with this solution.
Instead of only checking the exact match of entry range match
(PendingReadKey in the PR),
we can check if the requesting range is covered by several ongoing read
requests, and merge
the result of
+1 for this optimization for the specific scenario.
This simple patch is really effective, in some tests it reduces a lot
> (from 100K reads/s to 250 reads/s) the concurrent reads to BK.
> You may say that it is unlikely that the broker sends concurrent
> requests for exactly the same range of ent
+1 I support merging the temporary solution.
> Just one point, please make sure the change will not introduce too much
> heap memory overhead.
If we see that there is extra pressure, I wonder if we can dynamically
determine when to deduplicate requests. A broker only needs this
feature when a top
+1 for we can have a short-term solution first.
Just one point, please make sure the change will not introduce too much
heap memory overhead.
Thanks,
Penghui
On Wed, Aug 24, 2022 at 11:54 PM Enrico Olivelli
wrote:
> Hello folks,
> These days I am working on a use case in which I have many
> su
