Hi folks,
I rewrite the proposal with watermark way, and update the proposal
here https://github.com/apache/bookkeeper/issues/3231#issue-1210800448
And if you are interested in the implementation, I wrote a prototype here
https://github.com/apache/bookkeeper/pull/3139/files
Here are some logs fr
>What if the BK client followed netty's decision on this: raise a flag (e.g.
isWritable())?
Bookie clients can write when there have AQ bookies are alive. It also
can change the ledger's bookie ensemble if there has a bookie failure.
So looks like it is difficult to use netty's decision.
On Tue,
What if the BK client followed netty's decision on this: raise a flag (e.g.
isWritable())?
In this case it would be up to Pulsar (or any other app) to decide what to
do.
On Fri, Sep 30, 2022 at 10:02 PM Michael Marshall
wrote:
> Thank you for your points, Lari. They expanded on my thoughts very
Thank you for your points, Lari. They expanded on my thoughts very well.
One important design aspect of Netty's channel writability status is
that it is not strictly enforced. It is up to the application to stop
writing to an unwritable channel. Similarly, with a reactive solution,
it would be up
Sorry for the typo. I mean WQ > AQ.
Thanks for your information, Lari!
Let me try to reconsider this proposal with the watermark way.
Yong
On Thu, Sep 29, 2022 at 21:11 Enrico Olivelli wrote:
> Il giorno gio 29 set 2022 alle ore 15:06 Dave Fisher
> ha scritto:
> >
> >
> > >
> > > I think I n
Il giorno gio 29 set 2022 alle ore 15:06 Dave Fisher
ha scritto:
>
>
> >
> > I think I need to change this proposal title to `BookKeeper client write
> > memory
> > limits` to make it clearly. What we observed is bookie will easily OOM when
> > WQ < AQ. So the main problem we want to use this prop
>
> I think I need to change this proposal title to `BookKeeper client write
> memory
> limits` to make it clearly. What we observed is bookie will easily OOM when
> WQ < AQ. So the main problem we want to use this proposal to resolve is
> limit
> the adds request memory usage.
What is the use
On 2022/09/29 04:28:31 Michael Marshall wrote:
> I support adding back pressure based on client memory limits to the
> bookkeeper client.
>
> My biggest concern is how the back pressure is propagated to the
> client application. If I am reading the draft implementation
> correctly, it is via a blo
Thanks, Michael！
>In my use case (the Pulsar broker), I think a blocking implementation
will make this feature very hard to use. One quick thought is that
maybe some kind of event or listener could meet the requirements
without also blocking an application? The implementation could be
something si
I support adding back pressure based on client memory limits to the
bookkeeper client.
My biggest concern is how the back pressure is propagated to the
client application. If I am reading the draft implementation
correctly, it is via a blocking operation on the calling thread for
the `BookieClient
Yong,
Il giorno mer 28 set 2022 alle ore 10:23 Yong Zhang
ha scritto:
>
> We have improved the memory issue with backpressure with PR
> https://github.com/apache/bookkeeper/pull/3324
>
> The backpressure way can prevent there have too many Add requests
> pending to the client and waiting for the
We have improved the memory issue with backpressure with PR
https://github.com/apache/bookkeeper/pull/3324
The backpressure way can prevent there have too many Add requests
pending to the client and waiting for the response. It makes the add
requests
fail fast, so if the channel is not writable, i
Hello Yong:
It seems to be a very useful feature. In the production environment, you
can often see similar phenomena happening.
+1 (non-binding)
--
Thanks
Xiaolong Ran
Yong Zhang 于2022年4月21日周四 18:29写道：
> Hi all,
>
> The BP-51 BookKeeper client memory limits is ready for review.
> The proposal
Hi all,
The BP-51 BookKeeper client memory limits is ready for review.
The proposal is here: https://github.com/apache/bookkeeper/issues/3231
And the PR is here: https://github.com/apache/bookkeeper/pull/3139
Please help to review this proposal.
Thanks!
Yong
14 matches
Mail list logo
