Re: [DISCUSS] PIP-211: Introduce offload throttling

2022-11-11 Thread Michael Marshall
n FileSystem and BlobStore offloaders, > >>>> event custom offloaders. I think this is not universal. > >>>> > >>>>> One simple way to do this is to to limit how many threads can write > >>>>> offloaded ledgers. This is th

Re: [DISCUSS] PIP-211: Introduce offload throttling

2022-11-11 Thread Jiuming Tao
not >>>> effectively limit traffic. If the reading rate of BK is very fast, it also >>>> leads to high CPU/Memory/Network usage >>>> >>>> Thanks, >>>> Tao Jiuming >>>> >>>>> 2022年11月2日 上午1:43,Dave Fisher 写道: >>>>> >>>>> One alternative would be to throttle offload in the write path instead of >>>>> adding additional logic to the read path in managed ledgers. >>>>> >>>>> One simple way to do this is to to limit how many threads can write >>>>> offloaded ledgers. This is the same way that reading of offloaded ledgers >>>>> are already “throttled” by that thread count defaulting to 2. >>>>> >>>>> Regards, >>>>> Dave >>>>> >>>>> Sent from my iPhone >>>>> >>>>>> On Nov 1, 2022, at 10:27 AM, Jiuming Tao >>>>>> wrote: >>>>>> >>>>>> Hi pulsar community, >>>>>> >>>>>> I opened a PIP to discuss: PIP-211: Introduce offload throttling >>>>>> >>>>>> PIP link: https://github.com/apache/pulsar/issues/18004 >>>>>> <https://github.com/apache/pulsar/issues/18004> >>>>>> >>>>>> Thanks, >>>>>> Tao Jiuming >>>>> >>>> >>

Re: [DISCUSS] PIP-211: Introduce offload throttling

2022-11-10 Thread Michael Marshall
usage > >> > >> Thanks, > >> Tao Jiuming > >> > >>> 2022年11月2日 上午1:43,Dave Fisher 写道: > >>> > >>> One alternative would be to throttle offload in the write path instead of > >>> adding additional logic to the read path in

Re: [DISCUSS] PIP-211: Introduce offload throttling

2022-11-10 Thread Jiuming Tao
gt;>> One simple way to do this is to to limit how many threads can write >>> offloaded ledgers. This is the same way that reading of offloaded ledgers >>> are already “throttled” by that thread count defaulting to 2. >>> >>> Regards, >>> Dave

Re: [DISCUSS] PIP-211: Introduce offload throttling

2022-11-07 Thread Michael Marshall
> One simple way to do this is to to limit how many threads can write > > offloaded ledgers. This is the same way that reading of offloaded ledgers > > are already “throttled” by that thread count defaulting to 2. > > > > Regards, > > Dave > > > >

Re: [DISCUSS] PIP-211: Introduce offload throttling

2022-11-07 Thread Jiuming Tao
Sent from my iPhone > >> On Nov 1, 2022, at 10:27 AM, Jiuming Tao >> wrote: >> >> Hi pulsar community, >> >> I opened a PIP to discuss: PIP-211: Introduce offload throttling >> >> PIP link: https://github.com/apache/pulsar/issues/18004 >> <https://github.com/apache/pulsar/issues/18004> >> >> Thanks, >> Tao Jiuming >

Re: [DISCUSS] PIP-211: Introduce offload throttling

2022-11-01 Thread Dave Fisher
“throttled” by that thread count defaulting to 2. Regards, Dave Sent from my iPhone > On Nov 1, 2022, at 10:27 AM, Jiuming Tao > wrote: > > Hi pulsar community, > > I opened a PIP to discuss: PIP-211: Introduce offload throttling > > PIP link: https://github.com/ap

[DISCUSS] PIP-211: Introduce offload throttling

2022-11-01 Thread Jiuming Tao
Hi pulsar community, I opened a PIP to discuss: PIP-211: Introduce offload throttling PIP link: https://github.com/apache/pulsar/issues/18004 <https://github.com/apache/pulsar/issues/18004> Thanks, Tao Jiuming