Re: [DISCUSS] PIP-174: Provide new implementation for broker dispatch cache

2022-06-22 Thread Matteo Merli
> If all topics share the same cache space, and we won't recycle space by topic. > So when some topics are rebalanced to other brokers, the previous broker > keeps its cache > and results in cache space wasting. > The topic shedding would be quite frequent under some cases, I am not sure > how ba

Re: [DISCUSS] PIP-174: Provide new implementation for broker dispatch cache

2022-06-12 Thread Haiting Jiang
> Instead of dealing with individual caches for each topic, let's adopt > a model where > there is a single cache space for the broker. If all topics share the same cache space, and we won't recycle space by topic. So when some topics are rebalanced to other brokers, the previous broker keeps its

Re: [DISCUSS] PIP-174: Provide new implementation for broker dispatch cache

2022-06-09 Thread Aloys Zhang
+1 Thanks, Aloys Hang Chen 于2022年6月8日周三 08:53写道: > +1 Great idea! > > Thanks, > Hang > > Lari Hotari 于2022年6月8日周三 03:32写道: > > > > This is a very useful proposal. LGTM > > > > -Lari > > > > On Tue, Jun 7, 2022 at 3:48 AM Matteo Merli > wrote: > > > > > https://github.com/apache/pulsar/issues

Re: [DISCUSS] PIP-174: Provide new implementation for broker dispatch cache

2022-06-07 Thread Hang Chen
+1 Great idea! Thanks, Hang Lari Hotari 于2022年6月8日周三 03:32写道: > > This is a very useful proposal. LGTM > > -Lari > > On Tue, Jun 7, 2022 at 3:48 AM Matteo Merli wrote: > > > https://github.com/apache/pulsar/issues/15954 > > > > WIP can be seen at: https://github.com/apache/pulsar/pull/15955 > >

Re: [DISCUSS] PIP-174: Provide new implementation for broker dispatch cache

2022-06-07 Thread Lari Hotari
This is a very useful proposal. LGTM -Lari On Tue, Jun 7, 2022 at 3:48 AM Matteo Merli wrote: > https://github.com/apache/pulsar/issues/15954 > > WIP can be seen at: https://github.com/apache/pulsar/pull/15955 > > --- > > > ## Motivation > > The current implementation of the read cache

Re: [DISCUSS] PIP-174: Provide new implementation for broker dispatch cache

2022-06-07 Thread Matteo Merli
On Tue, Jun 7, 2022 at 6:37 AM Enrico Olivelli wrote: > Great idea. > I wonder which kind of metrics we could have. To see how each > tenant/namespace is using the cache It can be done, at the expense of some CPU cost, for example by adding a tag for the topic and computing the sizes at the time

Re: [DISCUSS] PIP-174: Provide new implementation for broker dispatch cache

2022-06-07 Thread Enrico Olivelli
Great idea. I wonder which kind of metrics we could have. To see how each tenant/namespace is using the cache Enrico Il giorno mar 7 giu 2022 alle ore 15:12 PengHui Li ha scritto: > > +1 > > Penghui > On Jun 7, 2022, 08:48 +0800, Matteo Merli , wrote: > > https://github.com/apache/pulsar/issues/1

Re: [DISCUSS] PIP-174: Provide new implementation for broker dispatch cache

2022-06-07 Thread PengHui Li
+1 Penghui On Jun 7, 2022, 08:48 +0800, Matteo Merli , wrote: > https://github.com/apache/pulsar/issues/15954 > > WIP can be seen at: https://github.com/apache/pulsar/pull/15955 > > --- > > > ## Motivation > > The current implementation of the read cache in the Pulsar broker has largely >

[DISCUSS] PIP-174: Provide new implementation for broker dispatch cache

2022-06-06 Thread Matteo Merli
https://github.com/apache/pulsar/issues/15954 WIP can be seen at: https://github.com/apache/pulsar/pull/15955 --- ## Motivation The current implementation of the read cache in the Pulsar broker has largely remained unchanged for a long time, except for a few minor tweaks. While the im