Re: [DISCUSS] Introduce FlowControl to metrics endpoint

2022-08-30 Thread Asaf Mesika
> > I'd like to add a new aspect to the proposal: there might be other > solutions that would be useful in the case of large amount of topics in a > Pulsar cluster. > Rate limiting on the /metrics endpoint doesn't sound like the correct > approach. > > When there's a huge about of metrics, instead

Re: [DISCUSS] Introduce FlowControl to metrics endpoint

2022-08-29 Thread Lari Hotari
Good reiteration of the problem and good points, Asaf. I'd like to add a new aspect to the proposal: there might be other solutions that would be useful in the case of large amount of topics in a Pulsar cluster. Rate limiting on the /metrics endpoint doesn't sound like the correct approach. When

Re: [DISCUSS] Introduce FlowControl to metrics endpoint

2022-08-29 Thread Asaf Mesika
Hi Jiuming, I would reiterate the problem statement to make it clear (at least for me): There are cases where a very large amount of topics exists (> 10k per broker) and are used in Pulsar. Those topics usually have multiple producers and multiple consumers. There are metrics that are in the gran

Re: [DISCUSS] Introduce FlowControl to metrics endpoint

2022-08-18 Thread Jiuming Tao
bump Jiuming Tao 于2022年8月8日周一 18:19写道: > Hi Pulsar community, > > In the situation of expose metrics data which has a big size, it will lead > to: > 1. A sudden increase of network usage > 2. The latency of pub/sub rising > > For the purpose of resolving these problems, I’d opened a PR: > https: