Hi Jiuming, I have a few questions:
Motivation > > Currently, there's no way to track CommandPartitionedTopicMetadata > requests. There's no metrics that indicate that a broker is handling > CommandPartitionedTopicMetadata requests. > > Misconfigured clients might flood brokers with > CommandPartitionedTopicMetadata requests and cause high CPU consumption. > I managed to understand the first line: You can't know how many such requests each broker received. In the second line which is the problem statement I couldn't figure out: 1. What do you mean by misconfigured clients? What configuration can you change in a wrong way that will cause the brokers with such requests? 2. Clients ask a random broker for that metadata? So for now, you say that by having the metrics in place, I'll be able to set up an alert on a spike of such requests, correlate that with the high CPU, and then do what - reconfigure the client somehow to avoid sending those requests? After reaching the end of the PIP I discovered a link to an issue that gives an example of a misconfiguration. Perhaps then it would be good to copy the example or at least link to it in the motivation section to say - here is a link that contains an example (I prefer to inline it myself). Other than that, I think it's good. I do have a thought I want to share, to hear what people think: When I read it a couple of things occurred to me: 1. Maybe we should put this code as a general wrapper for any command, to avoid copy-pasting it for any other command we may need in the future. I realize it's not an easy thing to do given the way the code is structured today. The metric name you gave is generic enough so this can be refactored as I said but in the future. 2. We provide some means for the operator to discover the issue, but definitely not easy means. Also, we're not really solving any user experience. In theory, we can say: * There should be a reasonable delay between requests for this metadata, why not enforce it? Perhaps apply a rate limit to this command which seems reasonable by default * The client can utilize the bi-directional nature of Pulsar Protocol and get notified when partitions change (subscribe) therefore limiting the need to pull based on frequency. Thanks, Asaf On Thu, Nov 3, 2022 at 12:19 PM Jiuming Tao <jm...@streamnative.io.invalid> wrote: > Hi pulsar community, > > I’ve opened a PIP to discuss: PIP-222: Add CommandPartitionedTopicMetadata > metrics > > The PIP link: https://github.com/apache/pulsar/issues/18319 < > https://github.com/apache/pulsar/issues/18319> > > Thanks, > Tao Jiuming
