Hi All, I would like to share with you a document I wrote during the last months titled Pulsar Metrics - Current State and Future Directions <https://docs.google.com/document/d/1vke4w1nt7EEgOvEerPEUS-Al3aqLTm9cl2wTBkKNXUA/edit?usp=sharing>, and most importantly *get your feedback.*
The initial motivation is to rethink/refactor the way metrics are used in Pulsar codebase to solve two large pain points: 1. *Metrics Cardinality: *As Pulsar can support up to 1M topics cluster-wide, this translates into ~100M unique time series, which becomes both an impossible cost and affects query performance and general usability of metrics. This issue starts surfacing even at 50k-100k topics. Today users work-around it by disabling topic-granularity metrics and scripting their own ETL for generating metrics they can use (based on admin stats API), switching between granular topic-level metrics to a group-by view of their choosing. The document outlines a solution built upon the notion of Groups, in which users can define a group of metrics, and specify if they wish to define a roll-up on it (i.e. remove labels) and filter (i.e. remove specific metrics). The solution should be able to bring the granularity from topic level (1M) to group level (1000). 2. *Consolidate into a single library:* Today there are 4 different metrics libraries/systems in Pulsar. This creates lots of confusion and unhappy developer experience, among other impacts. Also achieving (1) requires having (2). The document outlines the different libraries, their functionality and the problems they create. The doc also describes one idea for such a library, but it still requires a POC. The main goal of the document is mainly to garner feedback to see if the directions stipulated there are agreed upon, and if there is any other problem missing or existing functionality missed as it serves as the basis for the requirements for the solution that will be chosen. Thanks! Asaf Mesika Document link: https://docs.google.com/document/d/1vke4w1nt7EEgOvEerPEUS-Al3aqLTm9cl2wTBkKNXUA/edit?usp=sharing
