Just to chime in here since I recently went through a similar thing, I support adding the version as a tag instead of introducing an entirely new metric for this. In fact I just implemented exactly this in a project that uses Kafka, for these reasons:
1. Adding the version as a tag means that all metrics which are already collected will benefit, and lets you easily tell at a glance which version a specific client metric corresponds to. This is incredibly useful when looking at a dashboard covering multiple instances from different sources. For example, imagine a graph that plots the performance (eg bytes consumed rate) of many individual consumers and which shows several of them maxing out much lower than the rest. If the metric is tagged with the version already, you can easily check if the slow consumers are all using a specific version and may be displaying a performance regression. If the version info has to be plotted separately as its own metric, this is much more of a hassle to check. 2. Additional metrics can be expensive, but additional tags are almost always free (at least, that is my understanding) 3. As you guys already discussed, many systems (like Prometheus) require numeric values, and it's pretty much impossible to come up with a readable scheme for all the relevant versioning info -- even if we removed the dots we're left with a rather unreadable representation of the version and of course will need to solve the "-SNAPSHOT" issue somehow. But beyond that, in addition to the raw version we also wanted to emit the specific commit id, which really needs to be a string. I'm pretty sure Kafka client metrics also include the commit id in addition to the version. If we add the version to the tags, we should consider adding the commit id as well. This is incredibly useful for intermediate/SNAPSHOT versions, which don't uniquely identify the specific code that is running. I would personally love to see a KIP start tagging the existing metrics with the version info, and it sounds like this would also solve your problem in a very natural way On Tue, Oct 10, 2023 at 5:42 AM Mickael Maison <mickael.mai...@gmail.com> wrote: > Hi Hudeqi, > > Rather than creating a gauge with a dummy value, could we add the > version (and commitId) as tags to an existing metric. > For example, the alongside the existing Version and CommitId metrics > we have StartTimeMs. Maybe we can have a StartTimeMs metrics with the > version and commitId) as tags on it? The existing metric already has > the brokerid (id) as tag. WDYT? > > Thanks, > Mickael > > On Thu, Aug 31, 2023 at 4:59 AM hudeqi <16120...@bjtu.edu.cn> wrote: > > > > Thank you for your answer, Mickael. > > If set the value of gauge to a constant value of 1, adding that tag key > is "version" and value is the version value of the obtained string type, > does this solve the problem? We can get the version by tag in prometheus. > > > > best, > > hudeqi > > > > "Mickael Maison" <mickael.mai...@gmail.com>写道： > > > Hi, > > > > > > Prometheus only support numeric values for metrics. This means it's > > > not able to handle the kafka.server:type=app-info metric since Kafka > > > versions are not valid numbers (3.5.0). > > > As a workaround we could create a metric with the version without the > > > dots, for example with value 350 for Kafka 3.5.0. > > > > > > Also in between releases Kafka uses the -SNAPSHOT suffix (for example > > > trunk is currently 3.7.0-SNAPSHOT) so we should also consider a way to > > > handle those. > > > > > > Thanks, > > > Mickael > > > > > > On Wed, Aug 30, 2023 at 2:51 PM hudeqi <16120...@bjtu.edu.cn> wrote: > > > > > > > > Hi, Kamal, thanks your reminding, but I have a question: It seems > that I can't get this metric through "jmx_prometheus"? Although I observed > this metric through other tools. > > > > > > > > best, > > > > hudeqi > > > > > > > > "Kamal Chandraprakash" <kamal.chandraprak...@gmail.com > >写道： > > > > > Hi Hudeqi, > > > > > > > > > > Kafka already emits the version metric. Can you check whether the > below > > > > > metric satisfies your requirement? > > > > > > > > > > kafka.server:type=app-info,id=0 > > > > > > > > > > -- > > > > > Kamal > > > > > > > > > > On Mon, Aug 28, 2023 at 2:29 PM hudeqi <16120...@bjtu.edu.cn> > wrote: > > > > > > > > > > > Hi, all, I want to submit a minor kip to add a metric, which > supports to > > > > > > get the running kafka server verison, the wiki url is here > > > > > > > > > > > > Motivation > > > > > > > > > > > > At present, it is impossible to perceive the Kafka version that > the broker > > > > > > is running from the perspective of metrics. If multiple Kafka > versions are > > > > > > deployed in a cluster due to various reasons, it is difficult > for us to > > > > > > intuitively understand the version distribution. > > > > > > > > > > > > So, I want to add a kafka version metric indicating the version > of the > > > > > > current running kafka server, it can help us to perceive the > mixed > > > > > > distribution of multiple versions, and to perceive the progress > of version > > > > > > upgrade in the cluster in real time. > > > > > > > > > > > > Proposed Changes > > > > > > > > > > > > When instantiating kafkaServer/BrokerServer, register > `KafkaVersion` gauge > > > > > > metric, whose value is obtained by `VersionInfo.getVersion`. And > remove all > > > > > > related metrics when kafkaServer/BrokerServer shutdown. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > best, > > > > > > > > > > > > hudeqi > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
