Hi, Mickael, Sophie, Doğuşcan. Thanks for your replys. I like the idea of adding the version and commitId as tags for existed metric named "startTimeMs". I will update the KIP doc and start the vote process for this KIP. It seems that there is not an existing metric named "uptime", Doğuşcan.
best, hudeqi "Doğuşcan Namal" <namal.dogus...@gmail.com>写道： > Hello, do we have a metric showing the uptime? We could tag that metric > with version information as well. > > I like the idea of adding the version as a tag as well. However, I am not > inclined to tag each metric with a KafkaVersion information. We could > discuss which metrics could be tagged but let's keep that out of scope from > this discussion. > > On Wed, 11 Oct 2023 at 07:37, Sophie Blee-Goldman <ableegold...@gmail.com> > wrote: > > > Just to chime in here since I recently went through a similar thing, I > > support adding the version > > as a tag instead of introducing an entirely new metric for this. In fact I > > just implemented exactly this > > in a project that uses Kafka, for these reasons: > > > > 1. Adding the version as a tag means that all metrics which are already > > collected will benefit, and lets you easily tell > > at a glance which version a specific client metric corresponds to. This is > > incredibly useful when looking at a dashboard > > covering multiple instances from different sources. For example, imagine a > > graph that plots the performance (eg bytes > > consumed rate) of many individual consumers and which shows several of them > > maxing out much lower than the rest. > > If the metric is tagged with the version already, you can easily check if > > the slow consumers are all using a specific version > > and may be displaying a performance regression. If the version info has to > > be plotted separately as its own metric, this is > > much more of a hassle to check. > > 2. Additional metrics can be expensive, but additional tags are almost > > always free (at least, that is my understanding) > > 3. As you guys already discussed, many systems (like Prometheus) require > > numeric values, and it's pretty much impossible > > to come up with a readable scheme for all the relevant versioning info -- > > even if we removed the dots we're left with a rather > > unreadable representation of the version and of course will need to solve > > the "-SNAPSHOT" issue somehow. But beyond that, > > in addition to the raw version we also wanted to emit the specific commit > > id, which really needs to be a string. > > > > I'm pretty sure Kafka client metrics also include the commit id in addition > > to the version. If we add the version to the tags, > > we should consider adding the commit id as well. This is incredibly useful > > for intermediate/SNAPSHOT versions, which > > don't uniquely identify the specific code that is running. > > > > I would personally love to see a KIP start tagging the existing metrics > > with the version info, and it sounds like this would also > > solve your problem in a very natural way > > > > On Tue, Oct 10, 2023 at 5:42 AM Mickael Maison <mickael.mai...@gmail.com> > > wrote: > > > > > Hi Hudeqi, > > > > > > Rather than creating a gauge with a dummy value, could we add the > > > version (and commitId) as tags to an existing metric. > > > For example, the alongside the existing Version and CommitId metrics > > > we have StartTimeMs. Maybe we can have a StartTimeMs metrics with the > > > version and commitId) as tags on it? The existing metric already has > > > the brokerid (id) as tag. WDYT? > > > > > > Thanks, > > > Mickael > > > > > > On Thu, Aug 31, 2023 at 4:59 AM hudeqi <16120...@bjtu.edu.cn> wrote: > > > > > > > > Thank you for your answer, Mickael. > > > > If set the value of gauge to a constant value of 1, adding that tag key > > > is "version" and value is the version value of the obtained string type, > > > does this solve the problem? We can get the version by tag in prometheus. > > > > > > > > best, > > > > hudeqi > > > > > > > > "Mickael Maison" <mickael.mai...@gmail.com>写道： > > > > > Hi, > > > > > > > > > > Prometheus only support numeric values for metrics. This means it's > > > > > not able to handle the kafka.server:type=app-info metric since Kafka > > > > > versions are not valid numbers (3.5.0). > > > > > As a workaround we could create a metric with the version without the > > > > > dots, for example with value 350 for Kafka 3.5.0. > > > > > > > > > > Also in between releases Kafka uses the -SNAPSHOT suffix (for example > > > > > trunk is currently 3.7.0-SNAPSHOT) so we should also consider a way > > to > > > > > handle those. > > > > > > > > > > Thanks, > > > > > Mickael > > > > > > > > > > On Wed, Aug 30, 2023 at 2:51 PM hudeqi <16120...@bjtu.edu.cn> wrote: > > > > > > > > > > > > Hi, Kamal, thanks your reminding, but I have a question: It seems > > > that I can't get this metric through "jmx_prometheus"? Although I > > observed > > > this metric through other tools. > > > > > > > > > > > > best, > > > > > > hudeqi > > > > > > > > > > > > "Kamal Chandraprakash" & > > lt;kamal.chandraprak...@gmail.com > > > >写道： > > > > > > > Hi Hudeqi, > > > > > > > > > > > > > > Kafka already emits the version metric. Can you check whether the > > > below > > > > > > > metric satisfies your requirement? > > > > > > > > > > > > > > kafka.server:type=app-info,id=0 > > > > > > > > > > > > > > -- > > > > > > > Kamal > > > > > > > > > > > > > > On Mon, Aug 28, 2023 at 2:29 PM hudeqi <16120...@bjtu.edu.cn> > > > wrote: > > > > > > > > > > > > > > > Hi, all, I want to submit a minor kip to add a metric, which > > > supports to > > > > > > > > get the running kafka server verison, the wiki url is here > > > > > > > > > > > > > > > > Motivation > > > > > > > > > > > > > > > > At present, it is impossible to perceive the Kafka version that > > > the broker > > > > > > > > is running from the perspective of metrics. If multiple Kafka > > > versions are > > > > > > > > deployed in a cluster due to various reasons, it is difficult > > > for us to > > > > > > > > intuitively understand the version distribution. > > > > > > > > > > > > > > > > So, I want to add a kafka version metric indicating the version > > > of the > > > > > > > > current running kafka server, it can help us to perceive the > > > mixed > > > > > > > > distribution of multiple versions, and to perceive the progress > > > of version > > > > > > > > upgrade in the cluster in real time. > > > > > > > > > > > > > > > > Proposed Changes > > > > > > > > > > > > > > > > When instantiating kafkaServer/BrokerServer, register > > > `KafkaVersion` gauge > > > > > > > > metric, whose value is obtained by `VersionInfo.getVersion`. > > And > > > remove all > > > > > > > > related metrics when kafkaServer/BrokerServer shutdown. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > best, > > > > > > > > > > > > > > > > hudeqi > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
