Hi Kevin,
I recently ran into.a case where the controllers and brokers were
reporting a metadata version value in the metrics which was the
minimum value even though the cluster had been previously finalized
with a different value for the metadata version. After some
investigation, this is because
Hi Kevin,
On Thu, May 15, 2025 at 10:52 AM Kevin Wu wrote:
> Thanks for all the comments about the metric type field for the minimum and
> maximum supported feature levels. I agree they are software version
> specific. Also, since they are shared across the broker and controller like
> the Finali
Hi all,
Thanks for all the comments about the metric type field for the minimum and
maximum supported feature levels. I agree they are software version
specific. Also, since they are shared across the broker and controller like
the FinalizedLevel metric, having two separate metrics is redundant an
Hi Kevin,
Thanks for the KIP.
4. How about using a new metric type to collect all feature metrics?
So we don’t need to have different metric names in controller and broker.
For example:
* kafka.server:type=FeatureMetrics,name=FinalizedLevel,featureName=X
* kafka.server:type=FeatureMetrics,name=Mi
Hi, Kevin,
Thanks for the reply.
4. There seems to be some inconsistency. For MinimumSupportedLevel, we have
two metrics, one with the package kafka.controller and another with
kafka.server. For FinalizedLevel, we only have one metric with the package
kafka.server. Could we choose a more consiste
Hi Kevin,
The metrics that you added for supported versions, "
kafka.controller:type=KafkaController,name=MaximumSupportedLevel,featureName=X"
and
"kafka.server:type=broker-metadata-metrics,name=maximum-supported-feature-level,featureName=X",
the metric group name doesn't seem accurate. Those met
Hi Jun,
On Mon, May 12, 2025 at 7:21 PM Jun Rao wrote:
> 5. Some of the metric names are camel case and some others use dash. It
> would be useful to be consistent.
They are consistent within their metrics group or type in JMX. I would
be in favor of a KIP that standardizes how metrics are named
HI Kevin,
On Tue, May 13, 2025 at 9:56 AM Kevin Wu wrote:
> 4. Maybe I'm missing something, but I think the MetadataLoader is used by
> both the broker and controller, so having the one metric works for both
> node types. The CurrentMetadataVersion metric is currently reported on both
> the broke
Hey Jun,
Thanks for the comments:
4. Maybe I'm missing something, but I think the MetadataLoader is used by
both the broker and controller, so having the one metric works for both
node types. The CurrentMetadataVersion metric is currently reported on both
the broker and controller.
5. What is the
Hi, Kevin,
Thanks for the updated KIP. A couple of more comments.
4. Should we expose the FinalizedLevel metric on the controller too?
5. Some of the metric names are camel case and some others use dash. It
would be useful to be consistent.
Jun
On Mon, May 12, 2025 at 7:06 AM Kevin Wu wrote:
Hey Chia-Ping and Justine,
Okay, that makes sense about the minimum version changing at some point.
I'll add these metrics to this KIP. Thanks for the insightful discussion.
Best,
Kevin Wu
On Fri, May 9, 2025 at 4:54 PM Kevin Wu wrote:
> Hey Chia-Ping and Justine,
>
> Thanks for the explanatio
Hi all,
> so this metric's value should only change when a software upgrade/downgrade
> occurs. Otherwise, the range should not change. Is that correct?
Yes, Justine already gives great answers
> since the minimum is always 0?
As Justine’s response, we may have a feature which can’t be disabl
Yup, this would change during a software upgrade/downgrade, which often
takes longer than the feature upgrades.
I think for now minimum is 0, but I do recall some folks saying that could
change in the future.
Justine
On Fri, May 9, 2025 at 2:55 PM Kevin Wu wrote:
> Hey Chia-Ping and Justine,
>
Hey Chia-Ping and Justine,
Thanks for the explanation. I see where y'all are coming from, but I want
to make sure I understand how the value of this metric would change.
It seems to me that the supported feature range is determined by the
software version, so this metric's value should only chang
hi All
Thank you, Justine, for your explanation. You are right that my point is that
it would be useful to check the supported versions for each node at runtime,
especially in a hybrid cluster or during a binary upgrade.
I don't have a strong reason to push this into this KIP, as the functional
Hey Kevin,
I think Chia Ping was referring to not the time when the feature upgrade is
happening, but when a kafka image upgrade is happening for example.
We can't upgrade the feature until all brokers support it, so it would be
also helpful to see that in real time? Something like that.
I don't
Hey Chia-Ping,
I hadn't considered adding the supported versions for each feature as a
metric, but I'm not sure if it's helpful for monitoring the progress of an
upgrade/downgrade of a feature. For example, if a node doesn't support a
particular feature level we're upgrading to, we shouldn't even
hi Kevin
thanks for this KIP. It does offer the useful metrics.
One question: Have you considered adding the supported versions to the
metrics? Currently, admin#describeFeatures cannot return supported versions
for a specific server because we cannot configure the target server
handling the RPC r
Hey Jun,
Thanks for the comments.
1. I'll update the KIP. My trunk is a bit stale.
2. Yeah, the metric should report the finalized feature level for the
feature. And if it is not set, the metric will report 0.
3. I'll update the KIP with a timeline.
Thanks,
Kevin
On Wed, May 7, 2025 at 3:10 PM K
Hi, Kevin,
Thanks for the KIP. A few comments below.
1. For completeness, there is another feature streams.version.
2. "The default value of this metric should be the default value of the
feature." This metric reports the finalized level for each feature, right?
If a feature is not set, should t
Hi Kevin,
Thanks for the changes. For the "Rejected Alternatives" section you
can state why using "kafka-feature describe" is not sufficient.
Thanks,
--
-José
Hey Jose,
Thanks for the response. Yeah, the new metric should expose
metadata.version as well. Let me update the KIP to reflect that.
Thanks,
Kevin Wu
On Wed, May 7, 2025 at 2:54 PM Kevin Wu wrote:
> Hello all,
>
> I wrote a KIP to add a generic feature level metric.
> Here's the link:
> http
Hi Kevin,
Thanks for the improvement and discussion.
The "Monitoring" section says "Add a metric for each production
feature (i.e. kraft.version , transaction.version , group.version ,
eligible.leader.replicas.version , and share.version )." You mentioned
that the metadata version is already expo
Hello all,
I wrote a KIP to add a generic feature level metric.
Here's the link:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1180%3A+Add+a+generic+feature+level+metric
Thanks,
Kevin Wu
24 matches
Mail list logo
