Hello everyone,
I believe now this KIP is ready for a vote. I will be starting the vote
here momentarily.
Thanks again,
Brenden
On Wed, Jul 24, 2024 at 11:01 AM Brenden Deluna
wrote:
> Hi Mickael,
> Thank you for your feedback.
>
> 1. I can see that, I will get that changed.
> 2. Done
>
> Than
Hi Mickael,
Thank you for your feedback.
1. I can see that, I will get that changed.
2. Done
Thanks,
Brenden
On Wed, Jul 24, 2024 at 5:06 AM Mickael Maison
wrote:
> Hi,
>
> 1. The title is a bit misleading. It's proposing to add new metrics,
> JMX is just one of the mechanisms to export them.
Hi,
3. Sorry I misunderstood something and got confused as the KIP
mentions AsyncKafkaConsumer as a new consumer implementation.
I now see that it's not an alternative to KafkaConsumer but just the
"backend" used with the new protocol. So yes any new metrics added are
public API and I agree AsyncK
Hi Mickael,
1. I agree that the title is misleading. It should be something like
"New metrics for the AsyncKafkaConsumer". Maybe it should even be
"Metrics for the new consumer".
3. I am not sure I understand this comment. Exposed metrics are public
as far as I understand. So adding new metri
Hi,
1. The title is a bit misleading. It's proposing to add new metrics,
JMX is just one of the mechanisms to export them.
2. +1 to not register the new metrics when using the classic consumer,
instead of setting them to 0. Similarly I assume existing metrics that
don't apply to the new consumers
Hi,
I echo the commend about symmetry. I expect additional use of the background
events in AK 4.0, so adding these metrics while we are in here makes a lot of
sense
for that reason too.
Thanks,
Andrew
> On 23 Jul 2024, at 22:18, Lianet M. wrote:
>
> Hey all,
>
> Follow-up on Bruno's point BC2.
Hey all,
Follow-up on Bruno's point BC2. I personally did not suggest
background-event-queue-time-max and background-event-queue-time-avg mainly
because in practice we only have 2 types of events flowing from the
background to the app thread: errorEvents and callbackEvents, (vs the many
api-trigge
Hi Bruno,
Thank you for your comments.
BC1. I think that is a great point, I was not aware that we could not add
the metrics based on which protocol is being used. I will update the KIP to
reflect that change.
BC2. There is not any specific reason for this, really it has just never
been suggested
Hi Brenden,
BC1. In his first e-mail Andrew wrote "I would expect that the metrics
do not exist at all". I agree with him. I think it would be better to
not add those metrics at all if the CLASSIC protocol is used rather than
the metrics exist and are all constant 0. This should be possible by
Hi Apoorv,
Thank you for your comments, I will address each.
AM1. I can see the usefulness in also having an
'application-event-queue-age-max' to get an idea of outliers and how they
may be affecting the average metric. I will add that.
AM2. I agree with you there, I think 'time' is a better desc
Hi Brendan,
Thanks for the KIP. The metrics are always helpful.
AM1: Is `application-event-queue-age-avg` enough or do we require `
application-event-queue-age-max` as well to differentiate with outliers?
AM2: The kafka producer defines metric `record-queue-time-avg` which
captures the time spent
Hi Andrew,
Thank you for your reply.
AS4. Adding '.ms' was a suggestion that was brought up but not required in
any way. After going back and looking at our current set of metrics,
specifically for the consumer, it seems that only metrics measured in
nanoseconds include its unit in the metric name
Hi Brenden,
Thanks for the updates.
AS4. I see that you’ve added `.ms` to a bunch of the metrics reflecting the
fact that they’re measured in milliseconds. However, I observe that most metrics
in Kafka that are measured in milliseconds, with some exceptions in Kafka
Connect
and MirrorMaker do not
Hey Lianet,
Thank you for your suggestions and feedback!
LM1. This has now been addressed.
LM2. I think that would be a valuable addition to the current set of
metrics, I will get that added.
LM3. Again great idea, that would certainly be helpful. Will add that as
well.
Let me know if you
Hi Lucas,
Thank you for the feedback! I have addressed your comments:
LB1. Good catch there, I will update the names as needed.
LB2. Good catch again! I will update the name to be more consistent.
LB3. Thank you for pointing this out, I realized that all metric values
will actually be set to
Hey Brenden, thanks for the KIP! Great to get more visibility into the new
consumer.
LM1. +1 on Lucas's suggestion for including the unit in the name, seems
clearer and consistent (I do see several time metrics including ms)
LM2. What about a new metric for application-event-queue-time-ms. It wou
Hey Brenden,
thanks for the KIP! These will be great to observe and debug the
background thread of the new consumer.
LB1. `time-between-network-thread-poll-max` → I see several similar
metrics including the unit in the metric name (ms or us). We could
consider this, although it's probably not str
Hi Andrew,
Thank you for the feedback and your question.
AS1. Great idea, I will get that added.
AS2. For unsent-events-age-max, age will be calculated once the event is
sent, so you are correct.
AS3. I agree, I think that would be a helpful metric to add, thank you! I
will get that added.
Plea
Hi Brenden,
Thanks for the KIP. It fills a gap in the metrics for the new consumer nicely.
AS1. If using the CLASSIC protocol, and thus the LegacyKafkaConsumer,
I would expect that the metrics do not exist at all. Maybe say something like
“These metrics are for the new consumer implementation usin
Hi all,
This is the link to the KIP document.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1068%3A+New+JMX+metrics+for+the+new+KafkaConsumer
Any comment is appreciated,
On Tue, Jul 9, 2024 at 10:14 AM Brenden Deluna
wrote:
> Hello everyone,
>
> I would like to start the discussion th
Hello everyone,
I would like to start the discussion thread for KIP-1068. This is a
relatively small KIP, only proposing to add a couple of new metrics.
If you have any suggestions or feedback, let me know, it will be much
appreciated.
Hello everyone,
I would like to start the discussion thread for KIP-1068. This is a
relatively small KIP, only proposing to add a couple of new metrics.
If you have any suggestions or feedback, let me know, it will be much
appreciated.
22 matches
Mail list logo
