[jira] [Commented] (KAFKA-12713) Report "REAL" follower/consumer fetch latency

[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17390068#comment-17390068 ] Ming Liu commented on KAFKA-12713: -- Certainly, [~kaihuang], please take the ownership o

[jira] [Commented] (KAFKA-12713) Report "REAL" follower/consumer fetch latency

[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342778#comment-17342778 ] Ming Liu commented on KAFKA-12713: -- Here is PR [https://github.com/apache/kafka/pull/10

[jira] [Comment Edited] (KAFKA-12713) Report "REAL" follower/consumer fetch latency

[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331590#comment-17331590 ] Ming Liu edited comment on KAFKA-12713 at 4/27/21, 3:25 AM:

[jira] [Comment Edited] (KAFKA-12713) Report "REAL" follower/consumer fetch latency

[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331590#comment-17331590 ] Ming Liu edited comment on KAFKA-12713 at 4/27/21, 12:17 AM: -

[jira] [Updated] (KAFKA-12713) Report "REAL" follower/consumer fetch latency

[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Liu updated KAFKA-12713: - Summary: Report "REAL" follower/consumer fetch latency (was: Report "REAL" broker/consumer fetch latenc

[jira] [Comment Edited] (KAFKA-12713) Report "REAL" broker/consumer fetch latency

[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331590#comment-17331590 ] Ming Liu edited comment on KAFKA-12713 at 4/25/21, 11:16 PM: -

[jira] [Comment Edited] (KAFKA-12713) Report "REAL" broker/consumer fetch latency

[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331590#comment-17331590 ] Ming Liu edited comment on KAFKA-12713 at 4/25/21, 5:33 PM:

[jira] [Commented] (KAFKA-12713) Report "REAL" broker/consumer fetch latency

[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331590#comment-17331590 ] Ming Liu commented on KAFKA-12713: -- The idea is like this,  we can: 1. Add waitTimeMS i

[jira] [Updated] (KAFKA-12713) Report "REAL" broker/consumer fetch latency

[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Liu updated KAFKA-12713: - Summary: Report "REAL" broker/consumer fetch latency (was: Report broker/consumer fetch latency more ac

[jira] [Created] (KAFKA-12713) Report broker/consumer fetch latency more accurately

Ming Liu created KAFKA-12713: Summary: Report broker/consumer fetch latency more accurately Key: KAFKA-12713 URL: https://issues.apache.org/jira/browse/KAFKA-12713 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-7918) Streams store cleanup: inline byte-store generic parameters

[ https://issues.apache.org/jira/browse/KAFKA-7918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17240908#comment-17240908 ] Ming Liu commented on KAFKA-7918: - We also had the code to optimize the changelog support

[jira] [Commented] (KAFKA-7918) Streams store cleanup: inline byte-store generic parameters

[ https://issues.apache.org/jira/browse/KAFKA-7918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237861#comment-17237861 ] Ming Liu commented on KAFKA-7918: - [~ableegoldman], [~guozhang] [~mjsax],  one good benef

[jira] [Commented] (KAFKA-8733) Offline partitions occur when leader's disk is slow in reads while responding to follower fetch requests.

[ https://issues.apache.org/jira/browse/KAFKA-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17220139#comment-17220139 ] Ming Liu commented on KAFKA-8733: - One observation is after moving to 2.5 (so  _[replica.

[jira] [Commented] (KAFKA-10398) Intra-broker disk move failed with onPartitionFenced()

[ https://issues.apache.org/jira/browse/KAFKA-10398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177132#comment-17177132 ] Ming Liu commented on KAFKA-10398: -- [~lukech], that is another problem blocking the dis

[jira] [Updated] (KAFKA-10398) Intra-broker disk move failed with onPartitionFenced()

[ https://issues.apache.org/jira/browse/KAFKA-10398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Liu updated KAFKA-10398: - Description: When I tried the intra-broker disk move on 2.5.0, it always failed quickly in onPartitionF

[jira] [Created] (KAFKA-10398) Intra-broker disk move failed with onPartitionFenced()

Ming Liu created KAFKA-10398: Summary: Intra-broker disk move failed with onPartitionFenced() Key: KAFKA-10398 URL: https://issues.apache.org/jira/browse/KAFKA-10398 Project: Kafka Issue Type: Bu

[jira] [Commented] (KAFKA-8362) LogCleaner gets stuck after partition move between log directories

[ https://issues.apache.org/jira/browse/KAFKA-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177123#comment-17177123 ] Ming Liu commented on KAFKA-8362: - Awesome, what is the patch link? > LogCleaner gets st

[jira] [Commented] (KAFKA-8362) LogCleaner gets stuck after partition move between log directories

[ https://issues.apache.org/jira/browse/KAFKA-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176476#comment-17176476 ] Ming Liu commented on KAFKA-8362: - In that case, we need to modify *allCleanerCheckpoints

[jira] [Comment Edited] (KAFKA-8733) Offline partitions occur when leader's disk is slow in reads while responding to follower fetch requests.

[ https://issues.apache.org/jira/browse/KAFKA-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17098264#comment-17098264 ] Ming Liu edited comment on KAFKA-8733 at 7/25/20, 3:03 PM: --- We

[jira] [Commented] (KAFKA-10242) Adding metrics to track the total count of idempotent producers that Broker need to track

[ https://issues.apache.org/jira/browse/KAFKA-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153091#comment-17153091 ] Ming Liu commented on KAFKA-10242: -- Yes, correct, then it should be 2.7.0 instead of 2.

[jira] [Created] (KAFKA-10242) Adding metrics to track the total count of idempotent producers that Broker need to track

Ming Liu created KAFKA-10242: Summary: Adding metrics to track the total count of idempotent producers that Broker need to track Key: KAFKA-10242 URL: https://issues.apache.org/jira/browse/KAFKA-10242 Pro

[jira] [Commented] (KAFKA-10242) Adding metrics to track the total count of idempotent producers that Broker need to track

[ https://issues.apache.org/jira/browse/KAFKA-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152494#comment-17152494 ] Ming Liu commented on KAFKA-10242: -- The PR is [https://github.com/apache/kafka/pull/898

[jira] [Commented] (KAFKA-8362) LogCleaner gets stuck after partition move between log directories

[ https://issues.apache.org/jira/browse/KAFKA-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17135400#comment-17135400 ] Ming Liu commented on KAFKA-8362: -  In 2.5 Kafka, this still easily repro for me. With c

[jira] [Commented] (KAFKA-8733) Offline partitions occur when leader's disk is slow in reads while responding to follower fetch requests.

[ https://issues.apache.org/jira/browse/KAFKA-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17098264#comment-17098264 ] Ming Liu commented on KAFKA-8733: - We (at Twitter) also saw this issue almost every month

[jira] [Commented] (KAFKA-8325) Remove from the incomplete set failed. This should be impossible

[ https://issues.apache.org/jira/browse/KAFKA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16838718#comment-16838718 ] Ming Liu commented on KAFKA-8325: - We also run into this problem after we upgrade to Kafk

[jira] [Commented] (KAFKA-7698) Kafka Broker fail to start: ProducerFencedException thrown from producerstatemanager.scala!checkProducerEpoch

[ https://issues.apache.org/jira/browse/KAFKA-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16707621#comment-16707621 ] Ming Liu commented on KAFKA-7698: - The problem is seems coming from appendEndTxnMarker of

[jira] [Created] (KAFKA-7698) Kafka Broker fail to start: ProducerFencedException thrown from producerstatemanager.scala!checkProducerEpoch

Ming Liu created KAFKA-7698: --- Summary: Kafka Broker fail to start: ProducerFencedException thrown from producerstatemanager.scala!checkProducerEpoch Key: KAFKA-7698 URL: https://issues.apache.org/jira/browse/KAFKA-7698

[jira] [Commented] (KAFKA-7693) "IllegalArgumentException: Invalid negative sequence number used" in Kafka Client

[ https://issues.apache.org/jira/browse/KAFKA-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16706463#comment-16706463 ] Ming Liu commented on KAFKA-7693: - With some further debugging, the bug seems at client T

[jira] [Updated] (KAFKA-7693) "IllegalArgumentException: Invalid negative sequence number used" in Kafka Client

[ https://issues.apache.org/jira/browse/KAFKA-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Liu updated KAFKA-7693: Description: When operating Kafka, we have seen Kafka client logging this kind of exception: {noformat} o

[jira] [Comment Edited] (KAFKA-7692) updateFirstUnstableOffset NPE due to sequenceId overflow in ProducerStateManager.append

[ https://issues.apache.org/jira/browse/KAFKA-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16706462#comment-16706462 ] Ming Liu edited comment on KAFKA-7692 at 12/2/18 8:33 PM: -- Upon

[jira] [Created] (KAFKA-7693) "IllegalArgumentException: Invalid negative sequence number used" in Kafka Client

Ming Liu created KAFKA-7693: --- Summary: "IllegalArgumentException: Invalid negative sequence number used" in Kafka Client Key: KAFKA-7693 URL: https://issues.apache.org/jira/browse/KAFKA-7693 Project: Kafka

[jira] [Updated] (KAFKA-7692) updateFirstUnstableOffset NPE due to sequenceId overflow in ProducerStateManager.append

[ https://issues.apache.org/jira/browse/KAFKA-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Liu updated KAFKA-7692: Description: When operating Kafka, we frequently saw this exception on Kafka server log,  {noformat} ERRO

[jira] [Commented] (KAFKA-7692) updateFirstUnstableOffset NPE due to sequenceId overflow in ProducerStateManager.append

[ https://issues.apache.org/jira/browse/KAFKA-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16706462#comment-16706462 ] Ming Liu commented on KAFKA-7692: - Upon the close analysis, the bug is in ProducerStateMa

[jira] [Created] (KAFKA-7692) updateFirstUnstableOffset NPE due to sequenceId overflow in ProducerStateManager.append

Ming Liu created KAFKA-7692: --- Summary: updateFirstUnstableOffset NPE due to sequenceId overflow in ProducerStateManager.append Key: KAFKA-7692 URL: https://issues.apache.org/jira/browse/KAFKA-7692 Project: