[jira] [Commented] (KAFKA-13331) Slow reassignments in 2.8 because of large number of UpdateMetadataResponseReceived(UpdateMetadataResponseData(errorCode=0),) Events

[ https://issues.apache.org/jira/browse/KAFKA-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868860#comment-17868860 ] GEORGE LI commented on KAFKA-13331: --- >From another environment, it looks like the slow

[jira] [Commented] (KAFKA-13331) Slow reassignments in 2.8 because of large number of UpdateMetadataResponseReceived(UpdateMetadataResponseData(errorCode=0),) Events

[ https://issues.apache.org/jira/browse/KAFKA-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423761#comment-17423761 ] GEORGE LI commented on KAFKA-13331: --- besides slowness of the reassignments, when the

[jira] [Created] (KAFKA-13331) Slow reassignments in 2.8 because of large number of UpdateMetadataResponseReceived(UpdateMetadataResponseData(errorCode=0),) Events

GEORGE LI created KAFKA-13331: - Summary: Slow reassignments in 2.8 because of large number of UpdateMetadataResponseReceived(UpdateMetadataResponseData(errorCode=0),) Events Key: KAFKA-13331 URL: https://issues.apac

[jira] [Commented] (KAFKA-12971) Kakfa 1.1.x clients cache broker hostnames, client stuck when host is swapped for the same broker.id

[ https://issues.apache.org/jira/browse/KAFKA-12971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17367008#comment-17367008 ] GEORGE LI commented on KAFKA-12971: --- [~ijuma] The Flink 1.9 can use 2.x kafka-clients

[jira] [Commented] (KAFKA-12971) Kakfa 1.1.x clients cache broker hostnames, client stuck when host is swapped for the same broker.id

[ https://issues.apache.org/jira/browse/KAFKA-12971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17366011#comment-17366011 ] GEORGE LI commented on KAFKA-12971: --- This issue is fixed in the {{1.1.x}} kafka client

[jira] [Created] (KAFKA-12971) Kakfa 1.1.x clients cache broker hostnames, client stuck when host is swapped for the same broker.id

GEORGE LI created KAFKA-12971: - Summary: Kakfa 1.1.x clients cache broker hostnames, client stuck when host is swapped for the same broker.id Key: KAFKA-12971 URL: https://issues.apache.org/jira/browse/KAFKA-12971

[jira] [Commented] (KAFKA-8733) Offline partitions occur when leader's disk is slow in reads while responding to follower fetch requests.

[ https://issues.apache.org/jira/browse/KAFKA-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17220243#comment-17220243 ] GEORGE LI commented on KAFKA-8733: -- for lossy clusters, setting unclean.leader.election

[jira] [Commented] (KAFKA-8638) Preferred Leader Blacklist (deprioritized list)

[ https://issues.apache.org/jira/browse/KAFKA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17111858#comment-17111858 ] GEORGE LI commented on KAFKA-8638: -- [~hai_lin] some of the recent activities about KIP-

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17103028#comment-17103028 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] hmm...That's weird. auto.leader.reba

[jira] [Comment Edited] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17086772#comment-17086772 ] GEORGE LI edited comment on KAFKA-4084 at 4/19/20, 7:28 AM: [

[jira] [Comment Edited] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17086772#comment-17086772 ] GEORGE LI edited comment on KAFKA-4084 at 4/19/20, 7:20 AM: [

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17086772#comment-17086772 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] I am not very familiar with 5.4 setup.

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077958#comment-17077958 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] Probably doing PLE with too many parti

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076124#comment-17076124 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] If this turns out to be positive in t

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076089#comment-17076089 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] I think it's still ok to set auto.lead

[jira] [Comment Edited] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076006#comment-17076006 ] GEORGE LI edited comment on KAFKA-4084 at 4/6/20, 7:13 AM: --- [~b

[jira] [Comment Edited] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076006#comment-17076006 ] GEORGE LI edited comment on KAFKA-4084 at 4/6/20, 2:26 AM: --- [~b

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076006#comment-17076006 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] Have some free time this weekend to tr

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072259#comment-17072259 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] Very busy with work and this COVID-19

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051509#comment-17051509 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] sure. I am working on porting the pat

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17048116#comment-17048116 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] [~sriharsha] Hi Evan, I am so sorry

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032598#comment-17032598 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] Which kafka version you are running?

[jira] [Comment Edited] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032267#comment-17032267 ] GEORGE LI edited comment on KAFKA-4084 at 2/7/20 10:02 AM: --- [~b

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032267#comment-17032267 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] [~junrao][~sriharsha] With `auto.lead

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17030183#comment-17030183 ] GEORGE LI commented on KAFKA-4084: -- [~evanjpw] One solution we have in-house is a too

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029299#comment-17029299 ] GEORGE LI commented on KAFKA-4084: -- [~blodsbror] We have implemented KIP-491 internally.

[jira] [Comment Edited] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029299#comment-17029299 ] GEORGE LI edited comment on KAFKA-4084 at 2/3/20 9:26 PM: -- [~blo

[jira] [Updated] (KAFKA-8903) allow the new replica (offset 0) to catch up with current leader using latest offset

[ https://issues.apache.org/jira/browse/KAFKA-8903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GEORGE LI updated KAFKA-8903: - Description: It very common (and sometimes frequent) that a broker has hardware failures (disk, memory,

[jira] [Updated] (KAFKA-8903) allow the new replica (offset 0) to catch up with current leader using latest offset

[ https://issues.apache.org/jira/browse/KAFKA-8903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GEORGE LI updated KAFKA-8903: - Description: It very common (and sometimes frequent) that a broker has hardware failures (disk, memory,

[jira] [Commented] (KAFKA-8903) allow the new replica (offset 0) to catch up with current leader using latest offset

[ https://issues.apache.org/jira/browse/KAFKA-8903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928906#comment-16928906 ] GEORGE LI commented on KAFKA-8903: -- e.g. {code:java} /usr/lib/kafka/bin/kafka-configs

[jira] [Created] (KAFKA-8903) allow the new replica (offset 0) to catch up with current leader using latest offset

GEORGE LI created KAFKA-8903: Summary: allow the new replica (offset 0) to catch up with current leader using latest offset Key: KAFKA-8903 URL: https://issues.apache.org/jira/browse/KAFKA-8903 Project: K

[jira] [Commented] (KAFKA-4084) automated leader rebalance causes replication downtime for clusters with too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900461#comment-16900461 ] GEORGE LI commented on KAFKA-4084: -- [~jiaxinye] does the patch from https://issues.apac

[jira] [Commented] (KAFKA-8725) Improve LogCleaner error handling when failing to grab the filthiest log

[ https://issues.apache.org/jira/browse/KAFKA-8725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16899237#comment-16899237 ] GEORGE LI commented on KAFKA-8725: -- We saw similar issues. After the thread die, the "

[jira] [Resolved] (KAFKA-8663) partition assignment would be better original_assignment + new_reassignment during reassignments

[ https://issues.apache.org/jira/browse/KAFKA-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GEORGE LI resolved KAFKA-8663. -- Resolution: Won't Fix looks like RAR + OAR is required for KIP-455 to preserve the targetReplicas ex

[jira] [Commented] (KAFKA-8638) Preferred Leader Blacklist (deprioritized list)

[ https://issues.apache.org/jira/browse/KAFKA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884787#comment-16884787 ] GEORGE LI commented on KAFKA-8638: -- Here is the KIP: [KIP-491|https://cwiki.apache.org

[jira] [Commented] (KAFKA-8663) partition assignment would be better original_assignment + new_reassignment during reassignments

[ https://issues.apache.org/jira/browse/KAFKA-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16884526#comment-16884526 ] GEORGE LI commented on KAFKA-8663: -- As we can see from the original comments of the code

[jira] [Updated] (KAFKA-8663) partition assignment would be better original_assignment + new_reassignment during reassignments

[ https://issues.apache.org/jira/browse/KAFKA-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GEORGE LI updated KAFKA-8663: - Description: >From my observation/experience during reassignment, the partition assignment >replica ord

[jira] [Created] (KAFKA-8663) partition assignment would be better original_assignment + new_reassignment during reassignments

GEORGE LI created KAFKA-8663: Summary: partition assignment would be better original_assignment + new_reassignment during reassignments Key: KAFKA-8663 URL: https://issues.apache.org/jira/browse/KAFKA-8663

[jira] [Commented] (KAFKA-8638) Preferred Leader Blacklist (deprioritized list)

[ https://issues.apache.org/jira/browse/KAFKA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16881480#comment-16881480 ] GEORGE LI commented on KAFKA-8638: -- Hi Jose, Because a broker can have hundreds/thousan

[jira] [Updated] (KAFKA-8638) Preferred Leader Blacklist (deprioritized list)

[ https://issues.apache.org/jira/browse/KAFKA-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] GEORGE LI updated KAFKA-8638: - Description: Currently, the kafka preferred leader election will pick the broker_id in the topic/partiti

[jira] [Created] (KAFKA-8638) Preferred Leader Blacklist (deprioritized list)

GEORGE LI created KAFKA-8638: Summary: Preferred Leader Blacklist (deprioritized list) Key: KAFKA-8638 URL: https://issues.apache.org/jira/browse/KAFKA-8638 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-8527) add dynamic maintenance broker config

[ https://issues.apache.org/jira/browse/KAFKA-8527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16868914#comment-16868914 ] GEORGE LI commented on KAFKA-8527: -- We had this implemented in a separate admin service

[jira] [Commented] (KAFKA-6794) Support for incremental replica reassignment

[ https://issues.apache.org/jira/browse/KAFKA-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777257#comment-16777257 ] GEORGE LI commented on KAFKA-6794: -- Hi [~viktorsomogyi], I think this "Incremental Re

[jira] [Comment Edited] (KAFKA-6794) Support for incremental replica reassignment

[ https://issues.apache.org/jira/browse/KAFKA-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777149#comment-16777149 ] GEORGE LI edited comment on KAFKA-6794 at 2/25/19 7:56 PM: --- Hi,

[jira] [Commented] (KAFKA-6794) Support for incremental replica reassignment

[ https://issues.apache.org/jira/browse/KAFKA-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777149#comment-16777149 ] GEORGE LI commented on KAFKA-6794: -- Hi, [~viktorsomogyi], When I rebalance the whole c

[jira] [Comment Edited] (KAFKA-6794) Support for incremental replica reassignment

[ https://issues.apache.org/jira/browse/KAFKA-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775598#comment-16775598 ] GEORGE LI edited comment on KAFKA-6794 at 2/22/19 9:21 PM: --- I a

[jira] [Commented] (KAFKA-6794) Support for incremental replica reassignment

[ https://issues.apache.org/jira/browse/KAFKA-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775598#comment-16775598 ] GEORGE LI commented on KAFKA-6794: -- I also have seen this issue.  When more than one bro

[jira] [Commented] (KAFKA-7854) Behavior change in controller picking up partition reassignment tasks since 1.1.0

[ https://issues.apache.org/jira/browse/KAFKA-7854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773395#comment-16773395 ] GEORGE LI commented on KAFKA-7854: -- [KIP-236|https://cwiki.apache.org/confluence/display

[jira] [Commented] (KAFKA-6304) The controller should allow updating the partition reassignment for the partitions being reassigned

[ https://issues.apache.org/jira/browse/KAFKA-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773394#comment-16773394 ] GEORGE LI commented on KAFKA-6304: -- [KIP-236|https://cwiki.apache.org/confluence/display

[jira] [Commented] (KAFKA-6359) Work for KIP-236

[ https://issues.apache.org/jira/browse/KAFKA-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773351#comment-16773351 ] GEORGE LI commented on KAFKA-6359: -- New Pull Request:   https://github.com/apache/kafka/