[GitHub] [kafka] dajac commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-14 Thread via GitHub
dajac commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1326872198 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AbstractUniformAssignmentBuilder.java: ## @@ -0,0 +1,227 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] dajac commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-14 Thread via GitHub
dajac commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1326871177 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,413 @@ +/* + * Licensed to the Apache

[jira] [Resolved] (KAFKA-14503) Implement ListGroups

2023-09-14 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-14503. - Fix Version/s: 3.7.0 Reviewer: David Jacot Resolution: Fixed > Implement ListGro

[GitHub] [kafka] dajac merged pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-14 Thread via GitHub
dajac merged PR #14271: URL: https://github.com/apache/kafka/pull/14271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-14 Thread via GitHub
rreddy-22 commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1326838235 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,413 @@ +/* + * Licensed to the Ap

[GitHub] [kafka] dengziming commented on pull request #14272: KAFKA-15390: Do not return fenced broker in FetchResponse.preferredReplica

2023-09-14 Thread via GitHub
dengziming commented on PR #14272: URL: https://github.com/apache/kafka/pull/14272#issuecomment-1720724302 @cmccabe Yes, `metadataCache.getPartitionReplicaEndpoints` is only used when handling `FetchRequest`. -- This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-14 Thread via GitHub
rreddy-22 commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1326826466 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,413 @@ +/* + * Licensed to the Ap

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-14 Thread via GitHub
rreddy-22 commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1326819932 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,413 @@ +/* + * Licensed to the Ap

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-14 Thread via GitHub
rreddy-22 commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1326818424 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AbstractUniformAssignmentBuilder.java: ## @@ -0,0 +1,227 @@ +/* + * Licensed to the Apa

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-14 Thread via GitHub
rreddy-22 commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1326817433 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AbstractUniformAssignmentBuilder.java: ## @@ -0,0 +1,227 @@ +/* + * Licensed to the Apa

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-14 Thread via GitHub
rreddy-22 commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1326815225 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilderTest.java: ## @@ -0,0 +1,1090 @@ +/* + * Licensed to t

[GitHub] [kafka] drawxy commented on a diff in pull request #13847: KAFKA-15082: The log retention policy doesn't take effect after altering log dir

2023-09-14 Thread via GitHub
drawxy commented on code in PR #13847: URL: https://github.com/apache/kafka/pull/13847#discussion_r1326808988 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1808,7 +1809,10 @@ class ReplicaManager(val config: KafkaConfig, // pause cleaning for part

[GitHub] [kafka] cmccabe commented on pull request #14272: KAFKA-15390: Do not return fenced broker in FetchResponse.preferredReplica

2023-09-14 Thread via GitHub
cmccabe commented on PR #14272: URL: https://github.com/apache/kafka/pull/14272#issuecomment-1720696964 Thanks for the PR, @dengziming . If I understand correctly, the problem with returning a fenced broker from `metadataCache.getPartitionReplicaEndpoints` is that it will then be used as th

[GitHub] [kafka] philipnee commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-14 Thread via GitHub
philipnee commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1326800659 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataRequestManager.java: ## @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] github-actions[bot] commented on pull request #13796: KAFKA-14034 Idempotent producer should wait for preceding in-flight b…

2023-09-14 Thread via GitHub
github-actions[bot] commented on PR #13796: URL: https://github.com/apache/kafka/pull/13796#issuecomment-1720434737 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

[GitHub] [kafka] github-actions[bot] commented on pull request #13824: MINOR: Use Parametrized types correctly in RemoteLogMetadataSerde

2023-09-14 Thread via GitHub
github-actions[bot] commented on PR #13824: URL: https://github.com/apache/kafka/pull/13824#issuecomment-1720434714 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

[jira] [Commented] (KAFKA-15466) Add KIP-919 support for some admin APIs

2023-09-14 Thread Deng Ziming (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765426#comment-17765426 ] Deng Ziming commented on KAFKA-15466: - We can split this to some separate tickets fo

[jira] [Commented] (KAFKA-7739) Kafka Tiered Storage

2023-09-14 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765414#comment-17765414 ] Luke Chen commented on KAFKA-7739: -- This epic is created on Dec. 14, 2018 and is finally

[GitHub] [kafka] jolshan commented on a diff in pull request #14370: KAFKA-15449: Verify transactional offset commits (KIP-890 part 1)

2023-09-14 Thread via GitHub
jolshan commented on code in PR #14370: URL: https://github.com/apache/kafka/pull/14370#discussion_r1326648363 ## clients/src/main/java/org/apache/kafka/common/requests/TxnOffsetCommitResponse.java: ## @@ -47,6 +47,9 @@ * - {@link Errors#UNKNOWN_MEMBER_ID} * - {@link Err

[GitHub] [kafka] junrao commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-14 Thread via GitHub
junrao commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1326612426 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataRequestManager.java: ## @@ -0,0 +1,210 @@ +/* + * Licensed to the Apache Software Found

[jira] [Created] (KAFKA-15468) Prevent transaction coordinator reloads on already loaded leaders

2023-09-14 Thread Justine Olshan (Jira)
Justine Olshan created KAFKA-15468: -- Summary: Prevent transaction coordinator reloads on already loaded leaders Key: KAFKA-15468 URL: https://issues.apache.org/jira/browse/KAFKA-15468 Project: Kafka

[jira] [Updated] (KAFKA-15468) Prevent transaction coordinator reloads on already loaded leaders

2023-09-14 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan updated KAFKA-15468: --- Description: I was doing some research on txn coordinator loading and found that on a single

[GitHub] [kafka] junrao commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-14 Thread via GitHub
junrao commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1326602940 ## clients/src/main/java/org/apache/kafka/common/internals/IdempotentCloser.java: ## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] CalvinConfluent commented on a diff in pull request #14053: KAFKA-15221; Fix the race between fetch requests from a rebooted follower.

2023-09-14 Thread via GitHub
CalvinConfluent commented on code in PR #14053: URL: https://github.com/apache/kafka/pull/14053#discussion_r1326544485 ## core/src/main/scala/kafka/cluster/Replica.scala: ## @@ -98,14 +103,22 @@ class Replica(val brokerId: Int, val topicPartition: TopicPartition) extends Log

[GitHub] [kafka] CalvinConfluent commented on a diff in pull request #14053: KAFKA-15221; Fix the race between fetch requests from a rebooted follower.

2023-09-14 Thread via GitHub
CalvinConfluent commented on code in PR #14053: URL: https://github.com/apache/kafka/pull/14053#discussion_r1326544274 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -137,7 +137,8 @@ object Partition { delayedOperations = delayedOperations, metadataCac

[GitHub] [kafka] kirktrue commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-14 Thread via GitHub
kirktrue commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1326507379 ## clients/src/main/java/org/apache/kafka/common/internals/IdempotentCloser.java: ## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

[GitHub] [kafka] kirktrue commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-14 Thread via GitHub
kirktrue commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1326506750 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchBufferTest.java: ## @@ -0,0 +1,193 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] kirktrue commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-14 Thread via GitHub
kirktrue commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1326504874 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/CompletedFetchTest.java: ## @@ -115,13 +122,17 @@ public void testCommittedTransactionRecordsInclu

[GitHub] [kafka] kirktrue commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-14 Thread via GitHub
kirktrue commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1326504231 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractFetch.java: ## @@ -421,14 +309,45 @@ Node selectReadReplica(final TopicPartition partition

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-14 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1326478655 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,255 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-14 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1326463429 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,255 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] lianetm commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-14 Thread via GitHub
lianetm commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1326449328 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/TopicMetadataRequestManagerTest.java: ## @@ -0,0 +1,235 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] lianetm commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-14 Thread via GitHub
lianetm commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1326441512 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataRequestManager.java: ## @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [kafka] lucasbru commented on a diff in pull request #14226: KAFKA-15326: [8/N] Move consumer interaction out of processing methods

2023-09-14 Thread via GitHub
lucasbru commented on code in PR #14226: URL: https://github.com/apache/kafka/pull/14226#discussion_r1326423158 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/PartitionGroup.java: ## @@ -370,11 +374,30 @@ void clear() { nonEmptyQueuesByTime.clear(

[GitHub] [kafka] lucasbru commented on a diff in pull request #14226: KAFKA-15326: [8/N] Move consumer interaction out of processing methods

2023-09-14 Thread via GitHub
lucasbru commented on code in PR #14226: URL: https://github.com/apache/kafka/pull/14226#discussion_r1326421083 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java: ## @@ -590,6 +590,30 @@ public void prepareRecycle() { log.info("Closed

[GitHub] [kafka] lucasbru commented on a diff in pull request #14226: KAFKA-15326: [8/N] Move consumer interaction out of processing methods

2023-09-14 Thread via GitHub
lucasbru commented on code in PR #14226: URL: https://github.com/apache/kafka/pull/14226#discussion_r1326419704 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java: ## @@ -590,6 +590,30 @@ public void prepareRecycle() { log.info("Closed

[GitHub] [kafka] lucasbru commented on a diff in pull request #14226: KAFKA-15326: [8/N] Move consumer interaction out of processing methods

2023-09-14 Thread via GitHub
lucasbru commented on code in PR #14226: URL: https://github.com/apache/kafka/pull/14226#discussion_r1326415866 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java: ## @@ -590,6 +590,30 @@ public void prepareRecycle() { log.info("Closed

[jira] [Updated] (KAFKA-15467) On reconnect mm2 cannot find offset that exists in remote cluster and re-syncs the entire topic

2023-09-14 Thread Steve Jacobs (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Jacobs updated KAFKA-15467: - Description: I'm encountering an odd issue with mirrormaker2 with our remote replication setup

[jira] (KAFKA-15467) On reconnect mm2 cannot find offset that exists in remote cluster and re-syncs the entire topic

2023-09-14 Thread Steve Jacobs (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15467 ] Steve Jacobs deleted comment on KAFKA-15467: -- was (Author: steveatbat): Ok I've gone through this and the last 3 occurrences I can find, it's 12 topics each time that get reset. At differen

[jira] [Commented] (KAFKA-15467) On reconnect mm2 cannot find offset that exists in remote cluster and re-syncs the entire topic

2023-09-14 Thread Steve Jacobs (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765324#comment-17765324 ] Steve Jacobs commented on KAFKA-15467: -- Ok I've gone through this and the last 3 oc

[jira] [Created] (KAFKA-15467) On reconnect mm2 cannot find offset that exists in remote cluster and re-syncs the entire topic

2023-09-14 Thread Steve Jacobs (Jira)
Steve Jacobs created KAFKA-15467: Summary: On reconnect mm2 cannot find offset that exists in remote cluster and re-syncs the entire topic Key: KAFKA-15467 URL: https://issues.apache.org/jira/browse/KAFKA-15467

[GitHub] [kafka] philipnee opened a new pull request, #14391: MINOR: Implement toStringBase to avoid duplicated code

2023-09-14 Thread via GitHub
philipnee opened a new pull request, #14391: URL: https://github.com/apache/kafka/pull/14391 This is a follow up of https://github.com/apache/kafka/pull/14386. The goal of this PR is to refactor the toString method to reduce duplicated code. -- This is an automated message from the

[GitHub] [kafka] kirktrue commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-14 Thread via GitHub
kirktrue commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1326361661 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/CommitApplicationEvent.java: ## @@ -19,22 +19,23 @@ import org.apache.kafka.clients.consum

[GitHub] [kafka] junrao commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-14 Thread via GitHub
junrao commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1326276789 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/CompletedFetchTest.java: ## @@ -115,13 +122,17 @@ public void testCommittedTransactionRecordsInclude

[jira] [Assigned] (KAFKA-15452) Custom KafkaPrincipalBuilder Cannot Access SslPrincipalMapper

2023-09-14 Thread Raghu Baddam (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Baddam reassigned KAFKA-15452: Assignee: Raghu Baddam > Custom KafkaPrincipalBuilder Cannot Access SslPrincipalMapper >

[jira] [Commented] (KAFKA-15452) Custom KafkaPrincipalBuilder Cannot Access SslPrincipalMapper

2023-09-14 Thread Raghu Baddam (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765305#comment-17765305 ] Raghu Baddam commented on KAFKA-15452: -- Thanks [~mimaison], Let me take a look and

[GitHub] [kafka] mannoopj opened a new pull request, #14389: Metadata schema checker

2023-09-14 Thread via GitHub
mannoopj opened a new pull request, #14389: URL: https://github.com/apache/kafka/pull/14389 Check schema changes to ensure compatibility is not broken. Done through the use of comparing new json file to historical json file. Assume historical schemas are recursively compatible. -- This

[GitHub] [kafka] fvaleri commented on a diff in pull request #13201: KAFKA-14596: Move TopicCommand to tools

2023-09-14 Thread via GitHub
fvaleri commented on code in PR #13201: URL: https://github.com/apache/kafka/pull/13201#discussion_r1326292439 ## build.gradle: ## @@ -1880,9 +1880,10 @@ project(':tools:tools-api') { project(':tools') { archivesBaseName = "kafka-tools" - dependencies { +api libs.sc

[GitHub] [kafka] kirktrue closed pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-14 Thread via GitHub
kirktrue closed pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager URL: https://github.com/apache/kafka/pull/14359 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [kafka] arunmathew88 commented on pull request #11438: KAFKA-13403 Fix KafkaServer crashes when deleting topics due to the race in log deletion

2023-09-14 Thread via GitHub
arunmathew88 commented on PR #11438: URL: https://github.com/apache/kafka/pull/11438#issuecomment-1719780996 @divijvaidya If it helps, we have been running this patch in our primary cluster which has a lot of topic delete-recreates and has not had a related issue for almost 2 years. -- T

[GitHub] [kafka] philipnee commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-14 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1326232358 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/DefaultBackgroundThread.java: ## @@ -168,28 +171,41 @@ public DefaultBackgroundThread(final Time

[GitHub] [kafka] philipnee commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-14 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1326228037 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/DefaultBackgroundThread.java: ## @@ -168,28 +171,41 @@ public DefaultBackgroundThread(final Time

[GitHub] [kafka] philipnee commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-14 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1326205537 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,260 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [kafka] kirktrue commented on a diff in pull request #14384: KAFKA-15415 On producer-batch retry, skip-backoff on a new leader

2023-09-14 Thread via GitHub
kirktrue commented on code in PR #14384: URL: https://github.com/apache/kafka/pull/14384#discussion_r1326170737 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/ProducerBatch.java: ## @@ -94,9 +100,40 @@ public ProducerBatch(TopicPartition tp, MemoryRecordsB

[GitHub] [kafka] OmniaGM commented on a diff in pull request #13201: KAFKA-14596: Move TopicCommand to tools

2023-09-14 Thread via GitHub
OmniaGM commented on code in PR #13201: URL: https://github.com/apache/kafka/pull/13201#discussion_r1326193756 ## build.gradle: ## @@ -1880,9 +1880,10 @@ project(':tools:tools-api') { project(':tools') { archivesBaseName = "kafka-tools" - dependencies { +api libs.sc

[GitHub] [kafka] OmniaGM commented on a diff in pull request #13201: KAFKA-14596: Move TopicCommand to tools

2023-09-14 Thread via GitHub
OmniaGM commented on code in PR #13201: URL: https://github.com/apache/kafka/pull/13201#discussion_r1326193756 ## build.gradle: ## @@ -1880,9 +1880,10 @@ project(':tools:tools-api') { project(':tools') { archivesBaseName = "kafka-tools" - dependencies { +api libs.sc

[GitHub] [kafka] fvaleri commented on a diff in pull request #13201: KAFKA-14596: Move TopicCommand to tools

2023-09-14 Thread via GitHub
fvaleri commented on code in PR #13201: URL: https://github.com/apache/kafka/pull/13201#discussion_r1326137174 ## build.gradle: ## @@ -1880,9 +1880,10 @@ project(':tools:tools-api') { project(':tools') { archivesBaseName = "kafka-tools" - dependencies { +api libs.sc

[GitHub] [kafka] fvaleri commented on pull request #13201: KAFKA-14596: Move TopicCommand to tools

2023-09-14 Thread via GitHub
fvaleri commented on PR #13201: URL: https://github.com/apache/kafka/pull/13201#issuecomment-1719725886 > > Hi @OmniaGM, thanks. I left some comments. > > Additionally, this error message is different from previous implementation: > > ```shell > > ### OLD > > $ bin/kafka-topics.s

[jira] [Updated] (KAFKA-7739) Kafka Tiered Storage

2023-09-14 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-7739: -- Description: KIP:  [https://cwiki.apache.org/confluence/display/KAFKA/KIP-405%3A+Kafka+Tiered+S

[jira] [Updated] (KAFKA-7739) Kafka Tiered Storage

2023-09-14 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-7739: -- Labels: KIP-405 (was: needs-kip) > Kafka Tiered Storage > > >

[jira] [Comment Edited] (KAFKA-15221) Potential race condition between requests from rebooted followers

2023-09-14 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765241#comment-17765241 ] Satish Duggana edited comment on KAFKA-15221 at 9/14/23 3:50 PM: -

[jira] [Commented] (KAFKA-15221) Potential race condition between requests from rebooted followers

2023-09-14 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765241#comment-17765241 ] Satish Duggana commented on KAFKA-15221: Moving to 3.7.0 as code freeze is over.

[jira] [Updated] (KAFKA-15221) Potential race condition between requests from rebooted followers

2023-09-14 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-15221: --- Fix Version/s: 3.7.0 (was: 3.6.0) > Potential race condition between

[jira] [Commented] (KAFKA-15465) MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation

2023-09-14 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765240#comment-17765240 ] Satish Duggana commented on KAFKA-15465: Moving it to 3.7.0 as 3.6.0 code freeze

[jira] [Updated] (KAFKA-15465) MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation

2023-09-14 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-15465: --- Fix Version/s: 3.7.0 (was: 3.6.0) > MM2 not working when its internal

[GitHub] [kafka] dajac commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-14 Thread via GitHub
dajac commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1326105802 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,260 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-14 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1326089090 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,260 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] zhaohaidao commented on pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-14 Thread via GitHub
zhaohaidao commented on PR #14271: URL: https://github.com/apache/kafka/pull/14271#issuecomment-1719591231 > @zhaohaidao Thanks for the update and your patience. I just replied to your comments and left a few nits. The PR looks pretty good. We are almost there :). Hey @dajac The late

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-14 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1326031428 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,260 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-14 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1325999433 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/DefaultBackgroundThread.java: ## @@ -168,28 +171,41 @@ public DefaultBackgroundThread(final Time ti

[jira] [Resolved] (KAFKA-7739) Kafka Tiered Storage

2023-09-14 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana resolved KAFKA-7739. --- Resolution: Fixed > Kafka Tiered Storage > > > Key: KAFK

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-14 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1325999433 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/DefaultBackgroundThread.java: ## @@ -168,28 +171,41 @@ public DefaultBackgroundThread(final Time ti

[GitHub] [kafka] OmniaGM commented on pull request #14388: KAFKA-15465: Don't throw if MirrorMaker not authorized to create topics.

2023-09-14 Thread via GitHub
OmniaGM commented on PR #14388: URL: https://github.com/apache/kafka/pull/14388#issuecomment-1719474292 > @OmniaGM , Thank you for being very reactive. It seems `TopicAuthorizationException` is missing Added it now. -- This is an automated message from the Apache Git Service. To r

[GitHub] [kafka] ahibot commented on pull request #14388: KAFKA-15465: Don't throw if MirrorMaker not authorized to create topics.

2023-09-14 Thread via GitHub
ahibot commented on PR #14388: URL: https://github.com/apache/kafka/pull/14388#issuecomment-1719436507 @OmniaGM , Thank you for being very reactive. It seems `TopicAuthorizationException` is missing -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Updated] (KAFKA-15465) MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation

2023-09-14 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Omnia Ibrahim updated KAFKA-15465: -- Fix Version/s: 3.5.2 (was: 3.7.0) > MM2 not working when its internal t

[GitHub] [kafka] OmniaGM opened a new pull request, #14388: KAFKA-15465: Don't throw if MirrorMaker not authorized to create topics.

2023-09-14 Thread via GitHub
OmniaGM opened a new pull request, #14388: URL: https://github.com/apache/kafka/pull/14388 https://issues.apache.org/jira/browse/KAFKA-15465?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel MM2 should work without throwing errors if it's not authorised to create in

[jira] [Resolved] (KAFKA-15442) add document to introduce tiered storage feature and the usage

2023-09-14 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-15442. --- Resolution: Fixed > add document to introduce tiered storage feature and the usage > ---

[GitHub] [kafka] showuon commented on pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-14 Thread via GitHub
showuon commented on PR #14382: URL: https://github.com/apache/kafka/pull/14382#issuecomment-1719338018 Failed tests are unrelated: ``` Build / JDK 11 and Scala 2.13 / org.apache.kafka.connect.integration.ConnectorRestartApiIntegrationTest.testMultiWorkerRestartOnlyConnector

[GitHub] [kafka] showuon merged pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-14 Thread via GitHub
showuon merged PR #14382: URL: https://github.com/apache/kafka/pull/14382 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] (KAFKA-15465) MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation

2023-09-14 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15465 ] Omnia Ibrahim deleted comment on KAFKA-15465: --- was (Author: omnia_h_ibrahim): One thing I noticed is the test for `{{{}MirrorUtils.createCompactedTopic{}}}` [https://github.com/apache/kafk

[jira] [Comment Edited] (KAFKA-15465) MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation

2023-09-14 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765103#comment-17765103 ] Omnia Ibrahim edited comment on KAFKA-15465 at 9/14/23 10:59 AM: -

[jira] [Commented] (KAFKA-15465) MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation

2023-09-14 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765103#comment-17765103 ] Omnia Ibrahim commented on KAFKA-15465: --- One thing I noticed is the test for `{{{}

[jira] [Assigned] (KAFKA-15465) MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation

2023-09-14 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Omnia Ibrahim reassigned KAFKA-15465: - Assignee: Omnia Ibrahim > MM2 not working when its internal topics are pre-created on a

[jira] [Commented] (KAFKA-15465) MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation

2023-09-14 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765099#comment-17765099 ] Omnia Ibrahim commented on KAFKA-15465: --- Hi [~ahibot] , I re-checked the code of

[GitHub] [kafka] cadonna commented on pull request #12875: KAFKA-12679: Handle lock exceptions in state updater

2023-09-14 Thread via GitHub
cadonna commented on PR #12875: URL: https://github.com/apache/kafka/pull/12875#issuecomment-1719166276 @marcmesh Unfortunately, we did not manage to enable the state updater by default in 3.6. You can enable it with the internal `__state.updater.enabled__` set to true. However, be aware th

[GitHub] [kafka] OmniaGM commented on pull request #13204: KAFKA-14593: Move LeaderElectionCommand to tools

2023-09-14 Thread via GitHub
OmniaGM commented on PR #13204: URL: https://github.com/apache/kafka/pull/13204#issuecomment-1719152037 The test is failing for different packages https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13204/19/testReport/ and one of the tasks succeeded in building but failed with an er

[GitHub] [kafka] marcmesh commented on pull request #12875: KAFKA-12679: Handle lock exceptions in state updater

2023-09-14 Thread via GitHub
marcmesh commented on PR #12875: URL: https://github.com/apache/kafka/pull/12875#issuecomment-1719130457 > @andrashatvani Do you mean the state updater? I am currently working on enabling the state updater by default on trunk. We plan to officially release it in 3.6. However, you can alread

[GitHub] [kafka] dajac commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-14 Thread via GitHub
dajac commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1325561675 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AbstractUniformAssignmentBuilder.java: ## @@ -0,0 +1,227 @@ +/* + * Licensed to the Apache

[jira] [Commented] (KAFKA-15463) StreamsException: Accessing from an unknown node

2023-09-14 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765066#comment-17765066 ] Bruno Cadonna commented on KAFKA-15463: --- [~yevsh] Sorry to hear that you have issu

[GitHub] [kafka] showuon commented on pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-14 Thread via GitHub
showuon commented on PR #14382: URL: https://github.com/apache/kafka/pull/14382#issuecomment-1718998104 @tinaselenge , thanks for the comments. PR updated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [kafka] showuon commented on a diff in pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-14 Thread via GitHub
showuon commented on code in PR #14382: URL: https://github.com/apache/kafka/pull/14382#discussion_r1325573311 ## docs/ops.html: ## @@ -3859,6 +3859,98 @@ Finalizing the migration # Other configs ... + +6.11 Tiered Storage + +Tiered Storage Overview + +Kafka data is mostly

[GitHub] [kafka] tinaselenge commented on pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-14 Thread via GitHub
tinaselenge commented on PR #14382: URL: https://github.com/apache/kafka/pull/14382#issuecomment-1718963224 Left very minor comments which I'm happy if you want to address in subsequent PR. LGTM otherwise. -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [kafka] tinaselenge commented on a diff in pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-14 Thread via GitHub
tinaselenge commented on code in PR #14382: URL: https://github.com/apache/kafka/pull/14382#discussion_r1325541475 ## docs/ops.html: ## @@ -3859,6 +3859,98 @@ Finalizing the migration # Other configs ... + +6.11 Tiered Storage + +Tiered Storage Overview + +Kafka data is mo

[GitHub] [kafka] tinaselenge commented on a diff in pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-14 Thread via GitHub
tinaselenge commented on code in PR #14382: URL: https://github.com/apache/kafka/pull/14382#discussion_r1325542037 ## docs/ops.html: ## @@ -3859,6 +3859,98 @@ Finalizing the migration # Other configs ... + +6.11 Tiered Storage + +Tiered Storage Overview + +Kafka data is mo

[GitHub] [kafka] tinaselenge commented on a diff in pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-14 Thread via GitHub
tinaselenge commented on code in PR #14382: URL: https://github.com/apache/kafka/pull/14382#discussion_r1325541025 ## docs/ops.html: ## @@ -3859,6 +3859,98 @@ Finalizing the migration # Other configs ... + +6.11 Tiered Storage + +Tiered Storage Overview + +Kafka data is mo

[jira] [Commented] (KAFKA-15452) Custom KafkaPrincipalBuilder Cannot Access SslPrincipalMapper

2023-09-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765044#comment-17765044 ] Mickael Maison commented on KAFKA-15452: Hi [~raghu98...@gmail.com], I granted y

[jira] [Comment Edited] (KAFKA-15452) Custom KafkaPrincipalBuilder Cannot Access SslPrincipalMapper

2023-09-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765044#comment-17765044 ] Mickael Maison edited comment on KAFKA-15452 at 9/14/23 7:52 AM: -

[GitHub] [kafka] dajac commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-14 Thread via GitHub
dajac commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1325484973 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8603,6 +8614,86 @@ public void testHeartbeatDuringRebalanceCa