[jira] [Updated] (KAFKA-15321) Document consumer group member state machine

2023-10-16 Thread Yi Ding (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Ding updated KAFKA-15321: Labels: kip-848 kip-848-client-support (was: kip-848 kip-848-client-support kip-848-preview) > Document

Re: [PR] KAFKA-15390: Do not return fenced broker in FetchResponse.preferredReplica [kafka]

2023-10-16 Thread via GitHub
showuon commented on PR #14272: URL: https://github.com/apache/kafka/pull/14272#issuecomment-1765720506 Thanks for the fix @dengziming ! Nice catch! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] KAFKA-14884: Include check transaction is still ongoing right before append (take 2) [kafka]

2023-10-16 Thread via GitHub
artemlivshits commented on code in PR #13787: URL: https://github.com/apache/kafka/pull/13787#discussion_r1361539171 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -1279,7 +1283,7 @@ class Partition(val topicPartition: TopicPartition, } def appendRecordsToLe

Re: [PR] [WIP]KAFKA-15444: Native docker image [kafka]

2023-10-16 Thread via GitHub
VedarthConfluent commented on code in PR #14556: URL: https://github.com/apache/kafka/pull/14556#discussion_r1361495859 ## docker/test/requirements.txt: ## @@ -0,0 +1,6 @@ +confluent_kafka Review Comment: @ijuma Confluent Kafka for Python - https://github.com/confluentinc/c

Re: [PR] [WIP]KAFKA-15444: Native docker image [kafka]

2023-10-16 Thread via GitHub
ijuma commented on code in PR #14556: URL: https://github.com/apache/kafka/pull/14556#discussion_r1361488481 ## docker/test/requirements.txt: ## @@ -0,0 +1,6 @@ +confluent_kafka Review Comment: But why would we depend on a confluent kafka artifact. -- This is an automate

Re: [PR] [WIP]KAFKA-15444: Native docker image [kafka]

2023-10-16 Thread via GitHub
VedarthConfluent commented on code in PR #14556: URL: https://github.com/apache/kafka/pull/14556#discussion_r1361487179 ## docker/test/requirements.txt: ## @@ -0,0 +1,6 @@ +confluent_kafka Review Comment: It seems like it's a typo. Updating it to `confluent-kafka` -- Thi

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-16 Thread via GitHub
philipnee commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1361484401 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-16 Thread via GitHub
philipnee commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1361483615 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] KAFKA-15445: [WIP] Add JVM Docker image [kafka]

2023-10-16 Thread via GitHub
VedarthConfluent commented on code in PR #14552: URL: https://github.com/apache/kafka/pull/14552#discussion_r1361482780 ## docker/jvm/Dockerfile: ## @@ -0,0 +1,78 @@ +### +# Licensed to the Apache Softw

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-16 Thread via GitHub
philipnee commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1361482764 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,263 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-16 Thread via GitHub
philipnee commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1361480080 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -248,4 +261,28 @@ public void updateAssignment(ConsumerGroupHe

Re: [PR] KAFKA-15445: [WIP] Add JVM Docker image [kafka]

2023-10-16 Thread via GitHub
ijuma commented on code in PR #14552: URL: https://github.com/apache/kafka/pull/14552#discussion_r1361476400 ## docker/jvm/Dockerfile: ## @@ -0,0 +1,78 @@ +### +# Licensed to the Apache Software Foundat

[jira] [Commented] (KAFKA-15619) Deleted topics will come back again

2023-10-16 Thread Deng Ziming (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17776006#comment-17776006 ] Deng Ziming commented on KAFKA-15619: - Hello [~showuon] , As [~ijuma] mentioned, aut

Re: [PR] [WIP]KAFKA-15444: Native docker image [kafka]

2023-10-16 Thread via GitHub
ijuma commented on code in PR #14556: URL: https://github.com/apache/kafka/pull/14556#discussion_r1361476173 ## docker/test/requirements.txt: ## @@ -0,0 +1,6 @@ +confluent_kafka Review Comment: This doesn't seem right. -- This is an automated message from the Apache Git

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-16 Thread via GitHub
philipnee commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1361474872 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -135,6 +146,7 @@ public void updateState(ConsumerGroupHeartbeat

Re: [PR] [WIP]KAFKA-15444: Native docker image [kafka]

2023-10-16 Thread via GitHub
ijuma commented on code in PR #14556: URL: https://github.com/apache/kafka/pull/14556#discussion_r1361474850 ## core/src/main/scala/kafka/Kafka.scala: ## @@ -86,6 +86,10 @@ object Kafka extends Logging { } def main(args: Array[String]): Unit = { +process(args) + } +

Re: [PR] KAFKA-13152: Kip 770 buffer size fix [kafka]

2023-10-16 Thread via GitHub
github-actions[bot] commented on PR #13283: URL: https://github.com/apache/kafka/pull/13283#issuecomment-1765609193 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

Re: [PR] KAFKA-15204: Re-add throttling rate to KafkaConfig [kafka]

2023-10-16 Thread via GitHub
github-actions[bot] commented on PR #14002: URL: https://github.com/apache/kafka/pull/14002#issuecomment-1765608912 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

Re: [PR] KAFKA-15481: Fix concurrency bug in RemoteIndexCache [kafka]

2023-10-16 Thread via GitHub
showuon commented on code in PR #14483: URL: https://github.com/apache/kafka/pull/14483#discussion_r1361468155 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -193,7 +192,16 @@ public File cacheDir() { public void remove(Uuid key

Re: [PR] KAFKA-14884: Include check transaction is still ongoing right before append (take 2) [kafka]

2023-10-16 Thread via GitHub
ijuma commented on code in PR #13787: URL: https://github.com/apache/kafka/pull/13787#discussion_r1361468366 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -1279,7 +1283,7 @@ class Partition(val topicPartition: TopicPartition, } def appendRecordsToLeader(rec

Re: [PR] KAFKA-15481: Fix concurrency bug in RemoteIndexCache [kafka]

2023-10-16 Thread via GitHub
showuon commented on code in PR #14483: URL: https://github.com/apache/kafka/pull/14483#discussion_r1361468155 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -193,7 +192,16 @@ public File cacheDir() { public void remove(Uuid key

Re: [PR] KAFKA-14884: Include check transaction is still ongoing right before append (take 2) [kafka]

2023-10-16 Thread via GitHub
ijuma commented on code in PR #13787: URL: https://github.com/apache/kafka/pull/13787#discussion_r1361468169 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -1279,7 +1283,7 @@ class Partition(val topicPartition: TopicPartition, } def appendRecordsToLeader(rec

[jira] [Commented] (KAFKA-15619) Deleted topics will come back again

2023-10-16 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775996#comment-17775996 ] Luke Chen commented on KAFKA-15619: --- So, in this case, I think it works as designed. F

Re: [PR] MINOR: Rewrite/Move KafkaNetworkChannel to the `raft` module [kafka]

2023-10-16 Thread via GitHub
ijuma merged PR #14559: URL: https://github.com/apache/kafka/pull/14559 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] MINOR: Rewrite/Move KafkaNetworkChannel to the `raft` module [kafka]

2023-10-16 Thread via GitHub
ijuma commented on PR #14559: URL: https://github.com/apache/kafka/pull/14559#issuecomment-1765591706 JDK 11 build passed, unrelated failures: > Build / JDK 8 and Scala 2.12 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationExactlyOnceTest.testOffsetTranslationB

Re: [PR] KAFKA-15534: Inject request completion time when the request failed [kafka]

2023-10-16 Thread via GitHub
philipnee commented on code in PR #14532: URL: https://github.com/apache/kafka/pull/14532#discussion_r1361282001 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -184,13 +184,11 @@ private NetworkClientDelegate.UnsentReques

[jira] [Updated] (KAFKA-15619) Deleted topics will come back again

2023-10-16 Thread Deng Ziming (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deng Ziming updated KAFKA-15619: Description: Deleted topics will come back again in Apache Spark structured streaming stress test

[jira] [Commented] (KAFKA-15619) Deleted topics will come back again

2023-10-16 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775991#comment-17775991 ] Ismael Juma commented on KAFKA-15619: - auto.create.topics.enable means that _metadat

[jira] [Commented] (KAFKA-15619) Deleted topics will come back again

2023-10-16 Thread Deng Ziming (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775988#comment-17775988 ] Deng Ziming commented on KAFKA-15619: - [~showuon] Yes, the stress test is using defa

[jira] [Commented] (KAFKA-15602) Breaking change in 3.4.0 ByteBufferSerializer

2023-10-16 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775986#comment-17775986 ] Philip Nee commented on KAFKA-15602: Hi [~luke.kirby]  I think since the PR didn't

[jira] [Commented] (KAFKA-15619) Deleted topics will come back again

2023-10-16 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775985#comment-17775985 ] Luke Chen commented on KAFKA-15619: --- [~dengziming], I'd like to get more info why the

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-16 Thread via GitHub
lianetm commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1361441480 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -248,4 +261,28 @@ public void updateAssignment(ConsumerGroupHear

Re: [PR] KAFKA-7109: Close fetch sessions on close of consumer [kafka]

2023-10-16 Thread via GitHub
showuon commented on PR #12590: URL: https://github.com/apache/kafka/pull/12590#issuecomment-1765558756 @dengziming , let's discuss it in the [KAFKA-15619](https://issues.apache.org/jira/browse/KAFKA-15619) -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] MINOR: add quick start for tiered storage feature [kafka]

2023-10-16 Thread via GitHub
showuon merged PR #14528: URL: https://github.com/apache/kafka/pull/14528 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] MINOR: add quick start for tiered storage feature [kafka]

2023-10-16 Thread via GitHub
showuon commented on PR #14528: URL: https://github.com/apache/kafka/pull/14528#issuecomment-1765556613 > whether it is worthwhile directly adding the necessary configurations to the server.properties which comes with Kafka and commenting out the ones which will activate the sub-system? One

[jira] [Updated] (KAFKA-15619) Deleted topics will come back again

2023-10-16 Thread Deng Ziming (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deng Ziming updated KAFKA-15619: Summary: Deleted topics will come back again (was: Kafka) > Deleted topics will come back again >

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-16 Thread via GitHub
lianetm commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1361436357 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -248,4 +261,28 @@ public void updateAssignment(ConsumerGroupHear

Re: [PR] KAFKA-15569: test and add test cases in IQv2StoreIntegrationTest [kafka]

2023-10-16 Thread via GitHub
hanyuzheng7 commented on PR #14523: URL: https://github.com/apache/kafka/pull/14523#issuecomment-1765551586 @mjsax Thank you for reviewing the code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] KAFKA-7109: Close fetch sessions on close of consumer [kafka]

2023-10-16 Thread via GitHub
dengziming commented on PR #12590: URL: https://github.com/apache/kafka/pull/12590#issuecomment-1765549171 https://issues.apache.org/jira/browse/KAFKA-15619 Deleted topics will come back again in Apache Spark structured streaming stress test after upgrade Kafka from 3.4.0 to 3.5.0, re

[jira] [Created] (KAFKA-15619) Kafka

2023-10-16 Thread Deng Ziming (Jira)
Deng Ziming created KAFKA-15619: --- Summary: Kafka Key: KAFKA-15619 URL: https://issues.apache.org/jira/browse/KAFKA-15619 Project: Kafka Issue Type: Improvement Affects Versions: 3.5.1, 3.5.

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-16 Thread via GitHub
lianetm commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1361424454 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -135,6 +146,7 @@ public void updateState(ConsumerGroupHeartbeatRe

Re: [PR] KAFKA-15554: Client state changes for handling one assignment at a time & minor improvements [kafka]

2023-10-16 Thread via GitHub
lianetm commented on PR #14413: URL: https://github.com/apache/kafka/pull/14413#issuecomment-1765516635 Thanks for the comments @philipnee and @dajac, all addressed. Just [this](https://github.com/apache/kafka/pull/14413#discussion_r1361286203) comment where I would need some clarification

Re: [PR] KAFKA-15554: Client state changes for handling one assignment at a time & minor improvements [kafka]

2023-10-16 Thread via GitHub
lianetm commented on code in PR #14413: URL: https://github.com/apache/kafka/pull/14413#discussion_r1361411537 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManager.java: ## @@ -21,53 +21,76 @@ import java.util.Optional; /** - * Manages group

Re: [PR] KAFKA-15554: Client state changes for handling one assignment at a time & minor improvements [kafka]

2023-10-16 Thread via GitHub
lianetm commented on code in PR #14413: URL: https://github.com/apache/kafka/pull/14413#discussion_r1361271426 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -160,92 +228,91 @@ private boolean maybeTransitionToStable() {

Re: [PR] KAFKA-15554: Client state changes for handling one assignment at a time & minor improvements [kafka]

2023-10-16 Thread via GitHub
lianetm commented on code in PR #14413: URL: https://github.com/apache/kafka/pull/14413#discussion_r1361404673 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -160,92 +228,91 @@ private boolean maybeTransitionToStable() {

[jira] [Commented] (KAFKA-15609) Corrupted index uploaded to remote tier

2023-10-16 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775965#comment-17775965 ] Haruki Okada commented on KAFKA-15609: -- > is the OS intelligent enough to understan

[jira] [Comment Edited] (KAFKA-15609) Corrupted index uploaded to remote tier

2023-10-16 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775964#comment-17775964 ] Satish Duggana edited comment on KAFKA-15609 at 10/17/23 12:55 AM: ---

Re: [PR] KAFKA-15554: Client state changes for handling one assignment at a time & minor improvements [kafka]

2023-10-16 Thread via GitHub
lianetm commented on code in PR #14413: URL: https://github.com/apache/kafka/pull/14413#discussion_r1361271426 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -160,92 +228,91 @@ private boolean maybeTransitionToStable() {

[jira] [Commented] (KAFKA-15609) Corrupted index uploaded to remote tier

2023-10-16 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775964#comment-17775964 ] Satish Duggana commented on KAFKA-15609: These memory mapped operations are gene

Re: [PR] KAFKA-15554: Client state changes for handling one assignment at a time & minor improvements [kafka]

2023-10-16 Thread via GitHub
lianetm commented on code in PR #14413: URL: https://github.com/apache/kafka/pull/14413#discussion_r1361393280 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManager.java: ## @@ -21,53 +21,76 @@ import java.util.Optional; /** - * Manages group

Re: [PR] KAFKA-15592: Allow member to not sending a heartbeat when a group ID is configured [kafka]

2023-10-16 Thread via GitHub
lianetm commented on code in PR #14390: URL: https://github.com/apache/kafka/pull/14390#discussion_r1361391346 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -231,25 +246,23 @@ public NetworkClientDelegate.UnsentRequest toU

Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-16 Thread via GitHub
CalvinConfluent commented on code in PR #14312: URL: https://github.com/apache/kafka/pull/14312#discussion_r1361389729 ## metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java: ## @@ -1995,6 +2015,19 @@ private void listReassigningTopic(ListPartition

Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-16 Thread via GitHub
CalvinConfluent commented on code in PR #14312: URL: https://github.com/apache/kafka/pull/14312#discussion_r1361390777 ## metadata/src/main/resources/common/metadata/PartitionChangeRecord.json: ## @@ -17,7 +17,9 @@ "apiKey": 5, "type": "metadata", "name": "PartitionChan

Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-16 Thread via GitHub
CalvinConfluent commented on code in PR #14312: URL: https://github.com/apache/kafka/pull/14312#discussion_r1361390411 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -186,7 +186,10 @@ public enum MetadataVersion { IBP_3_6_IV2(14, "

Re: [PR] KAFKA-15554: Client state changes for handling one assignment at a time & minor improvements [kafka]

2023-10-16 Thread via GitHub
lianetm commented on code in PR #14413: URL: https://github.com/apache/kafka/pull/14413#discussion_r1361271426 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -160,92 +228,91 @@ private boolean maybeTransitionToStable() {

Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-16 Thread via GitHub
CalvinConfluent commented on code in PR #14312: URL: https://github.com/apache/kafka/pull/14312#discussion_r1361389729 ## metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java: ## @@ -1995,6 +2015,19 @@ private void listReassigningTopic(ListPartition

Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-16 Thread via GitHub
CalvinConfluent commented on code in PR #14312: URL: https://github.com/apache/kafka/pull/14312#discussion_r1361388641 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -362,6 +405,35 @@ private void setAssignmentChanges(PartitionChangeReco

Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-16 Thread via GitHub
CalvinConfluent commented on code in PR #14312: URL: https://github.com/apache/kafka/pull/14312#discussion_r1361388430 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -328,13 +346,38 @@ public Optional build() { completeReassig

[jira] [Commented] (KAFKA-15609) Corrupted index uploaded to remote tier

2023-10-16 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17775961#comment-17775961 ] Satish Duggana commented on KAFKA-15609: [~divijvaidya] How are index files read

Re: [PR] KAFKA-15569: test and add test cases in IQv2StoreIntegrationTest [kafka]

2023-10-16 Thread via GitHub
mjsax merged PR #14523: URL: https://github.com/apache/kafka/pull/14523 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-16 Thread via GitHub
CalvinConfluent commented on code in PR #14465: URL: https://github.com/apache/kafka/pull/14465#discussion_r1361373946 ## core/src/main/java/kafka/log/CleanShutdownFileHandler.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-16 Thread via GitHub
CalvinConfluent commented on code in PR #14465: URL: https://github.com/apache/kafka/pull/14465#discussion_r1361372633 ## core/src/main/java/kafka/log/CleanShutdownFileHandler.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-16 Thread via GitHub
CalvinConfluent commented on code in PR #14465: URL: https://github.com/apache/kafka/pull/14465#discussion_r1361372153 ## core/src/main/java/kafka/log/CleanShutdownFileHandler.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-16 Thread via GitHub
CalvinConfluent commented on code in PR #14465: URL: https://github.com/apache/kafka/pull/14465#discussion_r1361372008 ## core/src/main/java/kafka/log/CleanShutdownFileHandler.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-16 Thread via GitHub
CalvinConfluent commented on code in PR #14465: URL: https://github.com/apache/kafka/pull/14465#discussion_r1361371860 ## core/src/main/java/kafka/log/CleanShutdownFileHandler.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-16 Thread via GitHub
CalvinConfluent commented on code in PR #14465: URL: https://github.com/apache/kafka/pull/14465#discussion_r1361371469 ## core/src/main/scala/kafka/server/BrokerLifecycleManager.scala: ## @@ -187,6 +187,11 @@ class BrokerLifecycleManager( */ private var _channelManager: N

Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-16 Thread via GitHub
artemlivshits commented on code in PR #14312: URL: https://github.com/apache/kafka/pull/14312#discussion_r1353913582 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -328,13 +346,38 @@ public Optional build() { completeReassignm

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-16 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1361358254 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchBuffer.java: ## @@ -35,19 +41,24 @@ * * * - * Note: this class is not thread-safe and

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-16 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1361358097 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchCollector.java: ## @@ -162,7 +165,9 @@ private Fetch fetchRecords(final CompletedFetch nextI

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-16 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1361357471 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -621,56 +824,177 @@ public void assign(Collection partitions)

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-16 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1361357218 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThreadTest.java: ## @@ -0,0 +1,288 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-16 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1361355764 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchBuffer.java: ## @@ -37,7 +36,7 @@ * * Note: this class is not thread-safe and is intended

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-16 Thread via GitHub
junrao commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1361353808 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchBuffer.java: ## @@ -37,7 +36,7 @@ * * Note: this class is not thread-safe and is intended t

Re: [PR] KAFKA-15582: Identify clean shutdown broker [kafka]

2023-10-16 Thread via GitHub
junrao commented on code in PR #14465: URL: https://github.com/apache/kafka/pull/14465#discussion_r1361271632 ## core/src/main/java/kafka/log/CleanShutdownFileHandler.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contri

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-16 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1361346884 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java: ## @@ -0,0 +1,3573 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-16 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1361346244 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java: ## @@ -0,0 +1,3573 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-16 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1361345756 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java: ## @@ -0,0 +1,3573 @@ +/* + * Licensed to the Apache Software Founda

[jira] [Created] (KAFKA-15618) Implement MetricsCollector which collects metrics in OTLP format

2023-10-16 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15618: - Summary: Implement MetricsCollector which collects metrics in OTLP format Key: KAFKA-15618 URL: https://issues.apache.org/jira/browse/KAFKA-15618 Project: Kafka

[jira] [Created] (KAFKA-15617) Determine if testFetchingPendingPartitions and testInflightFetchOnPendingPartitions overlap

2023-10-16 Thread Kirk True (Jira)
Kirk True created KAFKA-15617: - Summary: Determine if testFetchingPendingPartitions and testInflightFetchOnPendingPartitions overlap Key: KAFKA-15617 URL: https://issues.apache.org/jira/browse/KAFKA-15617

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-16 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1361332784 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java: ## @@ -0,0 +1,3573 @@ +/* + * Licensed to the Apache Software Founda

[jira] [Updated] (KAFKA-15557) Fix duplicate metadata update in fetcher tests

2023-10-16 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15557: -- Parent: KAFKA-14246 Issue Type: Sub-task (was: Test) > Fix duplicate metadata update in fetch

[jira] [Updated] (KAFKA-15606) Verify & refactor correctness of FetcherTest.testCompletedFetchRemoval()

2023-10-16 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15606: -- Parent: KAFKA-14246 Issue Type: Sub-task (was: Improvement) > Verify & refactor correctness o

[jira] [Created] (KAFKA-15616) Define client telemetry states and their transitions

2023-10-16 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15616: - Summary: Define client telemetry states and their transitions Key: KAFKA-15616 URL: https://issues.apache.org/jira/browse/KAFKA-15616 Project: Kafka Issue

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-16 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1361329379 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -636,42 +857,148 @@ public void assign(Collection partitions)

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-16 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1361329900 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchBuffer.java: ## @@ -37,7 +36,7 @@ * * Note: this class is not thread-safe and is intended

[jira] [Updated] (KAFKA-15615) Improve handling of fetching during metadata updates

2023-10-16 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15615: -- Labels: consumer-threading-refactor (was: ) > Improve handling of fetching during metadata updates >

[jira] [Updated] (KAFKA-15615) Improve handling of fetching during metadata updates

2023-10-16 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15615: -- Description: [During a review of the new fetcher|https://github.com/apache/kafka/pull/14406#discussio

[jira] [Created] (KAFKA-15615) Improve handling of fetching during metadata updates

2023-10-16 Thread Kirk True (Jira)
Kirk True created KAFKA-15615: - Summary: Improve handling of fetching during metadata updates Key: KAFKA-15615 URL: https://issues.apache.org/jira/browse/KAFKA-15615 Project: Kafka Issue Type: Su

[jira] [Created] (KAFKA-15614) Define interfaces/classes for capturing telemetry metrics

2023-10-16 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15614: - Summary: Define interfaces/classes for capturing telemetry metrics Key: KAFKA-15614 URL: https://issues.apache.org/jira/browse/KAFKA-15614 Project: Kafka

[jira] [Updated] (KAFKA-15613) Add Client API in interfaces and client configurations.

2023-10-16 Thread Apoorv Mittal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apoorv Mittal updated KAFKA-15613: -- Description: Define [client API|https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+

Re: [PR] KAFKA-15532: non active controllers return 0 for ZkWriteBeforelag [kafka]

2023-10-16 Thread via GitHub
cmccabe merged PR #14478: URL: https://github.com/apache/kafka/pull/14478 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] KAFKA-14509 [WIP] [2/2] Implement server side logic for ConsumerGroupDescribe API [kafka]

2023-10-16 Thread via GitHub
dongnuo123 commented on code in PR #14544: URL: https://github.com/apache/kafka/pull/14544#discussion_r1361149813 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinator.java: ## @@ -151,6 +152,19 @@ CompletableFuture> describeGroup List

Re: [PR] KAFKA-15390: Do not return fenced broker in FetchResponse.preferredReplica [kafka]

2023-10-16 Thread via GitHub
jolshan commented on PR #14272: URL: https://github.com/apache/kafka/pull/14272#issuecomment-1765352032 Sorry I missed this. Thanks @dengziming -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] KAFKA-15390: Do not return fenced broker in FetchResponse.preferredReplica [kafka]

2023-10-16 Thread via GitHub
cmccabe merged PR #14272: URL: https://github.com/apache/kafka/pull/14272 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] KAFKA-15613: Client API definition and configurations (KIP-714) [kafka]

2023-10-16 Thread via GitHub
apoorvmittal10 commented on code in PR #14560: URL: https://github.com/apache/kafka/pull/14560#discussion_r1361306135 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -385,6 +391,10 @@ public class ProducerConfig extends AbstractConfig {

Re: [PR] KAFKA-15613: Client API definition and configurations (KIP-714) [kafka]

2023-10-16 Thread via GitHub
apoorvmittal10 commented on code in PR #14560: URL: https://github.com/apache/kafka/pull/14560#discussion_r1361305832 ## clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java: ## @@ -1892,6 +1892,37 @@ public Map committed(final Set

Re: [PR] KAFKA-15613: Client API definition and configurations (KIP-714) [kafka]

2023-10-16 Thread via GitHub
apoorvmittal10 commented on code in PR #14560: URL: https://github.com/apache/kafka/pull/14560#discussion_r1361305410 ## clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java: ## @@ -105,6 +105,10 @@ public class CommonClientConfigs { public static final i

Re: [PR] KAFKA-15613: Client API definition and configurations (KIP-714) [kafka]

2023-10-16 Thread via GitHub
apoorvmittal10 commented on code in PR #14560: URL: https://github.com/apache/kafka/pull/14560#discussion_r1361305203 ## clients/src/main/java/org/apache/kafka/clients/admin/Admin.java: ## @@ -1660,6 +1662,34 @@ default FenceProducersResult fenceProducers(Collection transaction

Re: [PR] KAFKA-15554: Client state changes for handling one assignment at a time & minor improvements [kafka]

2023-10-16 Thread via GitHub
lianetm commented on code in PR #14413: URL: https://github.com/apache/kafka/pull/14413#discussion_r1361299217 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -160,92 +228,91 @@ private boolean maybeTransitionToStable() {

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-16 Thread via GitHub
kirktrue commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1361294748 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundation

  1   2   3   >