[jira] [Updated] (KAFKA-15776) Configurable delay timeout for DelayedRemoteFetch request

2023-11-02 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15776: - Description: We are reusing the {{fetch.max.wait.ms}} config as a delay timeout

Re: [PR] [WIP]KAFKA-15444: Native docker image [kafka]

2023-11-02 Thread via GitHub
ijuma commented on code in PR #14556: URL: https://github.com/apache/kafka/pull/14556#discussion_r1381192520 ## docker/test/requirements.txt: ## @@ -0,0 +1,6 @@ +confluent_kafka Review Comment: Makes sense, thanks. -- This is an automated message from the Apache Git Serv

Re: [PR] KAFKA-15781: KIP-998, Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-02 Thread via GitHub
ijuma commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1381189992 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, props);

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-02 Thread via GitHub
philipnee commented on PR #14680: URL: https://github.com/apache/kafka/pull/14680#issuecomment-1791913330 Hi @dajac @kirktrue - Thanks for offering valuable feedbacks to this PR, much appreciated. I wonder how do you feel about using a local queue to stash the new callbacks? The goal really

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-02 Thread via GitHub
jeffkbkim commented on code in PR #14387: URL: https://github.com/apache/kafka/pull/14387#discussion_r1381174247 ## checkstyle/import-control.xml: ## @@ -240,6 +240,7 @@ + Review Comment: it's ready -- This is an automated message from the

Re: [PR] KAFKA-15769: Fix wrong log with exception [kafka]

2023-11-02 Thread via GitHub
GOODBOY008 commented on PR #14683: URL: https://github.com/apache/kafka/pull/14683#issuecomment-1791899817 > LGTM, Can you check in the code base if there are other cases like this? Yes , I'm willing to check all code like this. -- This is an automated message from the Apache Git S

[jira] [Created] (KAFKA-15782) Establish concrete project conventions to define public APIs that require a KIP

2023-11-02 Thread A. Sophie Blee-Goldman (Jira)
A. Sophie Blee-Goldman created KAFKA-15782: -- Summary: Establish concrete project conventions to define public APIs that require a KIP Key: KAFKA-15782 URL: https://issues.apache.org/jira/browse/KAFKA-1578

Re: [PR] KAFKA-15781: KIP-998, Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-02 Thread via GitHub
ableegoldman commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1381158659 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, pr

Re: [PR] MINOR: Small LogValidator clean ups [kafka]

2023-11-02 Thread via GitHub
ex172000 commented on code in PR #14697: URL: https://github.com/apache/kafka/pull/14697#discussion_r1381157089 ## core/src/test/scala/unit/kafka/log/LogValidatorTest.scala: ## @@ -399,12 +402,20 @@ class LogValidatorTest { assertEquals(i, offsetCounter.value); assertE

[jira] [Updated] (KAFKA-15781) Change ProducerConfig(props, doLog) constructor to protected

2023-11-02 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-15781: --- Description: See [https://github.com/apache/kafka/pull/14681]   KIP: http

[jira] [Created] (KAFKA-15781) Change ProducerConfig(props, doLog) constructor to protected

2023-11-02 Thread A. Sophie Blee-Goldman (Jira)
A. Sophie Blee-Goldman created KAFKA-15781: -- Summary: Change ProducerConfig(props, doLog) constructor to protected Key: KAFKA-15781 URL: https://issues.apache.org/jira/browse/KAFKA-15781 Proj

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-02 Thread via GitHub
ableegoldman commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r138110 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, pr

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-02 Thread via GitHub
mjsax commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1381144344 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, props);

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1381143945 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] [WIP]KAFKA-15444: Native docker image [kafka]

2023-11-02 Thread via GitHub
VedarthConfluent commented on code in PR #14556: URL: https://github.com/apache/kafka/pull/14556#discussion_r1381142228 ## docker/test/requirements.txt: ## @@ -0,0 +1,6 @@ +confluent_kafka Review Comment: It has been removed. These changes are part of KIP 975 PR raised here:

Re: [PR] KAFKA-14957: Update-Description-String [kafka]

2023-11-02 Thread via GitHub
mjsax commented on code in PR #13909: URL: https://github.com/apache/kafka/pull/13909#discussion_r1381139811 ## clients/src/main/java/org/apache/kafka/common/config/ConfigDef.java: ## @@ -1216,6 +1254,7 @@ public static class ConfigKey { public final List dependents;

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1381121312 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1381125394 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1381127886 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -251,144 +260,52 @@ private ConsumerGroupMember t

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1381123599 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1381125394 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1381125394 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

[jira] [Updated] (KAFKA-15776) Configurable delay timeout for DelayedRemoteFetch request

2023-11-02 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15776: - Description: We are reusing the {{fetch.max.wait.ms}} config as a delay timeout

[jira] [Updated] (KAFKA-15777) Configurable remote fetch bytes per partition from Consumer

2023-11-02 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15777: - Description: A consumer can configure the amount of local bytes to read from eac

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1381123599 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1381121312 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1381118918 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1381121312 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380922729 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1381118918 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15740: KRaft support in DeleteOffsetsConsumerGroupCommandIntegrationTest [kafka]

2023-11-02 Thread via GitHub
linzihao1999 commented on code in PR #14669: URL: https://github.com/apache/kafka/pull/14669#discussion_r1381106437 ## core/src/test/scala/unit/kafka/admin/DeleteOffsetsConsumerGroupCommandIntegrationTest.scala: ## @@ -20,7 +20,7 @@ package kafka.admin import java.util.Properti

Re: [PR] KAFKA-14957: Update-Description-String [kafka]

2023-11-02 Thread via GitHub
Owen-CH-Leung commented on PR #13909: URL: https://github.com/apache/kafka/pull/13909#issuecomment-1791810020 @mjsax Thanks. It really clarifies a lot! I've modified the code to allow overriding default value for `state.dir` by setting the system property `kafka.streams.state.dir`. So

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380954663 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Given

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380954345 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Given

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380954188 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Given

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380949384 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Given

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380943451 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -1094,58 +1087,54 @@ private void cancelConsumerGroupSessionTimeo

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380936875 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupMember.java: ## @@ -301,11 +322,9 @@ public String toString() { private

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380929415 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -1094,58 +1087,54 @@ private void cancelConsumerGroupSessionTimeo

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380923649 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380923649 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380923297 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -1094,58 +1087,54 @@ private void cancelConsumerGroupSessionTimeo

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380922729 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15774: introduce internal StoreFactory [kafka]

2023-11-02 Thread via GitHub
ableegoldman commented on code in PR #14659: URL: https://github.com/apache/kafka/pull/14659#discussion_r1380921878 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StoreBuilderWrapper.java: ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380913612 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -934,32 +934,25 @@ private CoordinatorResult consumerGr

Re: [PR] KAFKA-15774: introduce internal StoreFactory [kafka]

2023-11-02 Thread via GitHub
ableegoldman commented on code in PR #14659: URL: https://github.com/apache/kafka/pull/14659#discussion_r1380912197 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StoreFactory.java: ## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (AS

Re: [PR] KAFKA-15774: introduce internal StoreFactory [kafka]

2023-11-02 Thread via GitHub
ableegoldman commented on code in PR #14659: URL: https://github.com/apache/kafka/pull/14659#discussion_r1379396433 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KeyValueStoreMaterializer.java: ## @@ -27,26 +31,44 @@ import org.slf4j.Logger; import org.sl

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
jolshan commented on PR #14673: URL: https://github.com/apache/kafka/pull/14673#issuecomment-1791722750 > When partitions are revoked, the state machine cannot advanced until the consumer acknowledges the revocation. However, when partitions are assigned, the state machine does not wait on

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1380898562 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This me

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1380899701 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This me

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1380898562 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This me

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1380898562 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This me

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-02 Thread via GitHub
ijuma commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1380892982 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, props);

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-11-02 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1380888754 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This met

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-02 Thread via GitHub
ableegoldman commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1380884002 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, pr

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380879868 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1380878154 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -34,48 +33,68 @@ * consumer group protocol. Give

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-02 Thread via GitHub
ableegoldman commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1380876249 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, pr

[PR] MINOR: Small LogValidator clean ups [kafka]

2023-11-02 Thread via GitHub
hachikuji opened a new pull request, #14697: URL: https://github.com/apache/kafka/pull/14697 1. Set shallowOffsetOfMaxTimestamp consistently as the last offset in the batch for v2 compressed and non-compressed data. 2. Rename `RecordConversionStats` to `RecordValidationStats` since one of

[PR] [KAFKA-15022] optimization: detect negative cycle from one source [kafka]

2023-11-02 Thread via GitHub
lihaosky opened a new pull request, #14696: URL: https://github.com/apache/kafka/pull/14696 ### Description Introduce a dummy node connected to every other node and run Bellman-ford from the dummy node once instead of from every node in the graph. ### Testing Existing test

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-02 Thread via GitHub
philipnee commented on code in PR #14680: URL: https://github.com/apache/kafka/pull/14680#discussion_r1380850627 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -1096,4 +1114,47 @@ private void subscribeInternal(Collection

Re: [PR] KAFKA-15578: Migrating other system tests to use the group coordinator [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on PR #14582: URL: https://github.com/apache/kafka/pull/14582#issuecomment-1791632833 https://jenkins.confluent.io/job/system-test-kafka-branch-builder/5930/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[PR] KAFKA-15780: Wait for consistent KRaft metadata when creating topics [kafka]

2023-11-02 Thread via GitHub
splett2 opened a new pull request, #14695: URL: https://github.com/apache/kafka/pull/14695 ### What TestUtils.createTopicWithAdmin calls `waitForAllPartitionsMetadata` which waits for partition(s) to be present in each brokers' metadata cache. This is a sufficient check in ZK mode becaus

Re: [PR] KAFKA 14515: Optimized Uniform Rack Aware Assignor [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14416: URL: https://github.com/apache/kafka/pull/14416#discussion_r1380818617 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AbstractUniformAssignmentBuilder.java: ## @@ -0,0 +1,256 @@ +/* + * Licensed to the Apa

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1380469707 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This me

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-11-02 Thread via GitHub
artemlivshits commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1380813788 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: T

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1380469707 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This me

[jira] [Created] (KAFKA-15780) Wait for consistent kraft metadata when creating topics in tests

2023-11-02 Thread David Mao (Jira)
David Mao created KAFKA-15780: - Summary: Wait for consistent kraft metadata when creating topics in tests Key: KAFKA-15780 URL: https://issues.apache.org/jira/browse/KAFKA-15780 Project: Kafka I

Re: [PR] KAFKA 14515: Optimized Uniform Rack Aware Assignor [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14416: URL: https://github.com/apache/kafka/pull/14416#discussion_r1380781750 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/SubscribedTopicMetadataTest.java: ## @@ -43,27 +45,62 @@ public void testAttribute() {

Re: [PR] KAFKA 14515: Optimized Uniform Rack Aware Assignor [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14416: URL: https://github.com/apache/kafka/pull/14416#discussion_r1380777612 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,413 @@ +/* + * Licensed to the Ap

Re: [PR] [WIP]KAFKA-15444: Native docker image [kafka]

2023-11-02 Thread via GitHub
ijuma commented on code in PR #14556: URL: https://github.com/apache/kafka/pull/14556#discussion_r1380771281 ## docker/test/requirements.txt: ## @@ -0,0 +1,6 @@ +confluent_kafka Review Comment: Can we remove `confluent_kafka` from here then? -- This is an automated messa

Re: [PR] KAFKA 14515: Optimized Uniform Rack Aware Assignor [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14416: URL: https://github.com/apache/kafka/pull/14416#discussion_r1380768591 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AbstractUniformAssignmentBuilder.java: ## @@ -0,0 +1,256 @@ +/* + * Licensed to the Apa

Re: [PR] KAFKA 14515: Optimized Uniform Rack Aware Assignor [kafka]

2023-11-02 Thread via GitHub
rreddy-22 commented on code in PR #14416: URL: https://github.com/apache/kafka/pull/14416#discussion_r1380756376 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,419 @@ +/* + * Licensed to the Ap

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1380752726 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This me

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-11-02 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1380752726 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This me

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380747077 ## clients/src/test/java/org/apache/kafka/clients/consumer/ConsumerConfigTest.java: ## @@ -67,17 +67,11 @@ public void testOverrideClientId() { @Test public

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380746726 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/LegacyKafkaConsumer.java: ## @@ -0,0 +1,2554 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-11-02 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1380724965 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This met

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380719222 ## clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java: ## @@ -321,20 +320,6 @@ public class ConsumerConfig extends AbstractConfig { */

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380718362 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -982,23 +987,6 @@ private boolean initWithCommittedOffsetsIfNeeded(

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380717397 ## clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java: ## @@ -666,19 +651,6 @@ else if (newConfigs.get(VALUE_DESERIALIZER_CLASS_CONFIG) == nu

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380717923 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -250,7 +255,7 @@ public PrototypeAsyncConsumer(final Time time,

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
AndrewJSchofield commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380705419 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/LegacyKafkaConsumer.java: ## @@ -0,0 +1,2554 @@ +/* + * Licensed to the Apache Software Fo

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-02 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1380683789 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,33 +242,465 @@ public void updateState(ConsumerGroupHeartbe

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-02 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1380694833 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManager.java: ## @@ -73,30 +84,55 @@ public interface MembershipManager { /**

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-02 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1380693973 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -260,42 +743,16 @@ public Optional serverAssignor() { * {@i

[jira] [Created] (KAFKA-15779) Define broker configurations and exceptions

2023-11-02 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15779: - Summary: Define broker configurations and exceptions Key: KAFKA-15779 URL: https://issues.apache.org/jira/browse/KAFKA-15779 Project: Kafka Issue Type: Sub

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-02 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1380683789 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,33 +242,465 @@ public void updateState(ConsumerGroupHeartbe

[jira] [Created] (KAFKA-15778) Implement ClientMetricsManager to process request

2023-11-02 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15778: - Summary: Implement ClientMetricsManager to process request Key: KAFKA-15778 URL: https://issues.apache.org/jira/browse/KAFKA-15778 Project: Kafka Issue Typ

Re: [PR] MINOR: Log a warning when connectors generate greater than tasks.max task configs [kafka]

2023-11-02 Thread via GitHub
C0urante commented on PR #14694: URL: https://github.com/apache/kafka/pull/14694#issuecomment-1791420759 @gharris1727 @mimaison given your activity on [KAFKA-15575](https://issues.apache.org/jira/browse/KAFKA-15575), this may interest you. We can also backport this change (which is unlikely

[PR] MINOR: Log a warning when connectors generate greater than tasks.max task configs [kafka]

2023-11-02 Thread via GitHub
C0urante opened a new pull request, #14694: URL: https://github.com/apache/kafka/pull/14694 The discussion around [KIP-987](https://cwiki.apache.org/confluence/display/KAFKA/KIP-987%3A+Connect+Static+Assignments) has highlighted some strange behavior in the Connect runtime: if a connector's

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380667322 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerDelegateCreator.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r138030 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerDelegateCreator.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380667051 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerDelegateCreator.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380664987 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/LegacyKafkaConsumerTest.java: ## @@ -2582,7 +2581,7 @@ private FetchResponse fetchResponse(TopicPa

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380663703 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/LegacyKafkaConsumerTest.java: ## @@ -164,7 +163,7 @@ * Note to future authors in this class. If

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380662917 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/LegacyKafkaConsumer.java: ## @@ -0,0 +1,2623 @@ +/* Review Comment: Yes. The only changes are

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380662469 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -108,13 +106,19 @@ import static org.apache.kafka.common.utils.Uti

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380661924 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerDelegateCreator.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-02 Thread via GitHub
kirktrue commented on code in PR #14670: URL: https://github.com/apache/kafka/pull/14670#discussion_r1380661746 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerDelegateCreator.java: ## @@ -0,0 +1,132 @@ +/* + * Licensed to the Apache Software Foundat

  1   2   3   >