Re: [PR] MINOR: migrate ListConsumerGroupTest to use ClusterTestExtensions [kafka]

2024-05-24 Thread via GitHub
chia7712 commented on code in PR #15821: URL: https://github.com/apache/kafka/pull/15821#discussion_r1612965039 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/ConsumerGroupCommandTestUtils.java: ## @@ -68,15 +68,31 @@ static List generator() { ClusterConfi

[jira] [Assigned] (KAFKA-16520) Changes to DescribeQuorum response

2024-05-24 Thread Nikolay Izhikov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov reassigned KAFKA-16520: --- Assignee: Nikolay Izhikov (was: José Armando García Sancio) > Changes to DescribeQ

[jira] [Resolved] (KAFKA-16798) Mirrormaker2 dedicated mode - sync.group.offsets.interval not working

2024-05-24 Thread Thanos Athanasopoulos (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thanos Athanasopoulos resolved KAFKA-16798. --- Resolution: Not A Bug This issue was resolved after input stated in the prev

Re: [PR] KAFKA-16822: Abstract consumer group to share functionality with share group (KIP-932) [kafka]

2024-05-24 Thread via GitHub
dajac commented on code in PR #16054: URL: https://github.com/apache/kafka/pull/16054#discussion_r1613013960 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/AbstractGroup.java: ## @@ -0,0 +1,625 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] KAFKA-16626: Lazily convert subscribed topic names to topic ids [kafka]

2024-05-24 Thread via GitHub
dajac commented on PR #15970: URL: https://github.com/apache/kafka/pull/15970#issuecomment-2128833459 The failed tests are related to https://github.com/apache/kafka/pull/15972. Merging to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [PR] KAFKA-16626: Lazily convert subscribed topic names to topic ids [kafka]

2024-05-24 Thread via GitHub
dajac merged PR #15970: URL: https://github.com/apache/kafka/pull/15970 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[jira] [Resolved] (KAFKA-16626) Uuid to String for subscribed topic names in assignment spec

2024-05-24 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16626. - Fix Version/s: 3.8.0 Resolution: Fixed > Uuid to String for subscribed topic names in ass

Re: [PR] KAFKA-15265: Add Remote Log Manager quota manager [kafka]

2024-05-24 Thread via GitHub
abhijeetk88 commented on code in PR #15625: URL: https://github.com/apache/kafka/pull/15625#discussion_r1613033249 ## storage/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogManagerConfig.java: ## @@ -143,6 +143,38 @@ public final class RemoteLogManagerConfig {

Re: [PR] KAFKA-16831: CoordinatorRuntime should initialize MemoryRecordsBuilder with max batch size write limit [kafka]

2024-05-24 Thread via GitHub
dajac commented on code in PR #16059: URL: https://github.com/apache/kafka/pull/16059#discussion_r1613028497 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -1365,6 +1368,11 @@ public void onHighWatermarkUpdated(

[PR] KAFKA:16821 - Member subscription spec interface [kafka]

2024-05-24 Thread via GitHub
rreddy-22 opened a new pull request, #16068: URL: https://github.com/apache/kafka/pull/16068 Create a new interface called MemberSubscriptionSpec to hold the members subscription information. The assignment member spec is replaced by the MemberSubscriptionSpecImpl class. Eventually we

Re: [PR] KAFKA-15045: (KIP-924 pt. 11) Implemented StickyTaskAssignor [kafka]

2024-05-24 Thread via GitHub
ableegoldman commented on code in PR #16052: URL: https://github.com/apache/kafka/pull/16052#discussion_r1613071356 ## streams/src/main/java/org/apache/kafka/streams/processor/assignment/assignors/StickyTaskAssignor.java: ## @@ -0,0 +1,478 @@ +/* + * Licensed to the Apache Softw

Re: [PR] KAFKA-16713: Define initial set of RPCs for KIP-932 [kafka]

2024-05-24 Thread via GitHub
AndrewJSchofield commented on PR #16022: URL: https://github.com/apache/kafka/pull/16022#issuecomment-2128913832 There are 2 test failures introduced by this PR. I'll get them fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] KAFKA-15045: (KIP-924 pt. 11) Implemented StickyTaskAssignor [kafka]

2024-05-24 Thread via GitHub
ableegoldman commented on code in PR #16052: URL: https://github.com/apache/kafka/pull/16052#discussion_r1613071356 ## streams/src/main/java/org/apache/kafka/streams/processor/assignment/assignors/StickyTaskAssignor.java: ## @@ -0,0 +1,478 @@ +/* + * Licensed to the Apache Softw

Re: [PR] KAFKA-15045: (KIP-924 pt. 9) TaskAssignmentUtils implementation of optimizeRackAwareActiveTasks [kafka]

2024-05-24 Thread via GitHub
ableegoldman commented on code in PR #16033: URL: https://github.com/apache/kafka/pull/16033#discussion_r1613111098 ## streams/src/main/java/org/apache/kafka/streams/processor/assignment/TaskAssignmentUtils.java: ## @@ -74,20 +209,240 @@ public static Map optimizeRackAwareStand

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613112730 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() { itera

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613112730 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() { itera

Re: [PR] KAFKA-16639: AsyncKafkaConsumer#close dose not send heartbeat to leave group. [kafka]

2024-05-24 Thread via GitHub
frankvicky commented on PR #16017: URL: https://github.com/apache/kafka/pull/16017#issuecomment-2129009977 Hi @lianetm, I really appreciate your suggestion. These suggestions are very helpful to me. I will give them a try. 😸 -- This is an automated message from the Apache Git Service.

[jira] [Updated] (KAFKA-16835) Add Support for consumer to read in commit order.

2024-05-24 Thread Manjunath (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjunath updated KAFKA-16835: -- Language: java C++ (was: Java java) > Add Support for consumer to read in commit order. > ---

Re: [PR] KAFKA-16832: LeaveGroup API for upgrading ConsumerGroup [kafka]

2024-05-24 Thread via GitHub
dajac commented on code in PR #16057: URL: https://github.com/apache/kafka/pull/16057#discussion_r1613089003 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -2041,6 +2027,36 @@ private CoordinatorResult consumerGroupFenceM

[PR] Adding standalone parameter to kafka-storage.sh [KAFKA-16518] [kafka]

2024-05-24 Thread via GitHub
muralibasani opened a new pull request, #16069: URL: https://github.com/apache/kafka/pull/16069 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* - kafka-storag

[jira] [Commented] (KAFKA-16518) Storage tool changes for KIP-853

2024-05-24 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849236#comment-17849236 ] Muralidhar Basani commented on KAFKA-16518: --- [~jsancio] have a draft pr opened

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613112730 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() { itera

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613112730 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() { itera

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613112730 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() { itera

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613112730 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() { itera

Re: [PR] Update RemoteLogManager configuration in broker server - KAFKA-16790 [kafka]

2024-05-24 Thread via GitHub
nikramakrishnan commented on PR #16005: URL: https://github.com/apache/kafka/pull/16005#issuecomment-2129187298 > Instead we should throw an exception (or stateChangeLogger.error) like mentioned below here, and write a unit test just for this class Do you mean we mock `onPartitionLead

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-24 Thread via GitHub
omkreddy commented on PR #16027: URL: https://github.com/apache/kafka/pull/16027#issuecomment-2129194429 There is an failure due to missing license: ``` [2024-05-23T07:39:35.270Z] Execution failed for task ':rat'. [2024-05-23T07:39:35.270Z] > A failure occurred while executin

Re: [PR] Refactor SSL/SASL admin integration tests to not use a custom authorizer [kafka]

2024-05-24 Thread via GitHub
mimaison merged PR #15377: URL: https://github.com/apache/kafka/pull/15377 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[PR] MINOR: mention KAFKA-15905 in docs "Notable changes in 3.7.1" [kafka]

2024-05-24 Thread via GitHub
edoardocomar opened a new pull request, #16070: URL: https://github.com/apache/kafka/pull/16070 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade

Re: [PR] KAFKA-15905 Restarts of MirrorCheckpointTask should not permanently i… [kafka]

2024-05-24 Thread via GitHub
edoardocomar commented on PR #15910: URL: https://github.com/apache/kafka/pull/15910#issuecomment-2129247940 @gharris1727 please see https://github.com/apache/kafka/pull/16070 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] MINOR: mention KAFKA-15905 in docs "Notable changes in 3.7.1" [kafka]

2024-05-24 Thread via GitHub
edoardocomar commented on PR #16070: URL: https://github.com/apache/kafka/pull/16070#issuecomment-2129249297 As discussed in https://github.com/apache/kafka/pull/15910#issuecomment-2127588361 -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613293262 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() { itera

Re: [PR] KAFKA-16535: Implement AddVoter, RemoveVoter, UpdateVoter RPCs [kafka]

2024-05-24 Thread via GitHub
AndrewJSchofield commented on PR #16058: URL: https://github.com/apache/kafka/pull/16058#issuecomment-2129297575 LGTM apart from a few test failures related to the new RPCs, such as RequestConvertToJsonTest. -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] KAFKA-16822: Abstract consumer group to share functionality with share group (KIP-932) [kafka]

2024-05-24 Thread via GitHub
AndrewJSchofield commented on code in PR #16054: URL: https://github.com/apache/kafka/pull/16054#discussion_r1613325546 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/AbstractGroup.java: ## @@ -0,0 +1,625 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] KAFKA-16815: Handle FencedInstanceId in HB response [kafka]

2024-05-24 Thread via GitHub
dajac merged PR #16047: URL: https://github.com/apache/kafka/pull/16047 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[jira] [Resolved] (KAFKA-16815) Handle FencedInstanceId on heartbeat for new consumer

2024-05-24 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16815. - Fix Version/s: 3.8.0 Reviewer: David Jacot Resolution: Fixed > Handle FencedInst

[jira] [Created] (KAFKA-16837) Kafka Connect fails on update connector for incorrect previous Config Provider tasks

2024-05-24 Thread Sergey Ivanov (Jira)
Sergey Ivanov created KAFKA-16837: - Summary: Kafka Connect fails on update connector for incorrect previous Config Provider tasks Key: KAFKA-16837 URL: https://issues.apache.org/jira/browse/KAFKA-16837

[jira] [Updated] (KAFKA-16837) Kafka Connect fails on update connector for incorrect previous Config Provider tasks

2024-05-24 Thread Sergey Ivanov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Ivanov updated KAFKA-16837: -- Attachment: kafka_connect_config.png > Kafka Connect fails on update connector for incorrect p

[PR] [WIP] KAFKA-15265: Integrate RLMQuotaManager for throttling fetches from remote storage [kafka]

2024-05-24 Thread via GitHub
abhijeetk88 opened a new pull request, #16071: URL: https://github.com/apache/kafka/pull/16071 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of tes

[jira] [Created] (KAFKA-16838) Kafka Connect loads old tasks from removed connectors

2024-05-24 Thread Sergey Ivanov (Jira)
Sergey Ivanov created KAFKA-16838: - Summary: Kafka Connect loads old tasks from removed connectors Key: KAFKA-16838 URL: https://issues.apache.org/jira/browse/KAFKA-16838 Project: Kafka Issue

[PR] KAFKA-16371; fix lingering pending commit when handling OFFSET_METADATA_TOO_LARGE [kafka]

2024-05-24 Thread via GitHub
dajac opened a new pull request, #16072: URL: https://github.com/apache/kafka/pull/16072 This patch was initially created in https://github.com/apache/kafka/pull/15536. When there is a commit for multiple topic partitions and some, but not all, exceed the offset metadata limit, the p

[jira] [Updated] (KAFKA-16837) Kafka Connect fails on update connector for incorrect previous Config Provider tasks

2024-05-24 Thread Sergey Ivanov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Ivanov updated KAFKA-16837: -- Description: Hello, We faced an issue when is not possible to update Connector config if the

[jira] [Updated] (KAFKA-16838) Kafka Connect loads old tasks from removed connectors

2024-05-24 Thread Sergey Ivanov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Ivanov updated KAFKA-16838: -- Description: Hello, When creating connector we faced an error from one of our ConfigProviders

Re: [PR] KAFKA-16833: Fixing PartitionInfo and Cluster equals and hashCode [kafka]

2024-05-24 Thread via GitHub
chia7712 commented on code in PR #16062: URL: https://github.com/apache/kafka/pull/16062#discussion_r1613444662 ## clients/src/main/java/org/apache/kafka/common/PartitionInfo.java: ## @@ -88,6 +90,28 @@ public Node[] offlineReplicas() { return offlineReplicas; }

[jira] [Resolved] (KAFKA-10234) The key/value deserializer used by ConsoleConsumer is not closed

2024-05-24 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10234. Resolution: Invalid > The key/value deserializer used by ConsoleConsumer is not closed > -

Re: [PR] KAFKA-16821: Member Subscription Spec Interface [kafka]

2024-05-24 Thread via GitHub
dajac commented on code in PR #16068: URL: https://github.com/apache/kafka/pull/16068#discussion_r1613460654 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/TargetAssignmentBuilder.java: ## @@ -277,13 +277,22 @@ public TargetAssignmentBuilder remov

Re: [PR] Update RemoteLogManager configuration in broker server - KAFKA-16790 [kafka]

2024-05-24 Thread via GitHub
muralibasani commented on PR #16005: URL: https://github.com/apache/kafka/pull/16005#issuecomment-2129528563 @nikramakrishnan I have pushed a few changes. Please review. We explicitly throw a KafkaException now if rlm is not configured and remote storage is enabled -- This is an automa

[jira] [Updated] (KAFKA-16838) Kafka Connect loads old tasks from removed connectors

2024-05-24 Thread Sergey Ivanov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Ivanov updated KAFKA-16838: -- Description: Hello, When creating connector we faced an error from one of our ConfigProviders

[jira] [Updated] (KAFKA-16837) Kafka Connect fails on update connector for incorrect previous Config Provider tasks

2024-05-24 Thread Sergey Ivanov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Ivanov updated KAFKA-16837: -- Description: Hello, We faced an issue when is not possible to update Connector config if the

Re: [PR] KAFKA-16807: DescribeLogDirsResponseData#results#topics have unexpected topics having empty partitions [kafka]

2024-05-24 Thread via GitHub
m1a2st commented on code in PR #16042: URL: https://github.com/apache/kafka/pull/16042#discussion_r1613525989 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1247,7 +1247,8 @@ class ReplicaManager(val config: KafkaConfig, val usableBytes = adjustForLarg

[jira] [Updated] (KAFKA-16838) Kafka Connect loads old tasks from removed connectors

2024-05-24 Thread Sergey Ivanov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Ivanov updated KAFKA-16838: -- Description: Hello, When creating connector we faced an error from one of our ConfigProviders

Re: [PR] KAFKA-16807: DescribeLogDirsResponseData#results#topics have unexpected topics having empty partitions [kafka]

2024-05-24 Thread via GitHub
m1a2st commented on code in PR #16042: URL: https://github.com/apache/kafka/pull/16042#discussion_r1613531701 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -6594,6 +6594,64 @@ class ReplicaManagerTest { )) } } + + @Test + def testRepli

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613568734 ## streams/src/test/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStoreTest.java: ## @@ -490,6 +490,31 @@ public void shouldTimeIteratorDuration() {

[jira] [Assigned] (KAFKA-16503) getOrMaybeCreateClassicGroup should not thrown GroupIdNotFoundException

2024-05-24 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot reassigned KAFKA-16503: --- Assignee: David Jacot > getOrMaybeCreateClassicGroup should not thrown GroupIdNotFoundExcep

Re: [PR] KAFKA-16822: Abstract consumer group to share functionality with share group (KIP-932) [kafka]

2024-05-24 Thread via GitHub
lianetm commented on code in PR #16054: URL: https://github.com/apache/kafka/pull/16054#discussion_r1613579324 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/AbstractGroup.java: ## @@ -0,0 +1,625 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] KAFKA-16822: Abstract consumer group to share functionality with share group (KIP-932) [kafka]

2024-05-24 Thread via GitHub
lianetm commented on code in PR #16054: URL: https://github.com/apache/kafka/pull/16054#discussion_r1613579324 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/AbstractGroup.java: ## @@ -0,0 +1,625 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] KAFKA-16822: Abstract consumer group to share functionality with share group (KIP-932) [kafka]

2024-05-24 Thread via GitHub
lianetm commented on code in PR #16054: URL: https://github.com/apache/kafka/pull/16054#discussion_r1613579324 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/AbstractGroup.java: ## @@ -0,0 +1,625 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] KAFKA-16822: Abstract consumer group to share functionality with share group (KIP-932) [kafka]

2024-05-24 Thread via GitHub
lianetm commented on code in PR #16054: URL: https://github.com/apache/kafka/pull/16054#discussion_r1613579324 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/AbstractGroup.java: ## @@ -0,0 +1,625 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] KAFKA-16822: Abstract consumer group to share functionality with share group (KIP-932) [kafka]

2024-05-24 Thread via GitHub
apoorvmittal10 commented on code in PR #16054: URL: https://github.com/apache/kafka/pull/16054#discussion_r1613594739 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/AbstractGroup.java: ## @@ -0,0 +1,625 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-16832: LeaveGroup API for upgrading ConsumerGroup [kafka]

2024-05-24 Thread via GitHub
jeffkbkim commented on code in PR #16057: URL: https://github.com/apache/kafka/pull/16057#discussion_r1613555271 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -4424,14 +4441,113 @@ private ConsumerGroupMember validateCons

[PR] KAFKA-16503; getOrMaybeCreateClassicGroup should not thrown GroupIdNotFoundException [kafka]

2024-05-24 Thread via GitHub
dajac opened a new pull request, #16073: URL: https://github.com/apache/kafka/pull/16073 `getOrMaybeCreateClassicGroup` is mainly used by old APIs which expects UnknownMemberIdException when the group does not exist. ### Committer Checklist (excluded from commit message) - [ ] Veri

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-24 Thread via GitHub
junrao commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1612454534 ## server-common/src/main/java/org/apache/kafka/server/common/FinalizedFeatures.java: ## @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-16737: Enable unit tests for new consumer & cleanup TODOs [kafka]

2024-05-24 Thread via GitHub
lianetm commented on PR #16032: URL: https://github.com/apache/kafka/pull/16032#issuecomment-2129757705 Follow-up task [KAFKA-16823](https://issues.apache.org/jira/browse/KAFKA-16823) to refactor the test common functionality, and move out the unit tests that apply to the` LegacyConsumer`

Re: [PR] KAFKA-16574: The metrics of LogCleaner disappear after reconfiguration [kafka]

2024-05-24 Thread via GitHub
gaurav-narula commented on code in PR #15863: URL: https://github.com/apache/kafka/pull/15863#discussion_r1613630123 ## core/src/main/scala/kafka/log/LogCleaner.scala: ## @@ -223,8 +230,8 @@ class LogCleaner(initialConfig: CleanerConfig, info(s"Updating logCleanerIoMaxByt

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-24 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1613634445 ## server-common/src/main/java/org/apache/kafka/server/common/TestFeatureVersion.java: ## @@ -21,16 +21,17 @@ public enum TestFeatureVersion implements FeatureVersio

Re: [PR] KAFKA-16822: Abstract consumer group to share functionality with share group (KIP-932) [kafka]

2024-05-24 Thread via GitHub
dajac commented on code in PR #16054: URL: https://github.com/apache/kafka/pull/16054#discussion_r1613638293 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/AbstractGroup.java: ## @@ -0,0 +1,625 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-24 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1613639487 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -109,6 +111,52 @@ object StorageTool extends Logging { } } + private def validateMetadataVersion(

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-24 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1613639487 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -109,6 +111,52 @@ object StorageTool extends Logging { } } + private def validateMetadataVersion(

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-24 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1613656338 ## server-common/src/main/java/org/apache/kafka/server/common/TestFeatureVersion.java: ## @@ -21,16 +21,17 @@ public enum TestFeatureVersion implements FeatureVersio

[jira] [Commented] (KAFKA-16518) Storage tool changes for KIP-853

2024-05-24 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849332#comment-17849332 ] José Armando García Sancio commented on KAFKA-16518: HI [~muralibasa

Re: [PR] KAFKA-15045: (KIP-924 pt. 11) Implemented StickyTaskAssignor [kafka]

2024-05-24 Thread via GitHub
apourchet commented on code in PR #16052: URL: https://github.com/apache/kafka/pull/16052#discussion_r1613682502 ## streams/src/main/java/org/apache/kafka/streams/processor/assignment/assignors/StickyTaskAssignor.java: ## @@ -0,0 +1,478 @@ +/* + * Licensed to the Apache Software

[jira] [Commented] (KAFKA-16530) Fix high-watermark calculation to not assume the leader is in the voter set

2024-05-24 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849334#comment-17849334 ] José Armando García Sancio commented on KAFKA-16530: {quote}Perhaps

Re: [PR] KAFKA-16822: Abstract consumer group to share functionality with share group (KIP-932) [kafka]

2024-05-24 Thread via GitHub
dajac commented on code in PR #16054: URL: https://github.com/apache/kafka/pull/16054#discussion_r1613686172 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/AbstractGroup.java: ## @@ -0,0 +1,625 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[PR] KAFKA-15045: (KIP-924 pt. 13) Wiring in new assignment configs and logic [kafka]

2024-05-24 Thread via GitHub
apourchet opened a new pull request, #16074: URL: https://github.com/apache/kafka/pull/16074 This PR creates the new public config of KIP-924 in StreamsConfig and uses it to instantiate user-created TaskAssignors. If such a TaskAssignor is found and successfully created we then use that ass

Re: [PR] KAFKA-16832: LeaveGroup API for upgrading ConsumerGroup [kafka]

2024-05-24 Thread via GitHub
dongnuo123 commented on code in PR #16057: URL: https://github.com/apache/kafka/pull/16057#discussion_r1613724622 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -4424,14 +4441,113 @@ private ConsumerGroupMember validateCon

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-24 Thread via GitHub
junrao commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1613728690 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -109,6 +111,52 @@ object StorageTool extends Logging { } } + private def validateMetadataVersion(m

Re: [PR] KAFKA-15045: (KIP-924 pt. 13) Wiring in new assignment configs and logic [kafka]

2024-05-24 Thread via GitHub
agavra commented on code in PR #16074: URL: https://github.com/apache/kafka/pull/16074#discussion_r1613728858 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/AssignorConfiguration.java: ## @@ -253,6 +253,21 @@ public TaskAssignor taskAssignor() {

Re: [PR] KAFKA-16371; fix lingering pending commit when handling OFFSET_METADATA_TOO_LARGE [kafka]

2024-05-24 Thread via GitHub
chia7712 commented on code in PR #16072: URL: https://github.com/apache/kafka/pull/16072#discussion_r1613735230 ## core/src/test/scala/unit/kafka/coordinator/group/GroupMetadataManagerTest.scala: ## @@ -1664,6 +1664,134 @@ class GroupMetadataManagerTest { assertEquals(0, Te

Re: [PR] KAFKA-16371; fix lingering pending commit when handling OFFSET_METADATA_TOO_LARGE [kafka]

2024-05-24 Thread via GitHub
dajac commented on code in PR #16072: URL: https://github.com/apache/kafka/pull/16072#discussion_r1613744158 ## core/src/test/scala/unit/kafka/coordinator/group/GroupMetadataManagerTest.scala: ## @@ -1664,6 +1664,134 @@ class GroupMetadataManagerTest { assertEquals(0, TestU

Re: [PR] KAFKA-16371; fix lingering pending commit when handling OFFSET_METADATA_TOO_LARGE [kafka]

2024-05-24 Thread via GitHub
dajac commented on PR #16072: URL: https://github.com/apache/kafka/pull/16072#issuecomment-2129955822 > BTW, I notice that createPutCacheCallback has a input argument preAppendErrors which is never defined. Is it a unfinished feature or a stuff we can remove. Hum... I am not sure abo

Re: [PR] KAFKA-16832: LeaveGroup API for upgrading ConsumerGroup [kafka]

2024-05-24 Thread via GitHub
dongnuo123 commented on code in PR #16057: URL: https://github.com/apache/kafka/pull/16057#discussion_r1613748822 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -4424,14 +4441,113 @@ private ConsumerGroupMember validateCon

[jira] [Commented] (KAFKA-16530) Fix high-watermark calculation to not assume the leader is in the voter set

2024-05-24 Thread Alyssa Huang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17849342#comment-17849342 ] Alyssa Huang commented on KAFKA-16530: -- [~jsancio]  > This is not correct because t

Re: [PR] KAFKA-16452: Don't throw OOORE when converting the offset to metadata [kafka]

2024-05-24 Thread via GitHub
kamalcph commented on PR #15825: URL: https://github.com/apache/kafka/pull/15825#issuecomment-2129976827 Test failures are unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-24 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1613761681 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -109,6 +111,52 @@ object StorageTool extends Logging { } } + private def validateMetadataVersion(

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-24 Thread via GitHub
omkreddy merged PR #16027: URL: https://github.com/apache/kafka/pull/16027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-16516: Fix the controller node provider for broker to control channel [kafka]

2024-05-24 Thread via GitHub
cmccabe closed pull request #16008: KAFKA-16516: Fix the controller node provider for broker to control channel URL: https://github.com/apache/kafka/pull/16008 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-16516: Fix the controller node provider for broker to control channel [kafka]

2024-05-24 Thread via GitHub
cmccabe commented on PR #16008: URL: https://github.com/apache/kafka/pull/16008#issuecomment-2129992325 merged (from CLI, since GH is acting flaky) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] MINOR: Disable JDK 11 and 17 tests on PRs [kafka]

2024-05-24 Thread via GitHub
gharris1727 commented on PR #16051: URL: https://github.com/apache/kafka/pull/16051#issuecomment-2129998314 Hey @jolshan @mumrah @ijuma PTAL at the build to see the effect, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] KAFKA-16832: LeaveGroup API for upgrading ConsumerGroup [kafka]

2024-05-24 Thread via GitHub
dajac commented on code in PR #16057: URL: https://github.com/apache/kafka/pull/16057#discussion_r1613785754 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -4424,14 +4441,113 @@ private ConsumerGroupMember validateConsumer

[PR] Improve producer ID expiration performance [kafka]

2024-05-24 Thread via GitHub
CalvinConfluent opened a new pull request, #16075: URL: https://github.com/apache/kafka/pull/16075 Skip using steam when expiring the producer ID. This can improve the performance more than 30% Before ``` Benchmark(numProducerIds) Mode Cnt

Re: [PR] Improve producer ID expiration performance [kafka]

2024-05-24 Thread via GitHub
jolshan commented on PR #16075: URL: https://github.com/apache/kafka/pull/16075#issuecomment-2130025720 Thanks Calvin -- similar to this PR https://github.com/apache/kafka/pull/15969 I will take a look soon :) -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] KAFKA-16515: Fix the ZK Metadata cache confusion between brokers and controllers [kafka]

2024-05-24 Thread via GitHub
cmccabe closed pull request #16006: KAFKA-16515: Fix the ZK Metadata cache confusion between brokers and controllers URL: https://github.com/apache/kafka/pull/16006 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-16515: Fix the ZK Metadata cache confusion between brokers and controllers [kafka]

2024-05-24 Thread via GitHub
cmccabe commented on PR #16006: URL: https://github.com/apache/kafka/pull/16006#issuecomment-2130026904 failures not related. committed, thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-16574: The metrics of LogCleaner disappear after reconfiguration [kafka]

2024-05-24 Thread via GitHub
rishiraj88 commented on code in PR #15863: URL: https://github.com/apache/kafka/pull/15863#discussion_r1613799763 ## core/src/main/scala/kafka/log/LogCleaner.scala: ## @@ -223,8 +230,8 @@ class LogCleaner(initialConfig: CleanerConfig, info(s"Updating logCleanerIoMaxBytesP

Re: [PR] KAFKA-16371; fix lingering pending commit when handling OFFSET_METADATA_TOO_LARGE [kafka]

2024-05-24 Thread via GitHub
jolshan commented on PR #16072: URL: https://github.com/apache/kafka/pull/16072#issuecomment-2130038593 > Hum... I am not sure about this one. @jolshan Would you know? There was a lot of refactoring in this path so we may have just missed this. It was originally intended for any verif

Re: [PR] MINOR: mention KAFKA-15905 in docs "Notable changes in 3.7.1" [kafka]

2024-05-24 Thread via GitHub
edoardocomar commented on PR #16070: URL: https://github.com/apache/kafka/pull/16070#issuecomment-2130039173 Do you want an empty one? I’d assume what’s new for 3.7.1 is Included in 3.8 … -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] KAFKA-16371; fix lingering pending commit when handling OFFSET_METADATA_TOO_LARGE [kafka]

2024-05-24 Thread via GitHub
chia7712 commented on code in PR #16072: URL: https://github.com/apache/kafka/pull/16072#discussion_r1613803124 ## core/src/test/scala/unit/kafka/coordinator/group/GroupMetadataManagerTest.scala: ## @@ -1664,6 +1664,134 @@ class GroupMetadataManagerTest { assertEquals(0, Te

Re: [PR] KAFKA-16625: Reverse lookup map from topic partitions to members [kafka]

2024-05-24 Thread via GitHub
jeffkbkim commented on code in PR #15974: URL: https://github.com/apache/kafka/pull/15974#discussion_r1613806818 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/assignor/GroupSpecImplTest.java: ## @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] KAFKA-16625: Reverse lookup map from topic partitions to members [kafka]

2024-05-24 Thread via GitHub
jeffkbkim commented on code in PR #15974: URL: https://github.com/apache/kafka/pull/15974#discussion_r1613806818 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/assignor/GroupSpecImplTest.java: ## @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] MINOR: mention KAFKA-15905 in docs "Notable changes in 3.7.1" [kafka]

2024-05-24 Thread via GitHub
gharris1727 commented on PR #16070: URL: https://github.com/apache/kafka/pull/16070#issuecomment-2130052791 > I’d assume what’s new for 3.7.1 is Included in 3.8 … I don't think everyone thinks this way. Someone can easily read the 3.8.0 release notes and never read the 3.7.1 notes. I'

  1   2   >