[jira] [Updated] (KAFKA-16779) Kafka retains logs past specified retention

2024-05-16 Thread Lin Siyuan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Siyuan updated KAFKA-16779: --- Description: 在将所有主题设置为保留四天或更长时间(34560毫秒)的 Kafka 集群中,大多数代理似乎保留了六天的数据。 即使对于具有高吞吐量(500MB/s,50k 消息/

[jira] [Updated] (KAFKA-16779) Kafka retains logs past specified retention

2024-05-16 Thread Lin Siyuan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Siyuan updated KAFKA-16779: --- Description: In a Kafka cluster with all topics set to four days of retention or longer (34560m

[jira] [Commented] (KAFKA-15242) FixedKeyProcessor testing is unusable

2024-05-16 Thread Matej Sprysl (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846852#comment-17846852 ] Matej Sprysl commented on KAFKA-15242: -- Hi Matthias, did not find the issue you ar

[jira] [Commented] (KAFKA-16779) Kafka retains logs past specified retention

2024-05-16 Thread Lin Siyuan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846853#comment-17846853 ] Lin Siyuan commented on KAFKA-16779: I'm very sorry,Nicholas Feinberg. I misinterpre

Re: [PR] MINOR: Remove unused method in ToolsUtils [kafka]

2024-05-16 Thread via GitHub
chia7712 merged PR #15967: URL: https://github.com/apache/kafka/pull/15967 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Assigned] (KAFKA-16771) First log directory printed twice when formatting storage

2024-05-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reassigned KAFKA-16771: -- Assignee: xuanzhang gong > First log directory printed twice when formatting storage

[jira] [Commented] (KAFKA-16771) First log directory printed twice when formatting storage

2024-05-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846858#comment-17846858 ] Chia-Ping Tsai commented on KAFKA-16771: [~gongxuanzhang] I have assigned this t

Re: [PR] KAFKA-16763: Upgrade to scala 2.12.19 and scala 2.13.14 [kafka]

2024-05-16 Thread via GitHub
chia7712 commented on code in PR #15958: URL: https://github.com/apache/kafka/pull/15958#discussion_r1602785679 ## build.gradle: ## @@ -671,7 +671,8 @@ subprojects { "-Xlint:private-shadow", "-Xlint:stars-align", "-Xlint:type-parameter-shadow", - "-Xlin

Re: [PR] KAFKA-16762: SyncGroup API for upgrading ConsumerGroup [kafka]

2024-05-16 Thread via GitHub
dajac commented on code in PR #15954: URL: https://github.com/apache/kafka/pull/15954#discussion_r1602768290 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -1197,6 +1199,82 @@ private void throwIfClassicProtocolIsNotSupport

Re: [PR] KAFKA-16763: Upgrade to scala 2.12.19 and scala 2.13.14 [kafka]

2024-05-16 Thread via GitHub
m1a2st commented on code in PR #15958: URL: https://github.com/apache/kafka/pull/15958#discussion_r1602821237 ## build.gradle: ## @@ -671,7 +671,8 @@ subprojects { "-Xlint:private-shadow", "-Xlint:stars-align", "-Xlint:type-parameter-shadow", - "-Xlint:

Re: [PR] KAFKA-16763: Upgrade to scala 2.12.19 and scala 2.13.14 [kafka]

2024-05-16 Thread via GitHub
m1a2st commented on code in PR #15958: URL: https://github.com/apache/kafka/pull/15958#discussion_r1602821237 ## build.gradle: ## @@ -671,7 +671,8 @@ subprojects { "-Xlint:private-shadow", "-Xlint:stars-align", "-Xlint:type-parameter-shadow", - "-Xlint:

Re: [PR] KAFKA-16763: Upgrade to scala 2.12.19 and scala 2.13.14 [kafka]

2024-05-16 Thread via GitHub
m1a2st commented on code in PR #15958: URL: https://github.com/apache/kafka/pull/15958#discussion_r1602821237 ## build.gradle: ## @@ -671,7 +671,8 @@ subprojects { "-Xlint:private-shadow", "-Xlint:stars-align", "-Xlint:type-parameter-shadow", - "-Xlint:

Re: [PR] KAFKA-16763: Upgrade to scala 2.12.19 and scala 2.13.14 [kafka]

2024-05-16 Thread via GitHub
chia7712 commented on code in PR #15958: URL: https://github.com/apache/kafka/pull/15958#discussion_r1602851781 ## build.gradle: ## @@ -671,7 +671,8 @@ subprojects { "-Xlint:private-shadow", "-Xlint:stars-align", "-Xlint:type-parameter-shadow", - "-Xlin

Re: [PR] KAFKA-16763: Upgrade to scala 2.12.19 and scala 2.13.14 [kafka]

2024-05-16 Thread via GitHub
m1a2st commented on code in PR #15958: URL: https://github.com/apache/kafka/pull/15958#discussion_r1602865064 ## build.gradle: ## @@ -671,7 +671,8 @@ subprojects { "-Xlint:private-shadow", "-Xlint:stars-align", "-Xlint:type-parameter-shadow", - "-Xlint:

Re: [PR] KAFKA-16705 the flag "started" of RaftClusterInstance is false even though the cluster is started [kafka]

2024-05-16 Thread via GitHub
chia7712 commented on code in PR #15946: URL: https://github.com/apache/kafka/pull/15946#discussion_r1602875475 ## core/src/test/java/kafka/test/junit/RaftClusterInvocationContext.java: ## @@ -284,24 +259,59 @@ public void startBroker(int brokerId) { @Override

Re: [PR] KAFKA-16763: Upgrade to scala 2.12.19 and scala 2.13.14 [kafka]

2024-05-16 Thread via GitHub
m1a2st commented on code in PR #15958: URL: https://github.com/apache/kafka/pull/15958#discussion_r1602889444 ## build.gradle: ## @@ -671,7 +671,8 @@ subprojects { "-Xlint:private-shadow", "-Xlint:stars-align", "-Xlint:type-parameter-shadow", - "-Xlint:

[jira] [Created] (KAFKA-16780) Txn consumer exerts pressure on remote storage when reading non-txn topic

2024-05-16 Thread Kamal Chandraprakash (Jira)
Kamal Chandraprakash created KAFKA-16780: Summary: Txn consumer exerts pressure on remote storage when reading non-txn topic Key: KAFKA-16780 URL: https://issues.apache.org/jira/browse/KAFKA-16780

[jira] [Updated] (KAFKA-16780) Txn consumer exerts pressure on remote storage when reading non-txn topic

2024-05-16 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-16780: - Description: h3. Logic to read aborted txns: # When the consumer enables isolat

[jira] [Updated] (KAFKA-16780) Txn consumer exerts pressure on remote storage when reading non-txn topic

2024-05-16 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-16780: - Description: h3. Logic to read aborted txns: # When the consumer enables isolat

[jira] [Updated] (KAFKA-16780) Txn consumer exerts pressure on remote storage when reading non-txn topic

2024-05-16 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-16780: - Description: h3. Logic to read aborted txns: # When the consumer enables isolat

[jira] [Commented] (KAFKA-16448) Add Kafka Streams exception handler for exceptions occuring during processing (KIP-1033)

2024-05-16 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-16448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846880#comment-17846880 ] Loïc Greffier commented on KAFKA-16448: --- [~mjsax] - You can assign it to me > Add

Re: [PR] KAFKA-16598 Mirgrate `ResetConsumerGroupOffsetTest` to new test infra [kafka]

2024-05-16 Thread via GitHub
m1a2st commented on PR #15779: URL: https://github.com/apache/kafka/pull/15779#issuecomment-2114598318 @chia7712, Thanks for your conversations, I already rebase this code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] KAFKA-16709: abortAndPauseCleaning only when future log is not existed [kafka]

2024-05-16 Thread via GitHub
chia7712 commented on code in PR #15951: URL: https://github.com/apache/kafka/pull/15951#discussion_r1602929057 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -314,6 +314,77 @@ class ReplicaManagerTest { } } + @ParameterizedTest(name = "testM

[jira] [Updated] (KAFKA-16781) Expose advertised.listeners in controller node

2024-05-16 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-16781: -- Labels: need-kip newbie newbie++ (was: ) > Expose advertised.listeners in controller node > -

[jira] [Created] (KAFKA-16781) Expose advertised.listeners in controller node

2024-05-16 Thread Luke Chen (Jira)
Luke Chen created KAFKA-16781: - Summary: Expose advertised.listeners in controller node Key: KAFKA-16781 URL: https://issues.apache.org/jira/browse/KAFKA-16781 Project: Kafka Issue Type: Improvem

Re: [PR] KAFKA-16763: Upgrade to scala 2.12.19 and scala 2.13.14 [kafka]

2024-05-16 Thread via GitHub
chia7712 commented on code in PR #15958: URL: https://github.com/apache/kafka/pull/15958#discussion_r1602988530 ## build.gradle: ## @@ -671,7 +671,8 @@ subprojects { "-Xlint:private-shadow", "-Xlint:stars-align", "-Xlint:type-parameter-shadow", - "-Xlin

Re: [PR] Cherry-pick KAFKA-16539 to 3.7 [kafka]

2024-05-16 Thread via GitHub
chia7712 merged PR #15953: URL: https://github.com/apache/kafka/pull/15953 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Updated] (KAFKA-16539) Can't update specific broker configs in pre-migration mode

2024-05-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-16539: --- Fix Version/s: (was: 3.6.3) > Can't update specific broker configs in pre-migration mode

[jira] [Resolved] (KAFKA-16539) Can't update specific broker configs in pre-migration mode

2024-05-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16539. Resolution: Fixed > Can't update specific broker configs in pre-migration mode > -

[jira] [Commented] (KAFKA-16781) Expose advertised.listeners in controller node

2024-05-16 Thread TengYao Chi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846890#comment-17846890 ] TengYao Chi commented on KAFKA-16781: - I will handle this issue :) > Expose adverti

[jira] [Assigned] (KAFKA-16781) Expose advertised.listeners in controller node

2024-05-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reassigned KAFKA-16781: -- Assignee: TengYao Chi > Expose advertised.listeners in controller node >

Re: [PR] [Draft] KAFKA-16764: New consumer should throw InvalidTopicException on poll when invalid topic in metadata [kafka]

2024-05-16 Thread via GitHub
appchemist commented on code in PR #15961: URL: https://github.com/apache/kafka/pull/15961#discussion_r1603033581 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1607,6 +1607,7 @@ private Fetch pollForFetches(Timer timer) {

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-16 Thread via GitHub
dajac commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1602958576 ## server-common/src/main/java/org/apache/kafka/server/common/FeatureVersionUtils.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] [Draft] KAFKA-16764: New consumer should throw InvalidTopicException on poll when invalid topic in metadata [kafka]

2024-05-16 Thread via GitHub
appchemist commented on code in PR #15961: URL: https://github.com/apache/kafka/pull/15961#discussion_r1603033581 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1607,6 +1607,7 @@ private Fetch pollForFetches(Timer timer) {

Re: [PR] Kafka-16668: Add tags support in ClusterTestExtension [kafka]

2024-05-16 Thread via GitHub
chia7712 commented on PR #15861: URL: https://github.com/apache/kafka/pull/15861#issuecomment-2114763118 ``` ./gradlew cleanTest :connect:runtime:test --tests org.apache.kafka.connect.integration.ExactlyOnceSourceIntegrationTest.testSeparateOffsetsTopic --tests org.apache.kafka.connect.

Re: [PR] Kafka-16668: Add tags support in ClusterTestExtension [kafka]

2024-05-16 Thread via GitHub
chia7712 merged PR #15861: URL: https://github.com/apache/kafka/pull/15861 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Resolved] (KAFKA-16668) Enable to set tags by `ClusterTest`

2024-05-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16668. Fix Version/s: 3.8.0 Resolution: Fixed > Enable to set tags by `ClusterTest` > ---

Re: [PR] KAFKA-14588 [3/N] ConfigCommandTest rewritten in java [kafka]

2024-05-16 Thread via GitHub
chia7712 merged PR #15930: URL: https://github.com/apache/kafka/pull/15930 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-14588 [3/N] ConfigCommandTest rewritten in java [kafka]

2024-05-16 Thread via GitHub
chia7712 commented on PR #15930: URL: https://github.com/apache/kafka/pull/15930#issuecomment-2114777884 @nizhikov nice patch! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] MINOR: Remove deprecated constructors from Connect's Kafka*BackingStore classes [kafka]

2024-05-16 Thread via GitHub
chia7712 commented on PR #15865: URL: https://github.com/apache/kafka/pull/15865#issuecomment-2114786021 @yashmayya any update? I'm ok to merge it and open follow-up to address remaining comments. -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [PR] KAFKA-16625: Reverse lookup map from topic partitions to members [kafka]

2024-05-16 Thread via GitHub
dajac commented on code in PR #15974: URL: https://github.com/apache/kafka/pull/15974#discussion_r1603040292 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -147,6 +147,11 @@ public static class DeadlineAndEpoch { */

Re: [PR] KAFKA-14588 [4/N] ConfigCommandTest rewritten in java [kafka]

2024-05-16 Thread via GitHub
nizhikov commented on PR #15839: URL: https://github.com/apache/kafka/pull/15839#issuecomment-2114863966 Hello @chia7712 Looks like we are ready to replace whole scala version of `ConfigCommandTest` with the java one. Changes are reviewable now. Can you, please, take a look?

Re: [PR] KAFKA-14588 [3/N] ConfigCommandTest rewritten in java [kafka]

2024-05-16 Thread via GitHub
nizhikov commented on PR #15930: URL: https://github.com/apache/kafka/pull/15930#issuecomment-2114875511 @chia7712 Thanks for the review and merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] [Draft] KAFKA-16764: New consumer should throw InvalidTopicException on poll when invalid topic in metadata [kafka]

2024-05-16 Thread via GitHub
appchemist commented on code in PR #15961: URL: https://github.com/apache/kafka/pull/15961#discussion_r1603033581 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1607,6 +1607,7 @@ private Fetch pollForFetches(Timer timer) {

Re: [PR] MINOR: Remove deprecated constructors from Connect's Kafka*BackingStore classes [kafka]

2024-05-16 Thread via GitHub
yashmayya commented on code in PR #15865: URL: https://github.com/apache/kafka/pull/15865#discussion_r1603105885 ## connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaStatusBackingStore.java: ## @@ -160,7 +155,7 @@ public KafkaStatusBackingStore(Time time, Conve

[jira] [Commented] (KAFKA-16774) fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEnabled

2024-05-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846912#comment-17846912 ] Chia-Ping Tsai commented on KAFKA-16774: [~mjsax] nice question. It seems to me

Re: [PR] KAFKA-10199: Handle assignment with new remove operation in state updater [kafka]

2024-05-16 Thread via GitHub
cadonna commented on code in PR #15882: URL: https://github.com/apache/kafka/pull/15882#discussion_r1603147808 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -1750,6 +1761,7 @@ Map allTasks() { if (stateUpdater != null) {

[jira] [Comment Edited] (KAFKA-16774) fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEnabled

2024-05-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846912#comment-17846912 ] Chia-Ping Tsai edited comment on KAFKA-16774 at 5/16/24 11:11 AM:

[jira] [Updated] (KAFKA-15541) Improved StateStore Iterator metrics for detecting leaks

2024-05-16 Thread Nicholas Telford (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Telford updated KAFKA-15541: - Summary: Improved StateStore Iterator metrics for detecting leaks (was: RocksDB Iterato

Re: [PR] KAFKA-15541: Add num-open-iterators metric [kafka]

2024-05-16 Thread via GitHub
nicktelford commented on PR #15975: URL: https://github.com/apache/kafka/pull/15975#issuecomment-2114986567 @mjsax @lucasbru @ableegoldman I know the vote hasn't closed yet, but I thought I'd get a head-start on the review. I'm submitting each metric from KIP-989 as a separate PR, to aid re

Re: [PR] KAFKA-16654:Refactor kafka.test.annotation.Type and ClusterTestExtensions [kafka]

2024-05-16 Thread via GitHub
chia7712 commented on code in PR #15916: URL: https://github.com/apache/kafka/pull/15916#discussion_r1603194691 ## core/src/test/java/kafka/test/junit/ClusterTestExtensionsUnitTest.java: ## @@ -33,16 +31,16 @@ public class ClusterTestExtensionsUnitTest { void testProcessClu

Re: [PR] KAFKA-16654:Refactor kafka.test.annotation.Type and ClusterTestExtensions [kafka]

2024-05-16 Thread via GitHub
chia7712 commented on code in PR #15916: URL: https://github.com/apache/kafka/pull/15916#discussion_r1603229776 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/ConsumerGroupCommandTestUtils.java: ## @@ -71,7 +72,8 @@ static void generator(ClusterGenerator clusterGen

Re: [PR] KAFKA-16493: Avoid unneeded subscription regex check if metadata version unchanged [kafka]

2024-05-16 Thread via GitHub
Phuc-Hong-Tran commented on PR #15869: URL: https://github.com/apache/kafka/pull/15869#issuecomment-2115102574 @lianetm, I've added the required test and refactored according to your advices, may you have a look once you have time? Thanks. -- This is an automated message from the Apache G

[jira] [Commented] (KAFKA-16774) fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEnabled

2024-05-16 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846940#comment-17846940 ] Bruno Cadonna commented on KAFKA-16774: --- [~chia7712] Thank you for your analysis!

[jira] [Created] (KAFKA-16782) Some partition's segments are suddenly not deleted anymore

2024-05-16 Thread Roland Sommer (Jira)
Roland Sommer created KAFKA-16782: - Summary: Some partition's segments are suddenly not deleted anymore Key: KAFKA-16782 URL: https://issues.apache.org/jira/browse/KAFKA-16782 Project: Kafka

[PR] MINOR: Refactor write timeout in CoordinatorRuntime [kafka]

2024-05-16 Thread via GitHub
dajac opened a new pull request, #15976: URL: https://github.com/apache/kafka/pull/15976 This patch is a small refactor in the CoordinatorRuntime. It relies on existing tests for correctness. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementat

[PR] MINOR: Add debug enablement check when using log.debug [kafka]

2024-05-16 Thread via GitHub
johnnychhsu opened a new pull request, #15977: URL: https://github.com/apache/kafka/pull/15977 ## Context When using `log.debug()`, we should check whether the debug log level is enabled or not ## Solution Check whether debug level is enabled before logging ### Committer

Re: [PR] KAFKA-10551: Add topic id support to produce request and response [kafka]

2024-05-16 Thread via GitHub
OmniaGM commented on PR #15968: URL: https://github.com/apache/kafka/pull/15968#issuecomment-2115190250 Few of the failed tests are related to this change and am working on fixing them -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] KAFKA-7632: Support Compression Levels (KIP-390) [kafka]

2024-05-16 Thread via GitHub
mimaison commented on code in PR #15516: URL: https://github.com/apache/kafka/pull/15516#discussion_r1603325731 ## server-common/src/test/java/org/apache/kafka/server/record/BrokerCompressionTypeTest.java: ## @@ -16,23 +16,38 @@ */ package org.apache.kafka.server.record; +i

Re: [PR] KAFKA-7632: Support Compression Levels (KIP-390) [kafka]

2024-05-16 Thread via GitHub
mimaison commented on PR #15516: URL: https://github.com/apache/kafka/pull/15516#issuecomment-2115225074 I also added a couple of new tests in LogValidatorTest to check recompression only happens if the compression codec is different between the records from the producer and the topic confi

[jira] [Commented] (KAFKA-16774) fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEnabled

2024-05-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846950#comment-17846950 ] Chia-Ping Tsai commented on KAFKA-16774: [~cadonna] thanks for your confirm. Ple

Re: [PR] MINOR: rewrite TopicBasedRemoteLogMetadataManagerTest by ClusterTestE… [kafka]

2024-05-16 Thread via GitHub
chia7712 merged PR #15917: URL: https://github.com/apache/kafka/pull/15917 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-16493: Avoid unneeded subscription regex check if metadata version unchanged [kafka]

2024-05-16 Thread via GitHub
lianetm commented on code in PR #15869: URL: https://github.com/apache/kafka/pull/15869#discussion_r1603351190 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1968,9 +1972,13 @@ SubscriptionState subscriptions() { }

Re: [PR] KAFKA-16493: Avoid unneeded subscription regex check if metadata version unchanged [kafka]

2024-05-16 Thread via GitHub
lianetm commented on code in PR #15869: URL: https://github.com/apache/kafka/pull/15869#discussion_r1603351190 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1968,9 +1972,13 @@ SubscriptionState subscriptions() { }

[jira] [Created] (KAFKA-16783) Migrate RemoteLogMetadataManagerTest to new test infra

2024-05-16 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16783: -- Summary: Migrate RemoteLogMetadataManagerTest to new test infra Key: KAFKA-16783 URL: https://issues.apache.org/jira/browse/KAFKA-16783 Project: Kafka Is

Re: [PR] KAFKA-16493: Avoid unneeded subscription regex check if metadata version unchanged [kafka]

2024-05-16 Thread via GitHub
Phuc-Hong-Tran commented on code in PR #15869: URL: https://github.com/apache/kafka/pull/15869#discussion_r1603357750 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1968,9 +1972,13 @@ SubscriptionState subscriptions() {

[jira] [Created] (KAFKA-16784) Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest to new test infra

2024-05-16 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16784: -- Summary: Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest to new test infra Key: KAFKA-16784 URL: https://issues.apache.org/jira/browse/KAFKA-16784

Re: [PR] KAFKA-16493: Avoid unneeded subscription regex check if metadata version unchanged [kafka]

2024-05-16 Thread via GitHub
Phuc-Hong-Tran commented on code in PR #15869: URL: https://github.com/apache/kafka/pull/15869#discussion_r1603361345 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1968,9 +1972,13 @@ SubscriptionState subscriptions() {

[jira] [Commented] (KAFKA-16783) Migrate RemoteLogMetadataManagerTest to new test infra

2024-05-16 Thread PoAn Yang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846955#comment-17846955 ] PoAn Yang commented on KAFKA-16783: --- Hi [~chia7712], I'm interested in this issue. May

[jira] [Created] (KAFKA-16785) Migrate TopicBasedRemoteLogMetadataManagerRestartTest to new test infra

2024-05-16 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16785: -- Summary: Migrate TopicBasedRemoteLogMetadataManagerRestartTest to new test infra Key: KAFKA-16785 URL: https://issues.apache.org/jira/browse/KAFKA-16785 Project:

[jira] [Updated] (KAFKA-16783) Migrate RemoteLogMetadataManagerTest to new test infra

2024-05-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-16783: --- Labels: storage_test (was: ) > Migrate RemoteLogMetadataManagerTest to new test infra > ---

[jira] [Updated] (KAFKA-16784) Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest to new test infra

2024-05-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-16784: --- Labels: storage_test (was: ) > Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscripti

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-16 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1603368871 ## server-common/src/main/java/org/apache/kafka/server/common/FeatureVersionUtils.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[jira] [Assigned] (KAFKA-16783) Migrate RemoteLogMetadataManagerTest to new test infra

2024-05-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reassigned KAFKA-16783: -- Assignee: PoAn Yang (was: Chia-Ping Tsai) > Migrate RemoteLogMetadataManagerTest to

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-16 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1603375064 ## server-common/src/main/java/org/apache/kafka/server/common/FeatureVersionUtils.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-16 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1603375064 ## server-common/src/main/java/org/apache/kafka/server/common/FeatureVersionUtils.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-16 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1603376757 ## server-common/src/main/java/org/apache/kafka/server/common/FeatureVersion.java: ## @@ -0,0 +1,137 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-16 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1603378178 ## server-common/src/main/java/org/apache/kafka/server/common/FeatureVersion.java: ## @@ -0,0 +1,137 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

[jira] [Commented] (KAFKA-16774) fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEnabled

2024-05-16 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846959#comment-17846959 ] Bruno Cadonna commented on KAFKA-16774: --- Looking at the test, I think we can remov

Re: [PR] KAFKA-16992: InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka [kafka]

2024-05-16 Thread via GitHub
dajac commented on code in PR #15971: URL: https://github.com/apache/kafka/pull/15971#discussion_r1603385828 ## tests/kafkatest/tests/core/transactions_mixed_versions_test.py: ## @@ -0,0 +1,226 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contribut

Re: [PR] MINOR: Refactor write timeout in CoordinatorRuntime [kafka]

2024-05-16 Thread via GitHub
chia7712 commented on code in PR #15976: URL: https://github.com/apache/kafka/pull/15976#discussion_r1603390406 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -541,6 +541,28 @@ private void unload() { }

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-16 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1603394445 ## server-common/src/main/java/org/apache/kafka/server/common/TestFeatureVersion.java: ## @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

Re: [PR] MINOR: Refactor write timeout in CoordinatorRuntime [kafka]

2024-05-16 Thread via GitHub
dajac commented on code in PR #15976: URL: https://github.com/apache/kafka/pull/15976#discussion_r1603394774 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -541,6 +541,28 @@ private void unload() { } }

Re: [PR] MINOR: Refactor write timeout in CoordinatorRuntime [kafka]

2024-05-16 Thread via GitHub
dajac commented on code in PR #15976: URL: https://github.com/apache/kafka/pull/15976#discussion_r1603395535 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -541,6 +541,28 @@ private void unload() { } }

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-16 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1603396094 ## server-common/src/main/java/org/apache/kafka/server/common/FeatureVersionUtils.java: ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[jira] [Assigned] (KAFKA-16774) fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEnabled

2024-05-16 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna reassigned KAFKA-16774: - Assignee: Bruno Cadonna > fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnClose

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-16 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1603398767 ## metadata/src/main/java/org/apache/kafka/controller/ClusterControlManager.java: ## @@ -459,18 +459,20 @@ BrokerFeature processRegistrationFeature( FinalizedC

Re: [PR] MINOR: Refactor write timeout in CoordinatorRuntime [kafka]

2024-05-16 Thread via GitHub
dajac commented on code in PR #15976: URL: https://github.com/apache/kafka/pull/15976#discussion_r1603398900 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -541,6 +541,28 @@ private void unload() { } }

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-16 Thread via GitHub
jsancio merged PR #15859: URL: https://github.com/apache/kafka/pull/15859 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] KAFKA-16992: InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka [kafka]

2024-05-16 Thread via GitHub
jolshan commented on code in PR #15971: URL: https://github.com/apache/kafka/pull/15971#discussion_r1603399610 ## tests/kafkatest/tests/core/transactions_mixed_versions_test.py: ## @@ -0,0 +1,226 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contrib

Re: [PR] KAFKA-16992: InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka [kafka]

2024-05-16 Thread via GitHub
jolshan commented on code in PR #15971: URL: https://github.com/apache/kafka/pull/15971#discussion_r1603400713 ## core/src/main/java/kafka/server/NetworkUtils.java: ## @@ -82,7 +82,7 @@ public static NetworkClient buildNetworkClient(String prefix, config.connectionS

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-16 Thread via GitHub
dajac commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1603437357 ## server-common/src/main/java/org/apache/kafka/server/common/FeatureVersion.java: ## @@ -0,0 +1,137 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[jira] [Created] (KAFKA-16786) New consumer subscribe should not require the deprecated partition.assignment.strategy

2024-05-16 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-16786: -- Summary: New consumer subscribe should not require the deprecated partition.assignment.strategy Key: KAFKA-16786 URL: https://issues.apache.org/jira/browse/KAFKA-16786

Re: [PR] KAFKA-16493: Avoid unneeded subscription regex check if metadata version unchanged [kafka]

2024-05-16 Thread via GitHub
lianetm commented on code in PR #15869: URL: https://github.com/apache/kafka/pull/15869#discussion_r1603504298 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -463,6 +467,42 @@ public void onPartitionsAssigned(final Collec

[jira] [Commented] (KAFKA-16786) New consumer subscribe should not require the deprecated partition.assignment.strategy

2024-05-16 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846985#comment-17846985 ] Lianet Magrans commented on KAFKA-16786: Hey [~phuctran] , I noticed this issue

[jira] [Updated] (KAFKA-16786) New consumer subscribe should not require the deprecated partition.assignment.strategy

2024-05-16 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16786: --- Fix Version/s: 3.8 > New consumer subscribe should not require the deprecated > partition.a

Re: [PR] KAFKA-16762: SyncGroup API for upgrading ConsumerGroup [kafka]

2024-05-16 Thread via GitHub
dongnuo123 commented on code in PR #15954: URL: https://github.com/apache/kafka/pull/15954#discussion_r1603521402 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -1197,6 +1199,82 @@ private void throwIfClassicProtocolIsNotSu

[jira] [Commented] (KAFKA-16786) New consumer subscribe should not require the deprecated partition.assignment.strategy

2024-05-16 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846990#comment-17846990 ] Phuc Hong Tran commented on KAFKA-16786: Sure, @lianetm. I’ll take it > New con

Re: [PR] KAFKA-16762: SyncGroup API for upgrading ConsumerGroup [kafka]

2024-05-16 Thread via GitHub
dajac commented on code in PR #15954: URL: https://github.com/apache/kafka/pull/15954#discussion_r1603525427 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -1244,15 +1244,16 @@ private void throwIfClassicProtocolUnmatched(

[jira] [Comment Edited] (KAFKA-16786) New consumer subscribe should not require the deprecated partition.assignment.strategy

2024-05-16 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846990#comment-17846990 ] Phuc Hong Tran edited comment on KAFKA-16786 at 5/16/24 2:53 PM: -

  1   2   >