Re: [PR] MINOR: Add flag to read from specific partitions while checking consumer performance [kafka]

2024-05-15 Thread via GitHub
BOOTMGR commented on PR #15905: URL: https://github.com/apache/kafka/pull/15905#issuecomment-2111745484 > Hey @BOOTMGR, I would expect we need a KIP for this. It brings changes in the command line tool arguments, which is considered a public interface (check KIP page [here](https://cwiki.a

[jira] (KAFKA-16616) refactor mergeWith in MetadataSnapshot

2024-05-15 Thread Cao Manh Dat (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16616 ] Cao Manh Dat deleted comment on KAFKA-16616: -- was (Author: caomanhdat): Hi [~alyssahuang] , can I work on this item? > refactor mergeWith in MetadataSnapshot > -

[jira] [Commented] (KAFKA-16361) Rack aware sticky assignor minQuota violations

2024-05-15 Thread Laymain (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846522#comment-17846522 ] Laymain commented on KAFKA-16361: - Thanks [~ableegoldman] and [~flashmouse], I am glad t

[jira] [Created] (KAFKA-16769) Delete deprecated add.source.alias.to.metrics configuration

2024-05-15 Thread Mickael Maison (Jira)
Mickael Maison created KAFKA-16769: -- Summary: Delete deprecated add.source.alias.to.metrics configuration Key: KAFKA-16769 URL: https://issues.apache.org/jira/browse/KAFKA-16769 Project: Kafka

Re: [PR] KAFKA-14588 [3/N] ConfigCommandTest rewritten in java [kafka]

2024-05-15 Thread via GitHub
nizhikov commented on code in PR #15930: URL: https://github.com/apache/kafka/pull/15930#discussion_r1601121898 ## core/src/test/java/kafka/admin/ConfigCommandUnitTest.java: ## @@ -878,6 +886,486 @@ public void shouldNotDescribeUserScramCredentialsWithEntityDefaultUsingBootstra

Re: [PR] KAFKA-10199: Remove queue-based remove from state updater [kafka]

2024-05-15 Thread via GitHub
cadonna merged PR #15896: URL: https://github.com/apache/kafka/pull/15896 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[PR] KAFKA-16712: Fix race in TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest [kafka]

2024-05-15 Thread via GitHub
gaurav-narula opened a new pull request, #15962: URL: https://github.com/apache/kafka/pull/15962 `TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest` has a race when it sets `RemoteLogMetadataTopicPartitioner` using the setter. This change fixes the race condition by passing

Re: [PR] KAFKA-16712: Fix race in TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest [kafka]

2024-05-15 Thread via GitHub
gaurav-narula commented on PR #15962: URL: https://github.com/apache/kafka/pull/15962#issuecomment-2111822272 CC: @kamalcph this is following the discussion at https://github.com/apache/kafka/pull/15885#discussion_r1598102664 -- This is an automated message from the Apache Git Service. To

[jira] [Commented] (KAFKA-16700) Kafka Streams: possible message loss on KTable-KTable FK Left Join

2024-05-15 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846544#comment-17846544 ] Karsten Stöckmann commented on KAFKA-16700: --- [~ayoubomari] Hm, according to do

Re: [PR] KAFKA-7632: Support Compression Levels (KIP-390) [kafka]

2024-05-15 Thread via GitHub
mimaison commented on code in PR #15516: URL: https://github.com/apache/kafka/pull/15516#discussion_r1601222613 ## clients/src/test/java/org/apache/kafka/common/compress/NoCompressionTest.java: ## @@ -0,0 +1,57 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

[jira] [Commented] (KAFKA-16760) alterReplicaLogDirs failed even if responded with none error

2024-05-15 Thread Igor Soarez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846552#comment-17846552 ] Igor Soarez commented on KAFKA-16760: - [~showuon] thanks for updating the branch, I'

[jira] [Comment Edited] (KAFKA-16760) alterReplicaLogDirs failed even if responded with none error

2024-05-15 Thread Igor Soarez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846552#comment-17846552 ] Igor Soarez edited comment on KAFKA-16760 at 5/15/24 8:57 AM:

Re: [PR] KAFKA-7632: Support Compression Levels (KIP-390) [kafka]

2024-05-15 Thread via GitHub
mimaison commented on code in PR #15516: URL: https://github.com/apache/kafka/pull/15516#discussion_r1601240696 ## core/src/test/scala/unit/kafka/log/LogValidatorTest.scala: ## @@ -1587,7 +1612,7 @@ class LogValidatorTest { private def createTwoBatchedRecords(magicValue: By

Re: [PR] KAFKA-16629: Add broker-related tests to ConfigCommandIntegrationTest [kafka]

2024-05-15 Thread via GitHub
chia7712 commented on PR #15840: URL: https://github.com/apache/kafka/pull/15840#issuecomment-2111958927 ``` 错误 org.apache.kafka.common.errors.InvalidConfigurationException: All sensitive broker config entries must be specified for --alter, missing entries: Set(message.max.size) 栈跟

Re: [PR] MINOR: Change sys test describe topic parsing to improve extensibility [kafka]

2024-05-15 Thread via GitHub
lucasbru merged PR #15941: URL: https://github.com/apache/kafka/pull/15941 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-16287: Implement example tests for common rebalance callback [kafka]

2024-05-15 Thread via GitHub
lucasbru merged PR #15408: URL: https://github.com/apache/kafka/pull/15408 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-16577: New consumer fails with stop within allotted timeout in consumer_test.py system test [kafka]

2024-05-15 Thread via GitHub
lucasbru merged PR #15784: URL: https://github.com/apache/kafka/pull/15784 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-7632: Support Compression Levels (KIP-390) [kafka]

2024-05-15 Thread via GitHub
mimaison commented on code in PR #15516: URL: https://github.com/apache/kafka/pull/15516#discussion_r1601283629 ## clients/src/test/java/org/apache/kafka/common/compress/GzipCompressionTest.java: ## @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-16599: LegacyConsumer should always await pending async commits on commitSync and close [kafka]

2024-05-15 Thread via GitHub
lucasbru commented on code in PR #15693: URL: https://github.com/apache/kafka/pull/15693#discussion_r1601287604 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -984,6 +984,8 @@ public void close(final Timer timer) {

Re: [PR] KAFKA-16599: LegacyConsumer should always await pending async commits on commitSync and close [kafka]

2024-05-15 Thread via GitHub
lucasbru commented on code in PR #15693: URL: https://github.com/apache/kafka/pull/15693#discussion_r1601288163 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -1033,16 +1035,22 @@ public RequestFuture commitOffsetsAsync(final

Re: [PR] KAFKA-16599: LegacyConsumer should always await pending async commits on commitSync and close [kafka]

2024-05-15 Thread via GitHub
lucasbru commented on code in PR #15693: URL: https://github.com/apache/kafka/pull/15693#discussion_r1601290415 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -1061,25 +1069,29 @@ private RequestFuture doCommitOffsetsAsync(fi

Re: [PR] KAFKA-7632: Support Compression Levels (KIP-390) [kafka]

2024-05-15 Thread via GitHub
mimaison commented on code in PR #15516: URL: https://github.com/apache/kafka/pull/15516#discussion_r1601337698 ## clients/src/test/java/org/apache/kafka/common/compress/GzipCompressionTest.java: ## @@ -0,0 +1,84 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[jira] [Commented] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-15 Thread Angelos Kaltsikis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846572#comment-17846572 ] Angelos Kaltsikis commented on KAFKA-16692: --- We faced the same issue when we t

[jira] [Comment Edited] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-15 Thread Angelos Kaltsikis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846572#comment-17846572 ] Angelos Kaltsikis edited comment on KAFKA-16692 at 5/15/24 10:03 AM: -

Re: [PR] KAFKA-16712: Fix race in TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest [kafka]

2024-05-15 Thread via GitHub
chia7712 commented on code in PR #15962: URL: https://github.com/apache/kafka/pull/15962#discussion_r1601282218 ## storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerHarness.java: ## @@ -119,9 +120,6 @@ public void onPart

[jira] [Commented] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-15 Thread Angelos Kaltsikis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846577#comment-17846577 ] Angelos Kaltsikis commented on KAFKA-16692: --- It seems that adding the `transac

Re: [PR] KAFKA-16711: make sure update highestOffsetInRemoteStorage after log dir change [kafka]

2024-05-15 Thread via GitHub
kamalcph commented on PR #15947: URL: https://github.com/apache/kafka/pull/15947#issuecomment-2112196443 > In the next run of RLMTask, the log will be the one in target dir (dirB), and the log#highestOffsetInRemoteStorage (dirB) will be the default value (-1), which will block the log clean

Re: [PR] KAFKA-16712: Fix race in TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest [kafka]

2024-05-15 Thread via GitHub
gaurav-narula commented on code in PR #15962: URL: https://github.com/apache/kafka/pull/15962#discussion_r1601428859 ## storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest.java: ## @@ -108,7 +114,2

Re: [PR] KAFKA-15974: Enforce that event processing respects user-provided timeout [kafka]

2024-05-15 Thread via GitHub
cadonna commented on code in PR #15640: URL: https://github.com/apache/kafka/pull/15640#discussion_r1601414540 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -150,6 +151,7 @@ public class AsyncKafkaConsumerTest { priv

[jira] [Commented] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-15 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846590#comment-17846590 ] Justine Olshan commented on KAFKA-16692: Hi [~akaltsikis]. Yes the bug is due to

[jira] [Commented] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-15 Thread Angelos Kaltsikis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846592#comment-17846592 ] Angelos Kaltsikis commented on KAFKA-16692: --- Shall we explicitly state this on

[jira] [Commented] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-15 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846591#comment-17846591 ] Justine Olshan commented on KAFKA-16692: The delay in the fix is coming from the

[jira] [Comment Edited] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-15 Thread Angelos Kaltsikis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846592#comment-17846592 ] Angelos Kaltsikis edited comment on KAFKA-16692 at 5/15/24 11:36 AM: -

Re: [PR] KAFKA-16223: Replace EasyMock/PowerMock with Mockito for KafkaConfigBackingStoreTest [kafka]

2024-05-15 Thread via GitHub
FrankYang0529 commented on code in PR #15933: URL: https://github.com/apache/kafka/pull/15933#discussion_r1601493611 ## connect/runtime/src/test/java/org/apache/kafka/connect/storage/KafkaConfigBackingStoreTest.java: ## @@ -960,6 +666,14 @@ private void expectConvertWriteRead(fi

Re: [PR] KAFKA-16709: abortAndPauseCleaning only when future log is not existed [kafka]

2024-05-15 Thread via GitHub
showuon commented on code in PR #15951: URL: https://github.com/apache/kafka/pull/15951#discussion_r1601510877 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2114,19 +2114,16 @@ class ReplicaManager(val config: KafkaConfig, partition.log.foreach { _ =>

Re: [PR] KAFKA-16709: abortAndPauseCleaning only when future log is not existed [kafka]

2024-05-15 Thread via GitHub
showuon commented on code in PR #15951: URL: https://github.com/apache/kafka/pull/15951#discussion_r1601512176 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2114,19 +2114,16 @@ class ReplicaManager(val config: KafkaConfig, partition.log.foreach { _ =>

Re: [PR] KAFKA-16709: abortAndPauseCleaning only when future log is not existed [kafka]

2024-05-15 Thread via GitHub
showuon commented on code in PR #15951: URL: https://github.com/apache/kafka/pull/15951#discussion_r1601526314 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -314,6 +314,77 @@ class ReplicaManagerTest { } } + @ParameterizedTest(name = "testMa

[jira] [Commented] (KAFKA-16768) SocketServer leaks accepted SocketChannel instances due to race condition

2024-05-15 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846607#comment-17846607 ] Muralidhar Basani commented on KAFKA-16768: --- To fix this, when Processor#close

Re: [PR] KAFKA-16711: make sure update highestOffsetInRemoteStorage after log dir change [kafka]

2024-05-15 Thread via GitHub
showuon commented on PR #15947: URL: https://github.com/apache/kafka/pull/15947#issuecomment-2112378322 @kamalcph , > If there are some segments to upload from the dirB, will the log#highestOffsetInRemoteStorage updated? Yes, it will! That means, the inconsistent state will be fixe

Re: [PR] KAFKA-14588 [3/N] ConfigCommandTest rewritten in java [kafka]

2024-05-15 Thread via GitHub
nizhikov commented on PR #15930: URL: https://github.com/apache/kafka/pull/15930#issuecomment-2112395461 CI looks Ok for me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] KAFKA-7342: Migrate tests in remaining packages in o.a.k.streams [kafka]

2024-05-15 Thread via GitHub
cadonna opened a new pull request, #15963: URL: https://github.com/apache/kafka/pull/15963 Migrates tests in the following packages (excluding subpackages) to JUnit 5: - org.apache.kafka.streams.internals - org.apache.kafka.streams.kstream - org.apache.kafka.streams.processor - or

Re: [PR] KAFKA-16598 Mirgrate `ResetConsumerGroupOffsetTest` to new test infra [kafka]

2024-05-15 Thread via GitHub
m1a2st commented on PR #15779: URL: https://github.com/apache/kafka/pull/15779#issuecomment-2112434755 @chia7712 , Thanks for your review, these test passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] KAFKA-16709: abortAndPauseCleaning only when future log is not existed [kafka]

2024-05-15 Thread via GitHub
chia7712 commented on code in PR #15951: URL: https://github.com/apache/kafka/pull/15951#discussion_r1601591079 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -314,6 +314,77 @@ class ReplicaManagerTest { } } + @ParameterizedTest(name = "testM

Re: [PR] KAFKA-16629: Add broker-related tests to ConfigCommandIntegrationTest [kafka]

2024-05-15 Thread via GitHub
m1a2st commented on code in PR #15840: URL: https://github.com/apache/kafka/pull/15840#discussion_r1598586998 ## core/src/test/java/kafka/admin/ConfigCommandIntegrationTest.java: ## @@ -63,13 +73,26 @@ @SuppressWarnings("deprecation") // Added for Scala 2.12 compatibility for

[jira] [Commented] (KAFKA-15242) FixedKeyProcessor testing is unusable

2024-05-15 Thread Matej Sprysl (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846632#comment-17846632 ] Matej Sprysl commented on KAFKA-15242: -- I would also like this fixed. Currently, th

Re: [PR] KAFKA-16712: Fix race in TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest [kafka]

2024-05-15 Thread via GitHub
chia7712 commented on code in PR #15962: URL: https://github.com/apache/kafka/pull/15962#discussion_r1601713170 ## storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest.java: ## @@ -108,7 +114,23 @@

Re: [PR] KAFKA-16712: Fix race in TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest [kafka]

2024-05-15 Thread via GitHub
gaurav-narula commented on code in PR #15962: URL: https://github.com/apache/kafka/pull/15962#discussion_r1601731929 ## storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerHarness.java: ## @@ -119,9 +120,6 @@ public void o

Re: [PR] Kafka-16668: Add tags support in ClusterTestExtension [kafka]

2024-05-15 Thread via GitHub
johnnychhsu commented on code in PR #15861: URL: https://github.com/apache/kafka/pull/15861#discussion_r1601745613 ## core/src/test/java/kafka/test/ClusterConfig.java: ## @@ -153,13 +153,13 @@ public Map> perServerOverrideProperties() { return perServerProperties;

Re: [PR] Kafka-16668: Add tags support in ClusterTestExtension [kafka]

2024-05-15 Thread via GitHub
johnnychhsu commented on code in PR #15861: URL: https://github.com/apache/kafka/pull/15861#discussion_r1601750727 ## core/src/test/java/kafka/test/ClusterTestExtensionsTest.java: ## @@ -66,8 +67,8 @@ static void generate1(ClusterGenerator clusterGenerator) { serverProp

Re: [PR] Kafka-16668: Add tags support in ClusterTestExtension [kafka]

2024-05-15 Thread via GitHub
johnnychhsu commented on code in PR #15861: URL: https://github.com/apache/kafka/pull/15861#discussion_r1601753901 ## core/src/test/java/kafka/test/ClusterTestExtensionsTest.java: ## @@ -84,37 +85,47 @@ public void testClusterTest(ClusterInstance clusterInstance) { public

Re: [PR] Kafka-16668: Add tags support in ClusterTestExtension [kafka]

2024-05-15 Thread via GitHub
chia7712 commented on code in PR #15861: URL: https://github.com/apache/kafka/pull/15861#discussion_r1601754643 ## core/src/test/java/kafka/test/ClusterTestExtensionsTest.java: ## @@ -66,8 +67,8 @@ static void generate1(ClusterGenerator clusterGenerator) { serverPropert

Re: [PR] Kafka-16668: Add tags support in ClusterTestExtension [kafka]

2024-05-15 Thread via GitHub
johnnychhsu commented on code in PR #15861: URL: https://github.com/apache/kafka/pull/15861#discussion_r1601756235 ## core/src/test/java/kafka/test/ClusterTestExtensionsTest.java: ## @@ -84,37 +85,47 @@ public void testClusterTest(ClusterInstance clusterInstance) { public

[jira] [Commented] (KAFKA-16333) Removed Deprecated methods KTable#join

2024-05-15 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846658#comment-17846658 ] Muralidhar Basani commented on KAFKA-16333: --- I think very soon 4.0.0 might be

[jira] [Commented] (KAFKA-16329) Remove Deprecated Task/ThreadMetadata classes and related methods

2024-05-15 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846660#comment-17846660 ] Muralidhar Basani commented on KAFKA-16329: --- I think very soon 4.0.0 might be

[jira] [Commented] (KAFKA-16330) Remove Deprecated methods/variables from TaskId

2024-05-15 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846661#comment-17846661 ] Muralidhar Basani commented on KAFKA-16330: --- I think very soon 4.0.0 might be

[jira] [Commented] (KAFKA-16603) Data loss when kafka connect sending data to Kafka

2024-05-15 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846662#comment-17846662 ] Chris Egerton commented on KAFKA-16603: --- [~dasarianil] In most cases, a single wor

Re: [PR] KAFKA-15905 Restarts of MirrorCheckpointTask should not permanently i… [kafka]

2024-05-15 Thread via GitHub
edoardocomar commented on PR #15910: URL: https://github.com/apache/kafka/pull/15910#issuecomment-2112731563 Hi @gharris1727 commit e33edd2a72 hopefully address most of your comments. Thanks for the quick feedback. We also noticed that the loading of the checkpoints must complete befo

[jira] [Created] (KAFKA-16770) Coalesce records into bigger batches

2024-05-15 Thread David Jacot (Jira)
David Jacot created KAFKA-16770: --- Summary: Coalesce records into bigger batches Key: KAFKA-16770 URL: https://issues.apache.org/jira/browse/KAFKA-16770 Project: Kafka Issue Type: Sub-task

[PR] KAFKA-16770; [1/N] Coalesce records into bigger batches [kafka]

2024-05-15 Thread via GitHub
dajac opened a new pull request, #15964: URL: https://github.com/apache/kafka/pull/15964 We have discovered during large scale performance tests that the current write path of the new coordinator does not scale well. The issue is that each write operation writes synchronously from the coord

[PR] KAFKA-16675: Refactored and new rebalance callbacks integration tests [kafka]

2024-05-15 Thread via GitHub
lianetm opened a new pull request, #15965: URL: https://github.com/apache/kafka/pull/15965 - Move existing rebalance callback + consumer.position test to the PlaintextConsumerCallbackTest file (refactored to reuse the new helper funcs available) - Add new integration tests for callbacks

Re: [PR] KAFKA-16675: Refactored and new rebalance callbacks integration tests [kafka]

2024-05-15 Thread via GitHub
lianetm commented on PR #15965: URL: https://github.com/apache/kafka/pull/15965#issuecomment-2112758137 Hey @lucasbru, extending on top of your initial callbacks test file to get more coverage (very nice helper funcs btw). Could you take a look when you have a chance? Thanks! -- This is

Re: [PR] KAFKA-16675: Refactored and new rebalance callbacks integration tests [kafka]

2024-05-15 Thread via GitHub
lianetm commented on code in PR #15965: URL: https://github.com/apache/kafka/pull/15965#discussion_r1601804725 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerCallbackTest.scala: ## @@ -84,29 +84,87 @@ class PlaintextConsumerCallbackTest extends AbstractConsumerTe

[PR] KAFKA-16684: Remove cache in responseData [kafka]

2024-05-15 Thread via GitHub
johnnychhsu opened a new pull request, #15966: URL: https://github.com/apache/kafka/pull/15966 ## Context The response data should change accordingly to the input, however with the current design, it will not change even if the input changes. We should remove this cache logic to avoid re

[jira] [Created] (KAFKA-16771) First log directory printed twice when formatting storage

2024-05-15 Thread Mickael Maison (Jira)
Mickael Maison created KAFKA-16771: -- Summary: First log directory printed twice when formatting storage Key: KAFKA-16771 URL: https://issues.apache.org/jira/browse/KAFKA-16771 Project: Kafka

[jira] [Commented] (KAFKA-16448) Add Kafka Streams exception handler for exceptions occuring during processing (KIP-1033)

2024-05-15 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846675#comment-17846675 ] Muralidhar Basani commented on KAFKA-16448: --- [~Dabz] can I try looking into th

Re: [PR] KAFKA-16606 Gate JBOD configuration on 3.7-IV2 [kafka]

2024-05-15 Thread via GitHub
mimaison commented on PR #15834: URL: https://github.com/apache/kafka/pull/15834#issuecomment-2112826872 Thanks for the PR. If I set `inter.broker.protocol.version=3.6-IV2` and multiple log directories in my broker configuration, it now prevents me from formatting the storage: ```

[PR] MINOR: Remove unused method in ToolsUtils [kafka]

2024-05-15 Thread via GitHub
mimaison opened a new pull request, #15967: URL: https://github.com/apache/kafka/pull/15967 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgr

[jira] [Commented] (KAFKA-16768) SocketServer leaks accepted SocketChannel instances due to race condition

2024-05-15 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846684#comment-17846684 ] Greg Harris commented on KAFKA-16768: - [~muralibasani] Do you mean call Acceptor#clo

Re: [PR] KAFKA-16526; Quorum state data version 1 [kafka]

2024-05-15 Thread via GitHub
jsancio commented on PR #15859: URL: https://github.com/apache/kafka/pull/15859#issuecomment-2112867941 > @jsancio : It seems that the java 8 tests were aborted in the latest run? I see that. I merged trunk and push to kick off a new build. -- This is an automated message from the A

[jira] [Created] (KAFKA-16772) Introduce a type for kraft.version

2024-05-15 Thread Jira
José Armando García Sancio created KAFKA-16772: -- Summary: Introduce a type for kraft.version Key: KAFKA-16772 URL: https://issues.apache.org/jira/browse/KAFKA-16772 Project: Kafka

[PR] Kafka-10551: Add topic id support to produce request and response [kafka]

2024-05-15 Thread via GitHub
OmniaGM opened a new pull request, #15968: URL: https://github.com/apache/kafka/pull/15968 - Add support topicId in `ProduceRequest`/`ProduceResponse`. Topic name and Topic Id will become optional following the footstep of `FetchRequest`/`FetchResponse` - ReplicaManager still look for `H

Re: [PR] Refactor SSL/SASL admin integration tests to not use a custom authorizer [kafka]

2024-05-15 Thread via GitHub
mimaison commented on code in PR #15377: URL: https://github.com/apache/kafka/pull/15377#discussion_r1601900093 ## core/src/test/scala/unit/kafka/utils/TestUtils.scala: ## @@ -1776,6 +1776,33 @@ object TestUtils extends Logging { 45000) } + + def waitAndVerifyAcl(

Re: [PR] KAFKA-16759: Handle telemetry push response while terminating [kafka]

2024-05-15 Thread via GitHub
omkreddy merged PR #15957: URL: https://github.com/apache/kafka/pull/15957 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] MINOR: migrate ListConsumerGroupTest to use ClusterTestExtensions [kafka]

2024-05-15 Thread via GitHub
FrankYang0529 commented on code in PR #15821: URL: https://github.com/apache/kafka/pull/15821#discussion_r1601924534 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/ConsumerGroupCommandTestUtils.java: ## @@ -93,6 +95,27 @@ static AutoCloseable buildConsumers(int n

[jira] [Commented] (KAFKA-16760) alterReplicaLogDirs failed even if responded with none error

2024-05-15 Thread Igor Soarez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846705#comment-17846705 ] Igor Soarez commented on KAFKA-16760: - [~showuon] I a few hours looking into this bu

[PR] MINOR: Remove usage of Stream API in CoordinatorRecordHelpers [kafka]

2024-05-15 Thread via GitHub
dajac opened a new pull request, #15969: URL: https://github.com/apache/kafka/pull/15969 This patch removes the usage of the Stream API in CoordinatorRecordHelpers. I saw it in a couple of profiles so it is better to remove it. ### Committer Checklist (excluded from commit message)

Re: [PR] KAFKA-16762: SyncGroup API for upgrading ConsumerGroup [kafka]

2024-05-15 Thread via GitHub
dongnuo123 commented on code in PR #15954: URL: https://github.com/apache/kafka/pull/15954#discussion_r1601951216 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -3898,6 +3994,65 @@ public CoordinatorResult classicGroupSync(

Re: [PR] KAFKA-16711: make sure update highestOffsetInRemoteStorage after log dir change [kafka]

2024-05-15 Thread via GitHub
kamalcph commented on code in PR #15947: URL: https://github.com/apache/kafka/pull/15947#discussion_r1601958631 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -771,12 +772,127 @@ void testRemoteLogManagerTasksAvgIdlePercentAndMetadataCountMetrics() thro

Re: [PR] KAFKA-16764: New consumer should throw InvalidTopicException on poll … [kafka]

2024-05-15 Thread via GitHub
lianetm commented on code in PR #15961: URL: https://github.com/apache/kafka/pull/15961#discussion_r1601976218 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1607,6 +1607,7 @@ private Fetch pollForFetches(Timer timer) {

Re: [PR] KAFKA-16629: Add broker-related tests to ConfigCommandIntegrationTest [kafka]

2024-05-15 Thread via GitHub
chia7712 commented on code in PR #15840: URL: https://github.com/apache/kafka/pull/15840#discussion_r1601976384 ## core/src/test/java/kafka/admin/ConfigCommandIntegrationTest.java: ## @@ -65,219 +74,475 @@ @ExtendWith(value = ClusterTestExtensions.class) @Tag("integration") p

Re: [PR] KAFKA-16764: New consumer should throw InvalidTopicException on poll … [kafka]

2024-05-15 Thread via GitHub
lianetm commented on code in PR #15961: URL: https://github.com/apache/kafka/pull/15961#discussion_r1601985896 ## clients/src/test/java/org/apache/kafka/clients/consumer/KafkaConsumerTest.java: ## @@ -2974,7 +2974,7 @@ public void testSubscriptionOnInvalidTopic(GroupProtocol gr

Re: [PR] KAFKA-16598 Mirgrate `ResetConsumerGroupOffsetTest` to new test infra [kafka]

2024-05-15 Thread via GitHub
chia7712 commented on PR #15779: URL: https://github.com/apache/kafka/pull/15779#issuecomment-2113049965 @m1a2st could you please rebase code to trigger QA again? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-16223: Replace EasyMock/PowerMock with Mockito for KafkaConfigBackingStoreTest [kafka]

2024-05-15 Thread via GitHub
chia7712 commented on PR #15933: URL: https://github.com/apache/kafka/pull/15933#issuecomment-2113051164 @FrankYang0529 Could you please rebase code to trigger QA ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] KAFKA-16764: New consumer should throw InvalidTopicException on poll … [kafka]

2024-05-15 Thread via GitHub
lianetm commented on PR #15961: URL: https://github.com/apache/kafka/pull/15961#issuecomment-2113054675 Hey @appchemist , thanks a lot for taking this one! I left some comments, mainly concerns about the approach, this is definitely a tricky bit. Thanks! -- This is an automated message fr

Re: [PR] KAFKA-7632: Support Compression Levels (KIP-390) [kafka]

2024-05-15 Thread via GitHub
junrao commented on code in PR #15516: URL: https://github.com/apache/kafka/pull/15516#discussion_r1602012880 ## server-common/src/test/java/org/apache/kafka/server/record/BrokerCompressionTypeTest.java: ## @@ -16,23 +16,38 @@ */ package org.apache.kafka.server.record; +imp

Re: [PR] KAFKA-16762: SyncGroup API for upgrading ConsumerGroup [kafka]

2024-05-15 Thread via GitHub
dongnuo123 commented on code in PR #15954: URL: https://github.com/apache/kafka/pull/15954#discussion_r1602027889 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -3898,6 +3994,65 @@ public CoordinatorResult classicGroupSync(

Re: [PR] KAFKA-7342: Migrate tests in remaining packages in o.a.k.streams [kafka]

2024-05-15 Thread via GitHub
cadonna commented on PR #15963: URL: https://github.com/apache/kafka/pull/15963#issuecomment-2113159660 > LGTM > > I guess you don't want to create a huge PR to migrate all tests of streams at once? Yeah, that is the idea. -- This is an automated message from the Apache Git

[jira] [Created] (KAFKA-16773) Fix flaky QuorumControllerTest#testDelayedConfigurationOperations

2024-05-15 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16773: -- Summary: Fix flaky QuorumControllerTest#testDelayedConfigurationOperations Key: KAFKA-16773 URL: https://issues.apache.org/jira/browse/KAFKA-16773 Project: Kafka

[jira] [Created] (KAFKA-16774) fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEnabled

2024-05-15 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16774: -- Summary: fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEnabled Key: KAFKA-16774 URL: https://issues.apache.org/jira/browse/KAFKA-16774 Project

[jira] [Created] (KAFKA-16775) Fix flaky PlaintextAdminIntegrationTest#testCreateExistingTopicsThrowTopicExistsException

2024-05-15 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16775: -- Summary: Fix flaky PlaintextAdminIntegrationTest#testCreateExistingTopicsThrowTopicExistsException Key: KAFKA-16775 URL: https://issues.apache.org/jira/browse/KAFKA-16775

Re: [PR] MINOR: revisit LogValidatorTest#checkRecompression [kafka]

2024-05-15 Thread via GitHub
chia7712 commented on PR #15948: URL: https://github.com/apache/kafka/pull/15948#issuecomment-2113171124 |test|jira| |-|-| |testDelayedConfigurationOperations|https://issues.apache.org/jira/browse/KAFKA-16773| |testOffsetTranslationBehindReplicationFlow|https://issues.apach

Re: [PR] MINOR: revisit LogValidatorTest#checkRecompression [kafka]

2024-05-15 Thread via GitHub
chia7712 merged PR #15948: URL: https://github.com/apache/kafka/pull/15948 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-7342: Migrate tests in remaining packages in o.a.k.streams [kafka]

2024-05-15 Thread via GitHub
cadonna merged PR #15963: URL: https://github.com/apache/kafka/pull/15963 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] MINOR: fix timeouts of EosIntegrationTest [kafka]

2024-05-15 Thread via GitHub
gharris1727 commented on PR #15811: URL: https://github.com/apache/kafka/pull/15811#issuecomment-2113246873 @mjsax @chia7712 Thanks so much for this fix, the difference is dramatic! The median runtime for this test in CI went from 90 minutes to 25 minutes. -- This is an automated message

Re: [PR] MINOR: Fix rate metric spikes [kafka]

2024-05-15 Thread via GitHub
junrao commented on code in PR #15889: URL: https://github.com/apache/kafka/pull/15889#discussion_r1602125187 ## clients/src/main/java/org/apache/kafka/common/metrics/stats/Rate.java: ## @@ -63,33 +63,38 @@ public void record(MetricConfig config, double value, long timeMs) {

[jira] [Created] (KAFKA-16776) Builds flaky-failing with leaked client-metrics-reaper thread

2024-05-15 Thread Greg Harris (Jira)
Greg Harris created KAFKA-16776: --- Summary: Builds flaky-failing with leaked client-metrics-reaper thread Key: KAFKA-16776 URL: https://issues.apache.org/jira/browse/KAFKA-16776 Project: Kafka

[jira] [Updated] (KAFKA-16776) Builds flaky-failing with leaked client-metrics-reaper thread

2024-05-15 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-16776: Description: I observed the assertion added by KAFKA-16477 failing in my build: [https://ci-build

[jira] [Assigned] (KAFKA-16622) Mirromaker2 first Checkpoint not emitted until consumer group fully catches up once

2024-05-15 Thread Edoardo Comar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edoardo Comar reassigned KAFKA-16622: - Assignee: Edoardo Comar > Mirromaker2 first Checkpoint not emitted until consumer group

Re: [PR] MINOR: Fix rate metric spikes [kafka]

2024-05-15 Thread via GitHub
emitskevich-blp commented on code in PR #15889: URL: https://github.com/apache/kafka/pull/15889#discussion_r1602153395 ## clients/src/main/java/org/apache/kafka/common/metrics/stats/Rate.java: ## @@ -63,33 +63,38 @@ public void record(MetricConfig config, double value, long tim

Re: [PR] KAFKA-10199: Handle assignment with new remove operation in state updater [kafka]

2024-05-15 Thread via GitHub
chia7712 commented on code in PR #15882: URL: https://github.com/apache/kafka/pull/15882#discussion_r1602161837 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -1750,6 +1761,7 @@ Map allTasks() { if (stateUpdater != null) {

  1   2   >