[jira] [Resolved] (KAFKA-16615) JoinGroup API for upgrading ConsumerGroup

2024-05-07 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16615. - Fix Version/s: 3.8.0 Reviewer: David Jacot Assignee: Dongnuo Lyu Resolut

[jira] [Commented] (KAFKA-16677) Replace ClusterType#ALL and ClusterType#DEFAULT by Array

2024-05-07 Thread PoAn Yang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844163#comment-17844163 ] PoAn Yang commented on KAFKA-16677: --- Hi [~chia7712], I'm interested in this feature. I

[jira] [Created] (KAFKA-16679) Merge `DeleteRecordsCommandUnitTest` into `DeleteRecordsCommandTest`, `FeatureCommandUnitTest` into

2024-05-07 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16679: -- Summary: Merge `DeleteRecordsCommandUnitTest` into `DeleteRecordsCommandTest`, `FeatureCommandUnitTest` into Key: KAFKA-16679 URL: https://issues.apache.org/jira/browse/KAFKA

[jira] [Assigned] (KAFKA-16677) Replace ClusterType#ALL and ClusterType#DEFAULT by Array

2024-05-07 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reassigned KAFKA-16677: -- Assignee: PoAn Yang (was: Chia-Ping Tsai) > Replace ClusterType#ALL and ClusterType#

Re: [PR] KAFKA-10199: Remove lost tasks in state updater with new remove [kafka]

2024-05-07 Thread via GitHub
lucasbru commented on code in PR #15870: URL: https://github.com/apache/kafka/pull/15870#discussion_r1591960681 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -602,6 +608,47 @@ private void removeUnusedTaskFromStateUpdater(final Tas

Re: [PR] MINOR: Remove dev_version parameter from streams tests [kafka]

2024-05-07 Thread via GitHub
lucasbru commented on PR #15874: URL: https://github.com/apache/kafka/pull/15874#issuecomment-2097704867 @mjsax It does change, because we need to keep a list of tests (with parameters) that do not work with ARM. The `from_version` parameter does not change, because only old versions are af

[jira] [Created] (KAFKA-16680) Make ClusterTestExtensions support SASL

2024-05-07 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16680: -- Summary: Make ClusterTestExtensions support SASL Key: KAFKA-16680 URL: https://issues.apache.org/jira/browse/KAFKA-16680 Project: Kafka Issue Type: New F

[jira] [Updated] (KAFKA-16589) Consider removing `ClusterInstance#createAdminClient` since callers are not sure whether they need to call close

2024-05-07 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-16589: --- Parent: KAFKA-16680 Issue Type: Sub-task (was: Improvement) > Consider removing `Cl

Re: [PR] KAFKA-16665: Allow to initialize newly assigned partition's positions without allowing fetching while callback runs [kafka]

2024-05-07 Thread via GitHub
lucasbru merged PR #15856: URL: https://github.com/apache/kafka/pull/15856 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Created] (KAFKA-16681) Rewrite MiniKDC by Java

2024-05-07 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16681: -- Summary: Rewrite MiniKDC by Java Key: KAFKA-16681 URL: https://issues.apache.org/jira/browse/KAFKA-16681 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-16682) Rewrite JassTestUtils by Java

2024-05-07 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16682: -- Summary: Rewrite JassTestUtils by Java Key: KAFKA-16682 URL: https://issues.apache.org/jira/browse/KAFKA-16682 Project: Kafka Issue Type: Sub-task

[jira] [Updated] (KAFKA-16680) Make ClusterTestExtensions support SASL

2024-05-07 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-16680: --- Description: This is a umbrella issue. In order to migrate more tests to new test infra, we

[jira] [Created] (KAFKA-16683) Extract security-related helpers from scala.TestUtils to java class

2024-05-07 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16683: -- Summary: Extract security-related helpers from scala.TestUtils to java class Key: KAFKA-16683 URL: https://issues.apache.org/jira/browse/KAFKA-16683 Project: Kafk

[PR] KAFKA-16676: Add missing RPCs to security.html [kafka]

2024-05-07 Thread via GitHub
AndrewJSchofield opened a new pull request, #15878: URL: https://github.com/apache/kafka/pull/15878 KIP-714 and KIP-1000 introduced 3 new RPCs. These new RPCs were not added to docs/security.html to document the authorization checks required to perform the requests. While I was addin

Re: [PR] Allowing WriteTxnMarkers API to run with AlterCluster permissions [kafka]

2024-05-07 Thread via GitHub
sidyag commented on code in PR #15837: URL: https://github.com/apache/kafka/pull/15837#discussion_r1592106849 ## core/src/test/scala/unit/kafka/server/KafkaApisTest.scala: ## @@ -2822,6 +2822,31 @@ class KafkaApisTest extends Logging { () => kafkaApis.handleWriteTxnMarker

Re: [PR] Allowing WriteTxnMarkers API to run with AlterCluster permissions [kafka]

2024-05-07 Thread via GitHub
sidyag commented on code in PR #15837: URL: https://github.com/apache/kafka/pull/15837#discussion_r1592106849 ## core/src/test/scala/unit/kafka/server/KafkaApisTest.scala: ## @@ -2822,6 +2822,31 @@ class KafkaApisTest extends Logging { () => kafkaApis.handleWriteTxnMarker

[jira] [Commented] (KAFKA-16679) Merge `DeleteRecordsCommandUnitTest` into `DeleteRecordsCommandTest`, `FeatureCommandUnitTest` into

2024-05-07 Thread Cheng-Kai, Zhang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844222#comment-17844222 ] Cheng-Kai, Zhang commented on KAFKA-16679: -- Hi [~chia7712] , could I work on th

Re: [PR] KAFKA-16480: Bump ListOffsets version, IBP version and mark last version of ListOffsets as unstable [kafka]

2024-05-07 Thread via GitHub
clolov commented on code in PR #15673: URL: https://github.com/apache/kafka/pull/15673#discussion_r1592114443 ## core/src/test/scala/unit/kafka/server/ListOffsetsRequestTest.scala: ## @@ -219,7 +219,7 @@ class ListOffsetsRequestTest extends BaseRequestTest { TestUtils.gener

Re: [PR] KAFKA-16480: Bump ListOffsets version, IBP version and mark last version of ListOffsets as unstable [kafka]

2024-05-07 Thread via GitHub
clolov commented on code in PR #15673: URL: https://github.com/apache/kafka/pull/15673#discussion_r1592116050 ## clients/src/main/java/org/apache/kafka/common/requests/ListOffsetsRequest.java: ## @@ -70,7 +70,7 @@ else if (isolationLevel == IsolationLevel.READ_COMMITTED)

Re: [PR] KAFKA-16452: Don't throw OOORE when converting the offset to metadata [kafka]

2024-05-07 Thread via GitHub
kamalcph commented on PR #15825: URL: https://github.com/apache/kafka/pull/15825#issuecomment-2097869168 @junrao @satishd @chia7712 @showuon Updated the test plan in the summary. Verified that the patch fixes the issue by running the trunk and patched build. With the fix, the high-wa

[jira] [Assigned] (KAFKA-16679) Merge `DeleteRecordsCommandUnitTest` into `DeleteRecordsCommandTest`, `FeatureCommandUnitTest` into

2024-05-07 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reassigned KAFKA-16679: -- Assignee: Cheng-Kai, Zhang (was: Chia-Ping Tsai) > Merge `DeleteRecordsCommandUnitTe

Re: [PR] KAFKA-16480: Bump ListOffsets version, IBP version and mark last version of ListOffsets as unstable [kafka]

2024-05-07 Thread via GitHub
clolov commented on PR #15673: URL: https://github.com/apache/kafka/pull/15673#issuecomment-2097975706 Heya @junrao! Let me know your thoughts on my responses. I have futher updated the below + rebased. I am waiting on the tests to finish running and if they uncover something I will aim to

Re: [PR] KAFKA-14588 [2/N] ConfigCommandTest rewritten in java [kafka]

2024-05-07 Thread via GitHub
nizhikov commented on code in PR #15873: URL: https://github.com/apache/kafka/pull/15873#discussion_r1592246946 ## core/src/test/java/kafka/admin/ConfigCommandUnitTest.java: ## @@ -410,6 +448,430 @@ public void testOptionEntityTypeNames() { doTestOptionEntityTypeNames(f

Re: [PR] KAFKA-14588 [2/N] ConfigCommandTest rewritten in java [kafka]

2024-05-07 Thread via GitHub
nizhikov commented on code in PR #15873: URL: https://github.com/apache/kafka/pull/15873#discussion_r1592249836 ## core/src/test/java/kafka/admin/ConfigCommandUnitTest.java: ## @@ -418,4 +880,77 @@ public static String[] toArray(String... first) { public static String[] toA

Re: [PR] KAFKA-10199: Revoke tasks from state updater with new remove [kafka]

2024-05-07 Thread via GitHub
lucasbru commented on code in PR #15871: URL: https://github.com/apache/kafka/pull/15871#discussion_r1592261128 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -623,6 +623,21 @@ private void addToTasksToClose(final Map> futures, Rev

[jira] [Commented] (KAFKA-12385) Remove FetchResponse#responseData

2024-05-07 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844248#comment-17844248 ] Chia-Ping Tsai commented on KAFKA-12385: It seems to me `FetchResponse#responseD

Re: [PR] KAFKA-14588 [2/N] ConfigCommandTest rewritten in java [kafka]

2024-05-07 Thread via GitHub
nizhikov commented on code in PR #15873: URL: https://github.com/apache/kafka/pull/15873#discussion_r1592249836 ## core/src/test/java/kafka/admin/ConfigCommandUnitTest.java: ## @@ -418,4 +880,77 @@ public static String[] toArray(String... first) { public static String[] toA

[jira] [Created] (KAFKA-16684) FetchResponse#responseData could return incorrect data

2024-05-07 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16684: -- Summary: FetchResponse#responseData could return incorrect data Key: KAFKA-16684 URL: https://issues.apache.org/jira/browse/KAFKA-16684 Project: Kafka Is

[jira] [Commented] (KAFKA-16684) FetchResponse#responseData could return incorrect data

2024-05-07 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844251#comment-17844251 ] Chia-Ping Tsai commented on KAFKA-16684: After [https://github.com/apache/kafka

Re: [PR] KAFKA-10199: Revoke tasks from state updater with new remove [kafka]

2024-05-07 Thread via GitHub
cadonna commented on code in PR #15871: URL: https://github.com/apache/kafka/pull/15871#discussion_r1592297149 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -623,6 +623,21 @@ private void addToTasksToClose(final Map> futures, Revi

Re: [PR] KAFKA-16399: Add JBOD support in tiered storage [kafka]

2024-05-07 Thread via GitHub
showuon commented on code in PR #15690: URL: https://github.com/apache/kafka/pull/15690#discussion_r1592297330 ## storage/src/test/java/org/apache/kafka/tiered/storage/utils/TieredStorageTestUtils.java: ## @@ -55,7 +57,7 @@ public class TieredStorageTestUtils { // Log cle

Re: [PR] KAFKA-14588 [2/N] ConfigCommandTest rewritten in java [kafka]

2024-05-07 Thread via GitHub
nizhikov commented on code in PR #15873: URL: https://github.com/apache/kafka/pull/15873#discussion_r1592313103 ## core/src/test/java/kafka/admin/ConfigCommandUnitTest.java: ## @@ -410,6 +448,430 @@ public void testOptionEntityTypeNames() { doTestOptionEntityTypeNames(f

[PR] KAFKA-16678: Remove variable "unimplementedquorum" [kafka]

2024-05-07 Thread via GitHub
frankvicky opened a new pull request, #15879: URL: https://github.com/apache/kafka/pull/15879 Remove variable "unimplementedquorum" from EndToEndAuthorizationTest.scala and SaslEndToEndAuthorizationTest.scala - [ ] Verify design and implementation - [ ] Verify test coverage and CI

Re: [PR] a slight change. [kafka]

2024-05-07 Thread via GitHub
gongxuanzhang commented on PR #15812: URL: https://github.com/apache/kafka/pull/15812#issuecomment-2098224856 > Hi @gongxuanzhang and thank you for the contribution! > > This is actually an issue in a lot of places, I checked with this checkstyle.xml rule: > > ``` >

[jira] [Commented] (KAFKA-16682) Rewrite JassTestUtils by Java

2024-05-07 Thread TengYao Chi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844261#comment-17844261 ] TengYao Chi commented on KAFKA-16682: - I am able to handle this issue. > Rewrite Ja

[jira] [Assigned] (KAFKA-16682) Rewrite JassTestUtils by Java

2024-05-07 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reassigned KAFKA-16682: -- Assignee: TengYao Chi (was: Chia-Ping Tsai) > Rewrite JassTestUtils by Java > --

Re: [PR] KAFKA-16394: Fix null propagation in foreign key join result [kafka]

2024-05-07 Thread via GitHub
AyoubOm commented on code in PR #15607: URL: https://github.com/apache/kafka/pull/15607#discussion_r1592386867 ## streams/src/test/java/org/apache/kafka/streams/integration/KTableKTableForeignKeyJoinIntegrationTest.java: ## @@ -244,21 +248,21 @@ public void doJoinFromLeftThenDel

Re: [PR] KAFKA-16394: Fix null propagation in foreign key join result [kafka]

2024-05-07 Thread via GitHub
AyoubOm commented on code in PR #15607: URL: https://github.com/apache/kafka/pull/15607#discussion_r1592386867 ## streams/src/test/java/org/apache/kafka/streams/integration/KTableKTableForeignKeyJoinIntegrationTest.java: ## @@ -244,21 +248,21 @@ public void doJoinFromLeftThenDel

Re: [PR] KAFKA-16394: Fix null propagation in foreign key join result [kafka]

2024-05-07 Thread via GitHub
AyoubOm commented on code in PR #15607: URL: https://github.com/apache/kafka/pull/15607#discussion_r1592389047 ## streams/src/test/java/org/apache/kafka/streams/integration/KTableKTableForeignKeyJoinIntegrationTest.java: ## @@ -244,21 +248,21 @@ public void doJoinFromLeftThenDel

Re: [PR] KAFKA-10199: Remove lost tasks in state updater with new remove [kafka]

2024-05-07 Thread via GitHub
cadonna merged PR #15870: URL: https://github.com/apache/kafka/pull/15870 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Created] (KAFKA-16685) RSM Task warn logs do not include parent exception trace

2024-05-07 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-16685: Summary: RSM Task warn logs do not include parent exception trace Key: KAFKA-16685 URL: https://issues.apache.org/jira/browse/KAFKA-16685

[jira] [Updated] (KAFKA-16685) RLMTask warning logs do not include parent exception trace

2024-05-07 Thread Jorge Esteban Quilcate Otoya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Esteban Quilcate Otoya updated KAFKA-16685: - Description: When RLMTask warning exceptions happen and are logg

[jira] [Updated] (KAFKA-16685) RLMTask warn logs do not include parent exception trace

2024-05-07 Thread Jorge Esteban Quilcate Otoya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Esteban Quilcate Otoya updated KAFKA-16685: - Summary: RLMTask warn logs do not include parent exception trace

[jira] [Updated] (KAFKA-16685) RLMTask warning logs do not include parent exception trace

2024-05-07 Thread Jorge Esteban Quilcate Otoya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Esteban Quilcate Otoya updated KAFKA-16685: - Summary: RLMTask warning logs do not include parent exception tra

[PR] KAFKA-16685: Add parent exception to RLMTask warning logs [kafka]

2024-05-07 Thread via GitHub
jeqo opened a new pull request, #15880: URL: https://github.com/apache/kafka/pull/15880 [KAFKA-16685] -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

[PR] KAFKA-16640: Replace TestUtils#resource by scala.util.Using [kafka]

2024-05-07 Thread via GitHub
frankvicky opened a new pull request, #15881: URL: https://github.com/apache/kafka/pull/15881 Check all uses of `TestUtils#resource` and replace with `scala.util.Using` - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentati

[PR] KAFKA-10199: Handle assignment with new remove operation in state updater [kafka]

2024-05-07 Thread via GitHub
cadonna opened a new pull request, #15882: URL: https://github.com/apache/kafka/pull/15882 Uses the new remove operation of the state updater that returns a future to handle task assignment. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementa

Re: [PR] KAFKA-16307: fix coordinator thread idle ratio [kafka]

2024-05-07 Thread via GitHub
dajac merged PR #15835: URL: https://github.com/apache/kafka/pull/15835 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[jira] [Resolved] (KAFKA-16307) fix EventAccumulator thread idle ratio metric

2024-05-07 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16307. - Fix Version/s: 3.8.0 Reviewer: David Jacot Resolution: Fixed > fix EventAccumula

Re: [PR] KAFKA-16574: The metrics of LogCleaner disappear after reconfiguration [kafka]

2024-05-07 Thread via GitHub
chiacyu commented on code in PR #15863: URL: https://github.com/apache/kafka/pull/15863#discussion_r1592509827 ## core/src/main/scala/kafka/log/LogCleaner.scala: ## @@ -182,6 +183,27 @@ class LogCleaner(initialConfig: CleanerConfig, cleanerManager.removeMetrics() Review Co

Re: [PR] KAFKA-16574: The metrics of LogCleaner disappear after reconfiguration [kafka]

2024-05-07 Thread via GitHub
chiacyu commented on code in PR #15863: URL: https://github.com/apache/kafka/pull/15863#discussion_r1592516715 ## core/src/main/scala/kafka/log/LogCleaner.scala: ## @@ -159,6 +159,7 @@ class LogCleaner(initialConfig: CleanerConfig, cleaners += cleaner cleaner.start

Re: [PR] KAFKA-16574: The metrics of LogCleaner disappear after reconfiguration [kafka]

2024-05-07 Thread via GitHub
chia7712 commented on code in PR #15863: URL: https://github.com/apache/kafka/pull/15863#discussion_r1592508554 ## core/src/main/scala/kafka/log/LogCleaner.scala: ## @@ -101,6 +101,7 @@ class LogCleaner(initialConfig: CleanerConfig, time: Time = Time.SYSTEM) ex

[PR] MINOR: Rewrite OptimizedUniformAssignmentBuilder#assignStickyPartitions to improve performance [kafka]

2024-05-07 Thread via GitHub
dajac opened a new pull request, #15883: URL: https://github.com/apache/kafka/pull/15883 This PR is still WIP. I am still playing with an alternative approach. Trunk: ``` Benchmark (assignmentType) (assignorType) (isRackAware) (memberCoun

[jira] [Commented] (KAFKA-16645) CVEs in 3.7.0 docker image

2024-05-07 Thread Igor Soarez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844334#comment-17844334 ] Igor Soarez commented on KAFKA-16645: - The vulnerability report flags {{libexpat}} v

[PR] MINOR: merge unit test down to the class of integration test [kafka]

2024-05-07 Thread via GitHub
KevinZTW opened a new pull request, #15884: URL: https://github.com/apache/kafka/pull/15884 Merge the unit test functions down to the class of integration test to avoid two test class under the same file ### Committer Checklist (excluded from commit message) - [ ] Verify design

Re: [PR] MINOR: enable test for ensureInternalEndpointIsSecured [kafka]

2024-05-07 Thread via GitHub
chia7712 commented on code in PR #15868: URL: https://github.com/apache/kafka/pull/15868#discussion_r1592649000 ## connect/runtime/src/test/java/org/apache/kafka/connect/integration/SessionedProtocolIntegrationTest.java: ## @@ -115,11 +112,11 @@ public void ensureInternalEndpoin

Re: [PR] MINOR: use classic consumer with ZK mode for DeleteOffsetsConsumerGroupCommandIntegrationTest [kafka]

2024-05-07 Thread via GitHub
FrankYang0529 commented on code in PR #15872: URL: https://github.com/apache/kafka/pull/15872#discussion_r1592664550 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/DeleteOffsetsConsumerGroupCommandIntegrationTest.java: ## @@ -88,7 +113,7 @@ public void testDeleteOf

Re: [PR] KAFKA-13328, KAFKA-13329 (2): Add custom preflight validation support for connector header, key, and value converters [kafka]

2024-05-07 Thread via GitHub
C0urante commented on code in PR #14309: URL: https://github.com/apache/kafka/pull/14309#discussion_r1592678706 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractHerder.java: ## @@ -392,6 +399,146 @@ protected Map validateSourceConnectorConfig(SourceConn

Re: [PR] MINOR: migrate ListConsumerGroupTest to use ClusterTestExtensions [kafka]

2024-05-07 Thread via GitHub
FrankYang0529 commented on PR #15821: URL: https://github.com/apache/kafka/pull/15821#issuecomment-2098714976 Hi @chia7712, I rebase latest trunk branch, so we have `ConsumerGroupCommandTestUtils` now. The only remaining part is `SimpleConsumerGroupExecutor`. This is used by `ListConsumerGr

Re: [PR] KAFKA-16362: Fix type-unsafety in KStreamKStreamJoin caused by isLeftSide [kafka]

2024-05-07 Thread via GitHub
gharris1727 commented on code in PR #15601: URL: https://github.com/apache/kafka/pull/15601#discussion_r1591293880 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java: ## @@ -184,20 +146,34 @@ public void process(final Record record) {

Re: [PR] KAFKA-13328, KAFKA-13329 (2): Add custom preflight validation support for connector header, key, and value converters [kafka]

2024-05-07 Thread via GitHub
C0urante commented on PR #14309: URL: https://github.com/apache/kafka/pull/14309#issuecomment-2098726331 Thanks @gharris1727. I've reverted the changes to `Utils.java` and verified locally with tests. Everything else looked okay on the previous CI run, going to merge. -- This is an autom

Re: [PR] Allowing WriteTxnMarkers API to run with AlterCluster permissions [kafka]

2024-05-07 Thread via GitHub
jolshan commented on PR #15837: URL: https://github.com/apache/kafka/pull/15837#issuecomment-2098727340 Sorry I was out of town (at KSB). I will try to take a look today, but thanks Luke for approving as well :) -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] KAFKA-13328, KAFKA-13329 (2): Add custom preflight validation support for connector header, key, and value converters [kafka]

2024-05-07 Thread via GitHub
C0urante merged PR #14309: URL: https://github.com/apache/kafka/pull/14309 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Commented] (KAFKA-16681) Rewrite MiniKDC by Java

2024-05-07 Thread PoAn Yang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844345#comment-17844345 ] PoAn Yang commented on KAFKA-16681: --- Hi [~chia7712], I'm interested in this. If you ar

[jira] [Resolved] (KAFKA-13328) Connect does not perform preflight validation for per-connector header converters

2024-05-07 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-13328. --- Fix Version/s: 3.8.0 Resolution: Fixed > Connect does not perform preflight validatio

[jira] [Resolved] (KAFKA-13329) Connect does not perform preflight validation for per-connector key and value converters

2024-05-07 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-13329. --- Fix Version/s: 3.8.0 Resolution: Fixed > Connect does not perform preflight validatio

[jira] [Assigned] (KAFKA-16681) Rewrite MiniKDC by Java

2024-05-07 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reassigned KAFKA-16681: -- Assignee: PoAn Yang (was: Chia-Ping Tsai) > Rewrite MiniKDC by Java > --

Re: [PR] MINOR: enable test for ensureInternalEndpointIsSecured [kafka]

2024-05-07 Thread via GitHub
FrankYang0529 commented on code in PR #15868: URL: https://github.com/apache/kafka/pull/15868#discussion_r1592697934 ## connect/runtime/src/test/java/org/apache/kafka/connect/integration/SessionedProtocolIntegrationTest.java: ## @@ -115,11 +112,11 @@ public void ensureInternalEn

Re: [PR] MINOR: enable test for ensureInternalEndpointIsSecured [kafka]

2024-05-07 Thread via GitHub
FrankYang0529 commented on code in PR #15868: URL: https://github.com/apache/kafka/pull/15868#discussion_r1592700292 ## connect/runtime/src/test/java/org/apache/kafka/connect/integration/SessionedProtocolIntegrationTest.java: ## @@ -115,11 +112,11 @@ public void ensureInternalEn

Re: [PR] KAFKA-16287: Implement example tests for common rebalance callback [kafka]

2024-05-07 Thread via GitHub
cadonna commented on code in PR #15408: URL: https://github.com/apache/kafka/pull/15408#discussion_r1592700226 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerCallbackTest.scala: ## @@ -0,0 +1,138 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under o

[jira] [Created] (KAFKA-16686) Flakey tests in TopicBasedRemoteLogMetadataManagerTest

2024-05-07 Thread Gaurav Narula (Jira)
Gaurav Narula created KAFKA-16686: - Summary: Flakey tests in TopicBasedRemoteLogMetadataManagerTest Key: KAFKA-16686 URL: https://issues.apache.org/jira/browse/KAFKA-16686 Project: Kafka Issu

Re: [PR] KAFKA-16362: Fix type-unsafety in KStreamKStreamJoin caused by isLeftSide [kafka]

2024-05-07 Thread via GitHub
raminqaf commented on PR #15601: URL: https://github.com/apache/kafka/pull/15601#issuecomment-2098817942 @gharris1727 Thanks for the feedback! I reverted all the changes you requested and reverted a couple of other indentation problems that caused a diff. I can even go further and revert &

[jira] [Commented] (KAFKA-16108) Backport fix for KAFKA-16093 to 3.7

2024-05-07 Thread Igor Soarez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844365#comment-17844365 ] Igor Soarez commented on KAFKA-16108: - [~ChrisEgerton] is this still relevant? The p

Re: [PR] KAFKA-15649: Handle directory failure timeout [kafka]

2024-05-07 Thread via GitHub
viktorsomogyi commented on PR #15697: URL: https://github.com/apache/kafka/pull/15697#issuecomment-2098821932 @soarez at the end I chose the shortcut regarding detecting leaders before shutdown. The reason is complex as the solution that would be required for this is complex too. So on o

[jira] [Commented] (KAFKA-16656) Using a custom replication.policy.separator with DefaultReplicationPolicy

2024-05-07 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844368#comment-17844368 ] Chris Egerton commented on KAFKA-16656: --- Hi [~leninjoseph] it should be possible t

Re: [PR] Kafka-16668: Add tags support in ClusterTestExtension [kafka]

2024-05-07 Thread via GitHub
johnnychhsu commented on code in PR #15861: URL: https://github.com/apache/kafka/pull/15861#discussion_r1592759299 ## core/src/test/java/kafka/test/ClusterConfig.java: ## @@ -153,15 +148,19 @@ public Map> perBrokerOverrideProperties() { return perBrokerOverrideProperti

[jira] [Updated] (KAFKA-16250) Consumer group coordinator should perform sanity check on the offset commits.

2024-05-07 Thread Calvin Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Calvin Liu updated KAFKA-16250: --- Description: The current coordinator does not validate the offset commits before persisting it in t

[jira] [Created] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-07 Thread FTR (Jira)
FTR created KAFKA-16687: --- Summary: Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0 Key: KAFKA-16687 URL: https://issues.apache.org/jira/browse/KAFKA-16687 Project: Kafka Issue Typ

[PR] KAFKA-16686: Wait for given offset in TopicBasedRemoteLogMetadataManagerTest [kafka]

2024-05-07 Thread via GitHub
gaurav-narula opened a new pull request, #15885: URL: https://github.com/apache/kafka/pull/15885 Some tests in TopicBasedRemoteLogMetadataManagerTest flake because `waitUntilConsumerCatchesUp` may break early before consumer manager has caught up with all the events. This PR adds an

Re: [PR] KAFKA-16686: Wait for given offset in TopicBasedRemoteLogMetadataManagerTest [kafka]

2024-05-07 Thread via GitHub
gaurav-narula commented on PR #15885: URL: https://github.com/apache/kafka/pull/15885#issuecomment-2098848112 CC: @clolov @satishd -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[jira] [Commented] (KAFKA-16316) Make the restore behavior of GlobalKTables with custom processors configureable

2024-05-07 Thread Walker Carlson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844373#comment-17844373 ] Walker Carlson commented on KAFKA-16316: KIP-1024 was made for this, but it has

Re: [PR] Kafka-16668: Add tags support in ClusterTestExtension [kafka]

2024-05-07 Thread via GitHub
johnnychhsu commented on PR #15861: URL: https://github.com/apache/kafka/pull/15861#issuecomment-2098850247 @chia7712 > As https://github.com/apache/kafka/pull/15766 gets merged now, could you please fix the build error? sure, let me rebase the latest trunk and fix that since

Re: [PR] KAFKA-16598 Mirgrate `ResetConsumerGroupOffsetTest` to new test infra [kafka]

2024-05-07 Thread via GitHub
m1a2st commented on PR #15779: URL: https://github.com/apache/kafka/pull/15779#issuecomment-2098864822 @chia7712 , please take a look for this PR, thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-16629: Add broker-related tests to ConfigCommandIntegrationTest [kafka]

2024-05-07 Thread via GitHub
m1a2st commented on PR #15840: URL: https://github.com/apache/kafka/pull/15840#issuecomment-2098869565 @chia7712 , I have been add zookeeper `--bootstrap-server` tests, please review, thank you -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Commented] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-07 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844379#comment-17844379 ] Philip Nee commented on KAFKA-16687: Hey [~fortherightous] - we also observed some (

[jira] [Commented] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-07 Thread FTR (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844381#comment-17844381 ] FTR commented on KAFKA-16687: - That's it, there are some topics with 30 partitions. I have t

[jira] [Assigned] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-07 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee reassigned KAFKA-16687: -- Assignee: Philip Nee > Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0 >

[jira] [Commented] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-07 Thread FTR (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844382#comment-17844382 ] FTR commented on KAFKA-16687: - BTW, before that I had also encountered heap leak issue in K

[jira] [Commented] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-07 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844387#comment-17844387 ] Philip Nee commented on KAFKA-16687: But this is native memory leak so I suspect it'

[jira] [Commented] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-07 Thread FTR (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844390#comment-17844390 ] FTR commented on KAFKA-16687: - Surely, let me experiment with 3.6 and then feedback here. >

Re: [PR] KAFKA-9401: Reduce contention for Fetch requests [kafka]

2024-05-07 Thread via GitHub
soarez commented on PR #15836: URL: https://github.com/apache/kafka/pull/15836#issuecomment-2098918354 `:core:test` timed out only on JDK 8 and Scala 2.12. Restarted the build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[jira] [Commented] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-07 Thread FTR (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844394#comment-17844394 ] FTR commented on KAFKA-16687: - I am using stress testing to my Java application. Observing w

[jira] [Comment Edited] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-07 Thread FTR (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844394#comment-17844394 ] FTR edited comment on KAFKA-16687 at 5/7/24 5:12 PM: - I am using str

[jira] [Commented] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-07 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844395#comment-17844395 ] Philip Nee commented on KAFKA-16687: Your application uses KafkaConsumer only right?

[jira] [Commented] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-07 Thread FTR (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844397#comment-17844397 ] FTR commented on KAFKA-16687: - For Kafka client, I have used KafkaConsumer and AdminClient.

[jira] [Commented] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-07 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844400#comment-17844400 ] Philip Nee commented on KAFKA-16687: thanks for the information this is massively he

Re: [PR] MINOR: set test global timeout as 10 mins [kafka]

2024-05-07 Thread via GitHub
gaurav-narula commented on PR #15065: URL: https://github.com/apache/kafka/pull/15065#issuecomment-2098952056 Bumping this thread for more visibility. Would be nice if we give this a shot since we still run into builds which [time out](https://ci-builds.apache.org/blue/organizations/

Re: [PR] KAFKA-16445: Add PATCH method for connector config [kafka]

2024-05-07 Thread via GitHub
C0urante commented on code in PR #6934: URL: https://github.com/apache/kafka/pull/6934#discussion_r1592838802 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -1144,6 +1144,21 @@ public void putConnectorConfig(final Stri

[jira] [Commented] (KAFKA-16687) Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

2024-05-07 Thread FTR (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17844408#comment-17844408 ] FTR commented on KAFKA-16687: - Actually, this Kafka client is used for a web application. Ju

Re: [PR] KAFKA-16445: Add PATCH method for connector config [kafka]

2024-05-07 Thread via GitHub
C0urante commented on code in PR #6934: URL: https://github.com/apache/kafka/pull/6934#discussion_r1592847527 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/resources/ConnectorsResource.java: ## @@ -242,6 +242,19 @@ public Response putConnectorConfig(fina

  1   2   >