Re: [PR] KAFKA-16770; [1/N] Coalesce records into bigger batches [kafka]

2024-05-17 Thread via GitHub
dajac commented on PR #15964: URL: https://github.com/apache/kafka/pull/15964#issuecomment-2116983828 @jolshan Thanks for your comments. I replied to them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-15974: Enforce that event processing respects user-provided timeout [kafka]

2024-05-17 Thread via GitHub
cadonna commented on PR #15640: URL: https://github.com/apache/kafka/pull/15640#issuecomment-2117012544 > High level comment, just to clarify and make sure it's something we are considering and will cover with the follow-up PRs for timeouts. Here we're introducing a component to ensures tha

Re: [PR] KAFKA-14768_KIP913: add new method to provide possibility for warming up first record's sending and reducing the max.block.ms safely. [kafka]

2024-05-17 Thread via GitHub
AlEmerich commented on PR #13320: URL: https://github.com/apache/kafka/pull/13320#issuecomment-2117016372 Hello ! This is a feature I would like to have, actually. I see that the PR is marked as stale but do we have a workaround ? I tried using `partitionsFor` at the beggining of the ap

Re: [PR] KAFKA-15319: Upgrade rocksdb to fix CVE-2022-37434 [kafka]

2024-05-17 Thread via GitHub
mimaison commented on PR #14216: URL: https://github.com/apache/kafka/pull/14216#issuecomment-2117146908 Is there a reason why this PR downgraded requests to 2.24.0? https://github.com/apache/kafka/pull/14216/files#diff-08ac9c7e3ca9bc4f0dcdb04ab5d5c90bd853ea2175df49ae895d538cff7a3268L54

[jira] [Commented] (KAFKA-16708) Allow dynamic port for advertised listeners in Docker image

2024-05-17 Thread Vedarth Sharma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847214#comment-17847214 ] Vedarth Sharma commented on KAFKA-16708: Thanks a lot for raising the ticket and

Re: [PR] KAFKA-15905 Restarts of MirrorCheckpointTask should not permanently i… [kafka]

2024-05-17 Thread via GitHub
edoardocomar commented on code in PR #15910: URL: https://github.com/apache/kafka/pull/15910#discussion_r1604720566 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/OffsetSyncStore.java: ## @@ -105,7 +106,10 @@ private KafkaBasedLog createBackingStore(MirrorCheck

[jira] [Created] (KAFKA-16788) Resource leakage due to absence of close() call on connector start failure

2024-05-17 Thread Ashok (Jira)
Ashok created KAFKA-16788: - Summary: Resource leakage due to absence of close() call on connector start failure Key: KAFKA-16788 URL: https://issues.apache.org/jira/browse/KAFKA-16788 Project: Kafka

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-05-17 Thread via GitHub
vamossagar12 commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1604757990 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -2704,6 +2704,11 @@ public void onRevoked(String leader

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-05-17 Thread via GitHub
vamossagar12 commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1604759527 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/WorkerRebalanceListener.java: ## @@ -36,4 +37,15 @@ public interface WorkerRebalanceL

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-05-17 Thread via GitHub
vamossagar12 commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1604767686 ## connect/runtime/src/test/java/org/apache/kafka/connect/integration/ConnectWorkerIntegrationTest.java: ## @@ -850,6 +855,46 @@ public void testRequestTimeouts()

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-05-17 Thread via GitHub
vamossagar12 commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1604768491 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/WorkerCoordinator.java: ## @@ -267,6 +268,20 @@ public String memberId() { r

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-05-17 Thread via GitHub
vamossagar12 commented on PR #15305: URL: https://github.com/apache/kafka/pull/15305#issuecomment-2117312634 Thanks @gharris1727 for another round of review. I have made the changes. Please review when you have some time. -- This is an automated message from the Apache Git Service. To re

[PR] MINOR: use try-with resources in ClientMetricsManagerTest [kafka]

2024-05-17 Thread via GitHub
gaurav-narula opened a new pull request, #15982: URL: https://github.com/apache/kafka/pull/15982 Ensures we close resources even if an exception is thrown within a test. Also added an `@AfterAll` to ensure threads aren't leaked from the test class. ### Committer Checklist (excluded fr

Re: [PR] KAFKA-15319: Upgrade rocksdb to fix CVE-2022-37434 [kafka]

2024-05-17 Thread via GitHub
lucasbru commented on PR #14216: URL: https://github.com/apache/kafka/pull/14216#issuecomment-2117489283 @mimaison I remember I had problems running the system tests with that version, but I don't think I intended merging the version change. Sorry for that, I'll test the newer version and b

Re: [PR] KAFKA-15319: Upgrade rocksdb to fix CVE-2022-37434 [kafka]

2024-05-17 Thread via GitHub
mimaison commented on PR #14216: URL: https://github.com/apache/kafka/pull/14216#issuecomment-2117500715 Thanks @lucasbru ! I wonder if we also need to bump ducktape to 0.11.4 as it's seems previous versions depend on requests<2.31.0: https://ducktape.readthedocs.io/en/latest/changelog.h

[PR] KAFKA-16783: Migrate RemoteLogMetadataManagerTest to new test infra [kafka]

2024-05-17 Thread via GitHub
FrankYang0529 opened a new pull request, #15983: URL: https://github.com/apache/kafka/pull/15983 * Replace `TopicBasedRemoteLogMetadataManagerWrapperWithHarness` with `RemoteLogMetadataManagerTestUtils#builder` in `RemoteLogMetadataManagerTest`. * Use `ClusterTestExtention` for `RemoteLog

Re: [PR] KAFKA-15319: Upgrade rocksdb to fix CVE-2022-37434 [kafka]

2024-05-17 Thread via GitHub
lucasbru commented on PR #14216: URL: https://github.com/apache/kafka/pull/14216#issuecomment-2117519600 @mimaison -- Let me test this. Yes, I see that also the version of requests inside ducktape was reverted: https://github.com/confluentinc/ducktape/commit/29dcdbeb1a8048505ce1613eb9aad05b

[jira] [Commented] (KAFKA-16784) Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest to new test infra

2024-05-17 Thread PoAn Yang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847272#comment-17847272 ] PoAn Yang commented on KAFKA-16784: --- Hi [~chia7712], I'm interested in this issue. May

[jira] [Commented] (KAFKA-16448) Add Kafka Streams exception handler for exceptions occuring during processing (KIP-1033)

2024-05-17 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847294#comment-17847294 ] Muralidhar Basani commented on KAFKA-16448: --- Thanks [~mjsax]  [~lmunoz] if th

[jira] [Assigned] (KAFKA-16789) Thread leak detection checks for incorrect QuorumController thread name

2024-05-17 Thread Gaurav Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Narula reassigned KAFKA-16789: - Assignee: Gaurav Narula > Thread leak detection checks for incorrect QuorumController t

[jira] [Created] (KAFKA-16789) Thread leak detection checks for incorrect QuorumController thread name

2024-05-17 Thread Gaurav Narula (Jira)
Gaurav Narula created KAFKA-16789: - Summary: Thread leak detection checks for incorrect QuorumController thread name Key: KAFKA-16789 URL: https://issues.apache.org/jira/browse/KAFKA-16789 Project: Ka

Re: [PR] KAFKA-16362: Fix type-unsafety in KStreamKStreamJoin caused by isLeftSide [kafka]

2024-05-17 Thread via GitHub
cadonna commented on code in PR #15601: URL: https://github.com/apache/kafka/pull/15601#discussion_r1604937635 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java: ## @@ -187,17 +157,25 @@ public void process(final Record record) {

Re: [PR] KAFKA-16362: Fix type-unsafety in KStreamKStreamJoin caused by isLeftSide [kafka]

2024-05-17 Thread via GitHub
cadonna commented on code in PR #15601: URL: https://github.com/apache/kafka/pull/15601#discussion_r1605020116 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamLeftJoin.java: ## @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-16066: Upgrade apacheds to 2.0.0.AM27 With apache kerby [kafka]

2024-05-17 Thread via GitHub
mimaison commented on PR #15277: URL: https://github.com/apache/kafka/pull/15277#issuecomment-2117650240 @highluck Can you address the pending comments and rebase on trunk to resolve the conflicts? Thanks -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] KAFKA-16675: Refactored and new rebalance callbacks integration tests [kafka]

2024-05-17 Thread via GitHub
lianetm commented on code in PR #15965: URL: https://github.com/apache/kafka/pull/15965#discussion_r1605041571 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerCallbackTest.scala: ## @@ -84,29 +84,87 @@ class PlaintextConsumerCallbackTest extends AbstractConsumerTe

Re: [PR] KAFKA-16675: Refactored and new rebalance callbacks integration tests [kafka]

2024-05-17 Thread via GitHub
lianetm commented on PR #15965: URL: https://github.com/apache/kafka/pull/15965#issuecomment-2117652874 Thanks for the helpful comment @lucasbru , addressed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] KAFKA-15746: KRaft support in ControllerMutationQuotaTest [kafka]

2024-05-17 Thread via GitHub
mimaison commented on PR #15038: URL: https://github.com/apache/kafka/pull/15038#issuecomment-2117661783 @linzihao1999 Can you follow up this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] KAFKA-16789: Fix thread leak detection for event handler threads [kafka]

2024-05-17 Thread via GitHub
gaurav-narula opened a new pull request, #15984: URL: https://github.com/apache/kafka/pull/15984 Updates `TestUtils::verifyNoUnexpectedThreads` to check for all event queue threads instead of the incorrect QuorumController thread. Also updated `KafkaEventQueueTest` to use try-with-res

[jira] [Assigned] (KAFKA-16784) Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest to new test infra

2024-05-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reassigned KAFKA-16784: -- Assignee: PoAn Yang (was: Chia-Ping Tsai) > Migrate TopicBasedRemoteLogMetadataManag

Re: [PR] KAFKA-16762: SyncGroup API for upgrading ConsumerGroup [kafka]

2024-05-17 Thread via GitHub
dajac merged PR #15954: URL: https://github.com/apache/kafka/pull/15954 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[PR] KAFKA-12399: Deprecate KafkaLog4jAppender [kafka]

2024-05-17 Thread via GitHub
mimaison opened a new pull request, #15985: URL: https://github.com/apache/kafka/pull/15985 As per KIP-719, KafkaLog4jAppender is now deprecated and will be removed in Kafka 4.0. Users should migrate to the log4j2 Kafka Appender ### Committer Checklist (excluded from commit message)

[jira] [Resolved] (KAFKA-16762) SyncGroup API for upgrading ConsumerGroup

2024-05-17 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16762. - Fix Version/s: 3.8.0 Reviewer: David Jacot Resolution: Fixed > SyncGroup API for

[jira] [Commented] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2024-05-17 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847334#comment-17847334 ] Lucas Brutschy commented on KAFKA-12679: Hey [~coltmcnealy-lh]. It is true that

[jira] [Updated] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2024-05-17 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy updated KAFKA-12679: --- Fix Version/s: 3.8.0 (was: 3.7.0) > Rebalancing a restoring or runnin

Re: [PR] KAFKA-12399: Deprecate KafkaLog4jAppender [kafka]

2024-05-17 Thread via GitHub
mimaison commented on PR #15985: URL: https://github.com/apache/kafka/pull/15985#issuecomment-2117717230 @viktorsomogyi IIRC you volunteered to do the migration work to log4j2 a while back. Are you still able/willing to do so for Kafka 4.0? @jlprat As there's a chance 3.8 will be the

[jira] [Comment Edited] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2024-05-17 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847334#comment-17847334 ] Lucas Brutschy edited comment on KAFKA-12679 at 5/17/24 2:20 PM: -

[jira] [Comment Edited] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2024-05-17 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847334#comment-17847334 ] Lucas Brutschy edited comment on KAFKA-12679 at 5/17/24 2:26 PM: -

Re: [PR] [Draft] KAFKA-16764: New consumer should throw InvalidTopicException on poll when invalid topic in metadata [kafka]

2024-05-17 Thread via GitHub
appchemist commented on PR #15961: URL: https://github.com/apache/kafka/pull/15961#issuecomment-2117739675 @lianetm & @philipnee PTAL `testSubscriptionOnInvalidTopic` is still disabled for new consumer. I couldn't make the changes because `MockMetadataUpdater` is being used interna

Re: [PR] KAFKA-15974: Enforce that event processing respects user-provided timeout [kafka]

2024-05-17 Thread via GitHub
lianetm commented on PR #15640: URL: https://github.com/apache/kafka/pull/15640#issuecomment-2117763853 Hey @cadonna, the tricky bit is that, for some events, the request managers do expire requests too, so in this flow you described: > The event is processed in the ApplicationEventHa

[jira] [Commented] (KAFKA-16333) Removed Deprecated methods KTable#join

2024-05-17 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847348#comment-17847348 ] Muralidhar Basani commented on KAFKA-16333: --- Agree. thanks. > Removed Depreca

[jira] [Commented] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-17 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847352#comment-17847352 ] Justine Olshan commented on KAFKA-16692: [~soarez] I'd like to try to get this f

Re: [PR] KAFKA-16625: Reverse lookup map from topic partitions to members [kafka]

2024-05-17 Thread via GitHub
dajac commented on code in PR #15974: URL: https://github.com/apache/kafka/pull/15974#discussion_r1605223809 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AbstractUniformAssignmentBuilder.java: ## @@ -65,4 +65,33 @@ protected static Set topicIdPa

[jira] [Commented] (KAFKA-15197) Flaky test MirrorConnectorsIntegrationExactlyOnceTest.testOffsetTranslationBehindReplicationFlow()

2024-05-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847366#comment-17847366 ] Chia-Ping Tsai commented on KAFKA-15197: It seems that behavior is expected. T

[jira] [Assigned] (KAFKA-16758) Extend Consumer#close with option to leave the group or not

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans reassigned KAFKA-16758: -- Assignee: Lianet Magrans > Extend Consumer#close with option to leave the group or no

[jira] [Updated] (KAFKA-16774) fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEnabled

2024-05-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-16774: --- Fix Version/s: 3.8.0 > fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEna

[jira] [Commented] (KAFKA-14517) Implement regex subscriptions

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847369#comment-17847369 ] Lianet Magrans commented on KAFKA-14517: hey [~phuctran], [~dajac] , I would lik

Re: [PR] KAFKA-16763: Upgrade to scala 2.12.19 and scala 2.13.14 [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15958: URL: https://github.com/apache/kafka/pull/15958 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Resolved] (KAFKA-16763) Upgrade to scala 2.12.19 and scala 2.13.14

2024-05-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16763. Fix Version/s: 3.8.0 Resolution: Fixed > Upgrade to scala 2.12.19 and scala 2.13.14

[PR] KAFKA-16252; Dynamic KRaft network manager and channel [kafka]

2024-05-17 Thread via GitHub
jsancio opened a new pull request, #15986: URL: https://github.com/apache/kafka/pull/15986 DRAFT ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including

Re: [PR] KAFKA-7632: Support Compression Levels (KIP-390) [kafka]

2024-05-17 Thread via GitHub
junrao commented on code in PR #15516: URL: https://github.com/apache/kafka/pull/15516#discussion_r1605313096 ## core/src/test/scala/unit/kafka/log/LogValidatorTest.scala: ## @@ -1538,7 +1563,80 @@ class LogValidatorTest { assertEquals(e.recordErrors.size, 3) } - priv

[jira] [Created] (KAFKA-16790) Calls to RemoteLogManager are made before it is configured

2024-05-17 Thread Christo Lolov (Jira)
Christo Lolov created KAFKA-16790: - Summary: Calls to RemoteLogManager are made before it is configured Key: KAFKA-16790 URL: https://issues.apache.org/jira/browse/KAFKA-16790 Project: Kafka

Re: [PR] KAFKA-15905 Restarts of MirrorCheckpointTask should not permanently i… [kafka]

2024-05-17 Thread via GitHub
prestona commented on PR #15910: URL: https://github.com/apache/kafka/pull/15910#issuecomment-2118027385 Hi @gharris1727, hopefully the latest commits address your review comments. Once again, really appreciate all your feedback and suggestions. -- This is an automated message from the Ap

[jira] [Updated] (KAFKA-16790) Calls to RemoteLogManager are made before it is configured

2024-05-17 Thread Christo Lolov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov updated KAFKA-16790: -- Description: BrokerMetadataPublisher#onMetadataUpdate calls ReplicaManager#applyDelta (1) whi

[jira] [Updated] (KAFKA-16790) Calls to RemoteLogManager are made before it is configured

2024-05-17 Thread Christo Lolov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov updated KAFKA-16790: -- Description: BrokerMetadataPublisher#onMetadataUpdate calls ReplicaManager#applyDelta (1) whi

Re: [PR] KAFKA-16705 the flag "started" of RaftClusterInstance is false even though the cluster is started [kafka]

2024-05-17 Thread via GitHub
chia7712 commented on code in PR #15946: URL: https://github.com/apache/kafka/pull/15946#discussion_r1605361253 ## core/src/test/java/kafka/test/junit/RaftClusterInvocationContext.java: ## @@ -240,28 +215,61 @@ public void startBroker(int brokerId) { @Override

Re: [PR] KAFKA-16574: The metrics of LogCleaner disappear after reconfiguration [kafka]

2024-05-17 Thread via GitHub
chia7712 commented on code in PR #15863: URL: https://github.com/apache/kafka/pull/15863#discussion_r1605365153 ## core/src/main/scala/kafka/log/LogCleaner.scala: ## @@ -182,6 +183,27 @@ class LogCleaner(initialConfig: CleanerConfig, cleanerManager.removeMetrics() Review C

[jira] [Commented] (KAFKA-16788) Resource leakage due to absence of close() call on connector start failure

2024-05-17 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847383#comment-17847383 ] Greg Harris commented on KAFKA-16788: - Nice find [~ashok89]! Are you interested in w

Re: [PR] MINOR: Refactor write timeout in CoordinatorRuntime [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15976: URL: https://github.com/apache/kafka/pull/15976 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Commented] (KAFKA-16790) Calls to RemoteLogManager are made before it is configured

2024-05-17 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847384#comment-17847384 ] Muralidhar Basani commented on KAFKA-16790: --- [~christo_lolov] can I look into

[jira] [Commented] (KAFKA-16788) Resource leakage due to absence of close() call on connector start failure

2024-05-17 Thread Ashok (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847385#comment-17847385 ] Ashok commented on KAFKA-16788: --- [~gharris1727] Thanks. yes, I am working on the fix. I wi

[jira] [Commented] (KAFKA-14517) Implement regex subscriptions

2024-05-17 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847386#comment-17847386 ] Phuc Hong Tran commented on KAFKA-14517: [~lianetm] I’m in the process of implem

Re: [PR] KAFKA-16654:Refactor kafka.test.annotation.Type and ClusterTestExtensions [kafka]

2024-05-17 Thread via GitHub
chia7712 commented on code in PR #15916: URL: https://github.com/apache/kafka/pull/15916#discussion_r1605371498 ## core/src/test/java/kafka/test/ClusterTestExtensionsTest.java: ## @@ -62,10 +63,10 @@ public class ClusterTestExtensionsTest { } // Static methods can ge

Re: [PR] MINOR: Fix rate metric spikes [kafka]

2024-05-17 Thread via GitHub
junrao commented on code in PR #15889: URL: https://github.com/apache/kafka/pull/15889#discussion_r1605350575 ## clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java: ## @@ -110,40 +111,43 @@ public String toString() { protected void purgeObsoleteSam

Re: [PR] KAFKA-16625: Reverse lookup map from topic partitions to members [kafka]

2024-05-17 Thread via GitHub
rreddy-22 commented on code in PR #15974: URL: https://github.com/apache/kafka/pull/15974#discussion_r1605382569 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -147,6 +147,11 @@ public static class DeadlineAndEpoch {

[jira] [Commented] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-17 Thread Johnson Okorie (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847391#comment-17847391 ] Johnson Okorie commented on KAFKA-16692: Hi [~jolshan], I wanted to express my s

[jira] [Commented] (KAFKA-16790) Calls to RemoteLogManager are made before it is configured

2024-05-17 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847392#comment-17847392 ] Muralidhar Basani commented on KAFKA-16790: --- Even though brokerMetadataPublish

Re: [PR] KAFKA-16625: Reverse lookup map from topic partitions to members [kafka]

2024-05-17 Thread via GitHub
dajac commented on code in PR #15974: URL: https://github.com/apache/kafka/pull/15974#discussion_r1605415364 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -147,6 +147,11 @@ public static class DeadlineAndEpoch { */

Re: [PR] KAFKA-16626: Lazily convert subscribed topic names to topic ids [kafka]

2024-05-17 Thread via GitHub
jeffkbkim commented on PR #15970: URL: https://github.com/apache/kafka/pull/15970#issuecomment-2118184548 RangeAssignor baseline ``` Benchmark (memberCount) (partitionsToMemberRatio) (topicCount) Mode CntScoreError Units TargetAssignm

Re: [PR] KAFKA-7632: Support Compression Levels (KIP-390) [kafka]

2024-05-17 Thread via GitHub
mimaison commented on code in PR #15516: URL: https://github.com/apache/kafka/pull/15516#discussion_r1605420401 ## core/src/test/scala/unit/kafka/log/LogValidatorTest.scala: ## @@ -1538,7 +1563,80 @@ class LogValidatorTest { assertEquals(e.recordErrors.size, 3) } - pr

Re: [PR] KAFKA-16789: Fix thread leak detection for event handler threads [kafka]

2024-05-17 Thread via GitHub
chia7712 commented on code in PR #15984: URL: https://github.com/apache/kafka/pull/15984#discussion_r1605427011 ## core/src/test/scala/unit/kafka/utils/TestUtils.scala: ## @@ -1872,7 +1872,7 @@ object TestUtils extends Logging { AdminClientUnitTestEnv.kafkaAdminClientNetw

Re: [PR] MINOR: Add debug enablement check when using log.debug [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15977: URL: https://github.com/apache/kafka/pull/15977 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] MINOR: use try-with resources in ClientMetricsManagerTest [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15982: URL: https://github.com/apache/kafka/pull/15982 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
jeffkbkim opened a new pull request, #15987: URL: https://github.com/apache/kafka/pull/15987 DelayEventAccumulator should return immediately if there are no events in the queue. Also removed some unused fields inside EventProcessorThread ### Committer Checklist (excluded from commit m

[jira] [Created] (KAFKA-16791) Add thread detection to ClusterTestExtensions

2024-05-17 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16791: -- Summary: Add thread detection to ClusterTestExtensions Key: KAFKA-16791 URL: https://issues.apache.org/jira/browse/KAFKA-16791 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-16792) Enable related unit tests that fail to fetch offsets only for new consumer with to poll(0)

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16792: --- Description: Enable the following unit tests for the new async consumer in KafkaConsumerTest

[jira] [Updated] (KAFKA-16792) Enable related unit tests that fail to fetch offsets only for new consumer with to poll(0)

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16792: --- Summary: Enable related unit tests that fail to fetch offsets only for new consumer with to

[jira] [Created] (KAFKA-16792) Enable related unit tests that fail only for new consumer with to poll(0)

2024-05-17 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-16792: -- Summary: Enable related unit tests that fail only for new consumer with to poll(0) Key: KAFKA-16792 URL: https://issues.apache.org/jira/browse/KAFKA-16792 Project

[jira] [Commented] (KAFKA-16764) New consumer should throw InvalidTopicException on poll when invalid topic in metadata

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847427#comment-17847427 ] Lianet Magrans commented on KAFKA-16764: Sure! Thanks for helping out! Follow-up

[jira] [Assigned] (KAFKA-16764) New consumer should throw InvalidTopicException on poll when invalid topic in metadata

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans reassigned KAFKA-16764: -- Assignee: appchemist > New consumer should throw InvalidTopicException on poll when i

[jira] [Updated] (KAFKA-16792) Enable consumer unit tests that fail to fetch offsets only for new consumer with poll(0)

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16792: --- Summary: Enable consumer unit tests that fail to fetch offsets only for new consumer with po

Re: [PR] KAFKA-7342: Migrate tests in o.a.k.streams to JUnit 5 (except KafkaStreamsTest) [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15942: URL: https://github.com/apache/kafka/pull/15942 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-16544 DescribeTopicsResult#allTopicIds and DescribeTopicsResult#allTopicNames should return null instead of throwing NPE [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15979: URL: https://github.com/apache/kafka/pull/15979 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Resolved] (KAFKA-16544) DescribeTopicsResult#allTopicIds and DescribeTopicsResult#allTopicNames should return null instead of throwing NPE

2024-05-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16544. Fix Version/s: 3.8 Resolution: Fixed > DescribeTopicsResult#allTopicIds and Describ

Re: [PR] KAFKA-15723: KRaft support in ListOffsetsRequestTest [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15980: URL: https://github.com/apache/kafka/pull/15980 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Resolved] (KAFKA-15723) KRaft support in ListOffsetsRequestTest

2024-05-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-15723. Fix Version/s: 3.8.0 Resolution: Fixed > KRaft support in ListOffsetsRequestTest >

Re: [PR] KAFKA-16223: Replace EasyMock/PowerMock with Mockito for KafkaConfigBackingStoreTest [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15933: URL: https://github.com/apache/kafka/pull/15933 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
gaurav-narula commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605472276 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -475,9 +478,7 @@ public void testRecordT

Re: [PR] MINOR: Remove deprecated constructors from Connect's Kafka*BackingStore classes [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15865: URL: https://github.com/apache/kafka/pull/15865 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Created] (KAFKA-16793) Heartbeat API for upgrading ConsumerGroup

2024-05-17 Thread Dongnuo Lyu (Jira)
Dongnuo Lyu created KAFKA-16793: --- Summary: Heartbeat API for upgrading ConsumerGroup Key: KAFKA-16793 URL: https://issues.apache.org/jira/browse/KAFKA-16793 Project: Kafka Issue Type: Sub-task

[PR] KAFKA-16793: Heartbeat API for upgrading ConsumerGroup [kafka]

2024-05-17 Thread via GitHub
dongnuo123 opened a new pull request, #15988: URL: https://github.com/apache/kafka/pull/15988 This patch implements the heartbeat api to the members that use the classic protocol in a ConsumerGroup. ### Committer Checklist (excluded from commit message) - [ ] Verify design and impl

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
jeffkbkim commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605519141 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -61,8 +61,11 @@ public DelayEventAccumulator

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
gaurav-narula commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605525333 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -61,8 +61,11 @@ public DelayEventAccumul

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
gaurav-narula commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605528394 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -61,8 +61,11 @@ public DelayEventAccumul

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
gaurav-narula commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605530429 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -61,8 +61,11 @@ public DelayEventAccumul

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
gaurav-narula commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605530429 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -61,8 +61,11 @@ public DelayEventAccumul

Re: [PR] KAFKA-16625: Reverse lookup map from topic partitions to members [kafka]

2024-05-17 Thread via GitHub
jeffkbkim commented on code in PR #15974: URL: https://github.com/apache/kafka/pull/15974#discussion_r1605535757 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -59,7 +59,7 @@ public class OptimizedUnif

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
gaurav-narula commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605528394 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -61,8 +61,11 @@ public DelayEventAccumul

Re: [PR] KAFKA-15974: Enforce that event processing respects user-provided timeout [kafka]

2024-05-17 Thread via GitHub
kirktrue commented on PR #15640: URL: https://github.com/apache/kafka/pull/15640#issuecomment-2118578191 > Hey @cadonna, the tricky bit is that, for some events, the request managers do expire requests too, so in this flow you described: > > > The event is processed in the Application

Re: [PR] KAFKA-15974: Enforce that event processing respects user-provided timeout [kafka]

2024-05-17 Thread via GitHub
kirktrue commented on PR #15640: URL: https://github.com/apache/kafka/pull/15640#issuecomment-2118579935 @lianetm @cadonna—I believe I have addressed all the actionable feedback. Are there additional concerns about this PR that prevent it from being merged? Thanks. -- This is an automate

Re: [PR] KAFKA-15045: (KIP-924 pt. 5) Add rack information to ApplicationState [kafka]

2024-05-17 Thread via GitHub
ableegoldman commented on code in PR #15972: URL: https://github.com/apache/kafka/pull/15972#discussion_r1605639072 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java: ## @@ -472,23 +482,74 @@ public GroupAssignment assign(final C

  1   2   >