[GitHub] [kafka] C0urante opened a new pull request, #13137: KAFKA-15086: Intra-cluster communication for MirrorMaker 2

C0urante opened a new pull request, #13137: URL: https://github.com/apache/kafka/pull/13137 [Jira](https://issues.apache.org/jira/browse/KAFKA-10586) Implements the internal REST API changes described in [KIP-710](https://cwiki.apache.org/confluence/display/KAFKA/KIP-710%3A+Full+suppo

[GitHub] [kafka] C0urante commented on pull request #13137: KAFKA-15086: Intra-cluster communication for Mirror Maker 2

C0urante commented on PR #13137: URL: https://github.com/apache/kafka/pull/13137#issuecomment-1398797302 @gharris1727 @viktorsomogyi @mimaison would you mind taking a look at this when you have a moment? -- This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [kafka] hachikuji opened a new pull request, #13138: MINOR: Small cleanups in refactored consumer implementation

hachikuji opened a new pull request, #13138: URL: https://github.com/apache/kafka/pull/13138 This patch contains a few cleanups in the new refactored consumer logic: - Use `CompletableFuture` instead of `RequestFuture` in `NetworkClientDelegate`. This is a much more extensible API and

[GitHub] [kafka] philipnee commented on a diff in pull request #13138: MINOR: Small cleanups in refactored consumer implementation

philipnee commented on code in PR #13138: URL: https://github.com/apache/kafka/pull/13138#discussion_r1082970121 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/NetworkClientDelegate.java: ## @@ -97,8 +97,8 @@ private void trySend(final long currentTimeMs) {

[GitHub] [kafka] kirktrue commented on a diff in pull request #12813: KAFKA-14317: ProduceRequest timeouts are logged as network exceptions

kirktrue commented on code in PR #12813: URL: https://github.com/apache/kafka/pull/12813#discussion_r1082997576 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java: ## @@ -641,7 +643,7 @@ private void completeBatch(ProducerBatch batch, ProduceRespon

[GitHub] [kafka] kirktrue commented on a diff in pull request #12813: KAFKA-14317: ProduceRequest timeouts are logged as network exceptions

kirktrue commented on code in PR #12813: URL: https://github.com/apache/kafka/pull/12813#discussion_r1082997268 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java: ## @@ -552,9 +552,11 @@ private void handleProduceResponse(ClientResponse response,

[GitHub] [kafka] kirktrue commented on a diff in pull request #12813: KAFKA-14317: ProduceRequest timeouts are logged as network exceptions

kirktrue commented on code in PR #12813: URL: https://github.com/apache/kafka/pull/12813#discussion_r1082998126 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java: ## @@ -2519,7 +2519,7 @@ public void testInflightBatchesExpireOnDeliveryTimeout()

[GitHub] [kafka] hachikuji commented on a diff in pull request #13138: MINOR: Small cleanups in refactored consumer implementation

hachikuji commented on code in PR #13138: URL: https://github.com/apache/kafka/pull/13138#discussion_r1083069990 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/NetworkClientDelegate.java: ## @@ -97,8 +97,8 @@ private void trySend(final long currentTimeMs) {

[GitHub] [kafka] ableegoldman commented on pull request #13134: MINOR: fix flaky StickyAssignorTest.testLargeAssignmentAndGroupWithNonEqualSubscription

ableegoldman commented on PR #13134: URL: https://github.com/apache/kafka/pull/13134#issuecomment-1398999167 It's just timing out, there's no error beyond that -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [kafka] ableegoldman merged pull request #13134: MINOR: fix flaky StickyAssignorTest.testLargeAssignmentAndGroupWithNonEqualSubscription

ableegoldman merged PR #13134: URL: https://github.com/apache/kafka/pull/13134 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.ap

[GitHub] [kafka] ableegoldman commented on pull request #13134: MINOR: fix flaky StickyAssignorTest.testLargeAssignmentAndGroupWithNonEqualSubscription

ableegoldman commented on PR #13134: URL: https://github.com/apache/kafka/pull/13134#issuecomment-1399019480 Merged to trunk and cherrypicked to 3.4 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [kafka] ableegoldman merged pull request #13132: MINOR: fix warnings in Streams javadocs

ableegoldman merged PR #13132: URL: https://github.com/apache/kafka/pull/13132 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.ap

[GitHub] [kafka] philipnee opened a new pull request, #13139: MINOR: Update outdate documentation for the SubscriptionState

philipnee opened a new pull request, #13139: URL: https://github.com/apache/kafka/pull/13139 The current documentation indicates two positions are tracked, but these positions were removed a few years ago. Now we use a single position to track the last consumed record. Updated the documenta

[GitHub] [kafka] hachikuji opened a new pull request, #13140: KAFKA-14644: Process should crash after failure in Raft IO thread

hachikuji opened a new pull request, #13140: URL: https://github.com/apache/kafka/pull/13140 Unexpected errors caught in the Raft IO thread should cause the process to stop. This is similar to the handling of exceptions in the controller. ### Committer Checklist (excluded from commit

[GitHub] [kafka] guozhangwang merged pull request #13139: MINOR: Update outdate documentation for the SubscriptionState

guozhangwang merged PR #13139: URL: https://github.com/apache/kafka/pull/13139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.ap

[GitHub] [kafka] cmccabe commented on a diff in pull request #13140: KAFKA-14644: Process should crash after failure in Raft IO thread

cmccabe commented on code in PR #13140: URL: https://github.com/apache/kafka/pull/13140#discussion_r1083120749 ## core/src/main/scala/kafka/raft/RaftManager.scala: ## @@ -47,19 +47,27 @@ import org.apache.kafka.raft.RaftConfig.{AddressSpec, InetAddressSpec, NON_ROUTA import or

[GitHub] [kafka] cmccabe commented on a diff in pull request #13140: KAFKA-14644: Process should crash after failure in Raft IO thread

cmccabe commented on code in PR #13140: URL: https://github.com/apache/kafka/pull/13140#discussion_r1083122187 ## core/src/test/scala/unit/kafka/raft/RaftManagerTest.scala: ## @@ -221,16 +225,23 @@ class RaftManagerTest { @Test def testUncaughtExceptionInIoThread(): Unit =

[GitHub] [kafka] guozhangwang commented on a diff in pull request #13138: MINOR: Small cleanups in refactored consumer implementation

guozhangwang commented on code in PR #13138: URL: https://github.com/apache/kafka/pull/13138#discussion_r1083125061 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CoordinatorRequestManager.java: ## @@ -149,11 +156,14 @@ private void onSuccessfulResponse(fin

[GitHub] [kafka] guozhangwang commented on a diff in pull request #12813: KAFKA-14317: ProduceRequest timeouts are logged as network exceptions

guozhangwang commented on code in PR #12813: URL: https://github.com/apache/kafka/pull/12813#discussion_r1083134655 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java: ## @@ -549,7 +549,13 @@ private boolean awaitNodeReady(Node node, FindCoordinato

[GitHub] [kafka] kirktrue closed pull request #12945: KAFKA-14365: Refactor Fetcher to allow different implementations

kirktrue closed pull request #12945: KAFKA-14365: Refactor Fetcher to allow different implementations URL: https://github.com/apache/kafka/pull/12945 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] philipnee commented on pull request #13139: MINOR: Update outdate documentation for the SubscriptionState

philipnee commented on PR #13139: URL: https://github.com/apache/kafka/pull/13139#issuecomment-1399093410 Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

[GitHub] [kafka] ableegoldman opened a new pull request, #13141: MINOR: fix flaky integrations tests by using 60s default timeout for startup

ableegoldman opened a new pull request, #13141: URL: https://github.com/apache/kafka/pull/13141 The timeouts used for starting up Streams and waiting for the RUNNING state are all over the place across our integration tests, with some as low as 15s (which are unsurprisingly rather flaky). W

[GitHub] [kafka] vcrfxia opened a new pull request, #13142: KAFKA-14491: [2/N] Refactor RocksDB store open iterator management

vcrfxia opened a new pull request, #13142: URL: https://github.com/apache/kafka/pull/13142 This PR refactors how the list of open iterators for RocksDB stores is managed. Prior to this PR, the `openIterators` list was passed into the constructor for the iterators themselves, allowing `Rocks

[GitHub] [kafka] vcrfxia opened a new pull request, #13143: KAFKA-14491: [3/N] Add logical key value segments

vcrfxia opened a new pull request, #13143: URL: https://github.com/apache/kafka/pull/13143 (This PR is stacked on https://github.com/apache/kafka/pull/13142. The first commit does not need to be reviewed separately.) Today's KeyValueSegments create a new RocksDB instance for each Key

[GitHub] [kafka] hachikuji commented on a diff in pull request #13138: MINOR: Small cleanups in refactored consumer implementation

hachikuji commented on code in PR #13138: URL: https://github.com/apache/kafka/pull/13138#discussion_r1083199766 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CoordinatorRequestManager.java: ## @@ -149,11 +156,14 @@ private void onSuccessfulResponse(final

[GitHub] [kafka] mjsax commented on a diff in pull request #13126: KAFKA-14491: [1/N] Add segment value format for RocksDB versioned store

mjsax commented on code in PR #13126: URL: https://github.com/apache/kafka/pull/13126#discussion_r1083223501 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java: ## @@ -0,0 +1,523 @@ +/* + * Licensed to the Apache Sof

[GitHub] [kafka] mjsax commented on a diff in pull request #13126: KAFKA-14491: [1/N] Add segment value format for RocksDB versioned store

mjsax commented on code in PR #13126: URL: https://github.com/apache/kafka/pull/13126#discussion_r1083224137 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java: ## @@ -0,0 +1,523 @@ +/* + * Licensed to the Apache Sof

[GitHub] [kafka] mjsax commented on a diff in pull request #13126: KAFKA-14491: [1/N] Add segment value format for RocksDB versioned store

mjsax commented on code in PR #13126: URL: https://github.com/apache/kafka/pull/13126#discussion_r1081924210 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java: ## @@ -0,0 +1,523 @@ +/* + * Licensed to the Apache Sof

[GitHub] [kafka] mjsax commented on a diff in pull request #13126: KAFKA-14491: [1/N] Add segment value format for RocksDB versioned store

mjsax commented on code in PR #13126: URL: https://github.com/apache/kafka/pull/13126#discussion_r1083231834 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java: ## @@ -0,0 +1,523 @@ +/* + * Licensed to the Apache Sof

[GitHub] [kafka] yufeiyan1220 commented on pull request #13125: KAFKA-14626: Kafka Consumer Coordinator does not cleanup all metrics after shutdown

yufeiyan1220 commented on PR #13125: URL: https://github.com/apache/kafka/pull/13125#issuecomment-1399206848 > Seems like we aren't particularly consistent at removing these metrics and sensors, fetcher would be another example. Mind making the clean up more comprehensive? I have mad

[GitHub] [kafka] nizhikov opened a new pull request, #13144: KAFKA-14463 Invoke of ConnectorClientConfigOverridePolicy#close

nizhikov opened a new pull request, #13144: URL: https://github.com/apache/kafka/pull/13144 `ConnectorClientConfigOverridePolicy` implements `AutoCloseable` but close method not called. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation

[GitHub] [kafka] vamossagar12 commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

vamossagar12 commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1083271427 ## core/src/main/scala/kafka/admin/ConfigCommand.scala: ## @@ -771,8 +772,8 @@ object ConfigCommand extends Logging { .withRequiredArg .describedAs("

[GitHub] [kafka] vamossagar12 commented on pull request #11592: KAFKA-13501: Avoid state restore via rebalance if standbys are enabled

vamossagar12 commented on PR #11592: URL: https://github.com/apache/kafka/pull/11592#issuecomment-1399237152 @mjsax , This is a very old PR of mine which didn't get merged. On the ticket I see `new-streams-runtime-should-fix` added as a label. Is this fix needed anymore or should I close it

[GitHub] [kafka] vamossagar12 closed pull request #9756: KAFKA-10652: Adding size based linger semnatics to Raft metadata

vamossagar12 closed pull request #9756: KAFKA-10652: Adding size based linger semnatics to Raft metadata URL: https://github.com/apache/kafka/pull/9756 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [kafka] ijuma commented on pull request #13121: MINOR: Remove some connect tests from Java 17 block list

ijuma commented on PR #13121: URL: https://github.com/apache/kafka/pull/13121#issuecomment-1399275209 The JDK 17 failed tests are not one of the ones I unblocked in this PR: > Build / JDK 17 and Scala 2.13 / org.apache.kafka.streams.integration.SmokeTestDriverIntegrationTest.[1] true

[GitHub] [kafka] ijuma merged pull request #13121: MINOR: Remove some connect tests from Java 17 block list

ijuma merged PR #13121: URL: https://github.com/apache/kafka/pull/13121 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] fvaleri commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

fvaleri commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1083318714 ## core/src/main/scala/kafka/admin/ConfigCommand.scala: ## @@ -771,8 +772,8 @@ object ConfigCommand extends Logging { .withRequiredArg .describedAs("comma

[GitHub] [kafka] fvaleri commented on pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

fvaleri commented on PR #13131: URL: https://github.com/apache/kafka/pull/13131#issuecomment-1399304541 @clolov @vamossagar12 I took your suggestions. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [kafka] fvaleri commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

fvaleri commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1083319516 ## server-common/src/main/java/org/apache/kafka/server/util/CommandLineUtils.java: ## @@ -0,0 +1,201 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

[GitHub] [kafka] vamossagar12 commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

vamossagar12 commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1083413458 ## core/src/main/scala/kafka/utils/ToolsUtils.scala: ## @@ -64,4 +65,18 @@ object ToolsUtils { println(s"%-${maxLengthOfDisplayName}s : $specifier".forma

[GitHub] [kafka] vamossagar12 commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

vamossagar12 commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1083413458 ## core/src/main/scala/kafka/utils/ToolsUtils.scala: ## @@ -64,4 +65,18 @@ object ToolsUtils { println(s"%-${maxLengthOfDisplayName}s : $specifier".forma

[GitHub] [kafka] vamossagar12 commented on pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

vamossagar12 commented on PR #13131: URL: https://github.com/apache/kafka/pull/13131#issuecomment-1399432431 > @clolov @vamossagar12 took your suggestions. Thanks. Thanks @fvaleri ! Couple of minor comments. This should be good to go after that! -- This is an automated message from

[GitHub] [kafka] vamossagar12 commented on pull request #12561: KAFKA-12495: Exponential backoff retry to prevent rebalance storms when worker joins after revoking rebalance

vamossagar12 commented on PR #12561: URL: https://github.com/apache/kafka/pull/12561#issuecomment-1399432804 hwy @C0urante , I was thinking should the exponential backoff thing that we have introduced as part of this PR should go somewhere in the docs? I am saying this since this is a devia

[GitHub] [kafka] mimaison opened a new pull request, #13145: MINOR: Remove duplicate empty string check

mimaison opened a new pull request, #13145: URL: https://github.com/apache/kafka/pull/13145 `Values.parseString()` handles empty strings and returns the same `SchemaAndValue` [[0]](https://github.com/apache/kafka/blob/trunk/connect/api/src/main/java/org/apache/kafka/connect/data/Values.java

[GitHub] [kafka] mimaison commented on a diff in pull request #13094: MINOR: Various cleanups in client tests

mimaison commented on code in PR #13094: URL: https://github.com/apache/kafka/pull/13094#discussion_r1083436604 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/RecordAccumulatorTest.java: ## @@ -331,17 +331,15 @@ public void testStressfulSituation() throws E

[GitHub] [kafka] mimaison merged pull request #13080: KAFKA-14575: Move ClusterTool to tools module

mimaison merged PR #13080: URL: https://github.com/apache/kafka/pull/13080 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] hgeraldino opened a new pull request, #13146: KIP-991 Add deletedConnector flag when stopping tasks

hgeraldino opened a new pull request, #13146: URL: https://github.com/apache/kafka/pull/13146 Companion PR for [KIP-901: Add flag connectorDeleted flag when stopping task](https://cwiki.apache.org/confluence/display/KAFKA/KIP-901%3A+Add+flag+connectorDeleted+flag+when+stopping+task) This

[GitHub] [kafka] ableegoldman commented on pull request #13141: MINOR: fix flaky integrations tests by using 60s default timeout for startup

ableegoldman commented on PR #13141: URL: https://github.com/apache/kafka/pull/13141#issuecomment-1399645849 test failures are unrelated, will merge to trunk and cherrypick to 3.4 to help stabilize the release -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] ableegoldman merged pull request #13141: MINOR: fix flaky integrations tests by using 60s default timeout for startup

ableegoldman merged PR #13141: URL: https://github.com/apache/kafka/pull/13141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.ap

[GitHub] [kafka] sgn2607 commented on a diff in pull request #11748: KAFKA-12635: Don't emit checkpoints for partitions without any offset…

sgn2607 commented on code in PR #11748: URL: https://github.com/apache/kafka/pull/11748#discussion_r1083630069 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorCheckpointTask.java: ## @@ -169,6 +172,7 @@ public String version() { return listConsumer

[GitHub] [kafka] sgn2607 commented on a diff in pull request #11748: KAFKA-12635: Don't emit checkpoints for partitions without any offset…

sgn2607 commented on code in PR #11748: URL: https://github.com/apache/kafka/pull/11748#discussion_r1083630069 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorCheckpointTask.java: ## @@ -169,6 +172,7 @@ public String version() { return listConsumer

[GitHub] [kafka] sgn2607 commented on a diff in pull request #11748: KAFKA-12635: Don't emit checkpoints for partitions without any offset…

sgn2607 commented on code in PR #11748: URL: https://github.com/apache/kafka/pull/11748#discussion_r1083630069 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorCheckpointTask.java: ## @@ -169,6 +172,7 @@ public String version() { return listConsumer

[GitHub] [kafka] abbccdda closed pull request #8725: KAFKA-9608: Transaction Event Simulation Test

abbccdda closed pull request #8725: KAFKA-9608: Transaction Event Simulation Test URL: https://github.com/apache/kafka/pull/8725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [kafka] fvaleri commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

fvaleri commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1083709872 ## core/src/main/scala/kafka/utils/ToolsUtils.scala: ## @@ -64,4 +65,18 @@ object ToolsUtils { println(s"%-${maxLengthOfDisplayName}s : $specifier".format(met

[GitHub] [kafka] clolov commented on pull request #12818: KAFKA-14133: Replace EasyMock with Mockito in streams tests

clolov commented on PR #12818: URL: https://github.com/apache/kafka/pull/12818#issuecomment-1399926748 Thank you very much for the review and merge! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [kafka] vamossagar12 commented on a diff in pull request #13127: Kafka 14586: Moving StreamResetter to tools

vamossagar12 commented on code in PR #13127: URL: https://github.com/apache/kafka/pull/13127#discussion_r1083769752 ## build.gradle: ## @@ -1757,6 +1757,7 @@ project(':tools') { archivesBaseName = "kafka-tools" dependencies { +implementation project(':core') Review

[GitHub] [kafka] vamossagar12 commented on pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

vamossagar12 commented on PR #13131: URL: https://github.com/apache/kafka/pull/13131#issuecomment-1399976298 Thanks @fvaleri . LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [kafka] ableegoldman commented on pull request #13147: MINOR: temporarily disable the 'false' parameter of SmokeTestDriverIntegrationTest

ableegoldman commented on PR #13147: URL: https://github.com/apache/kafka/pull/13147#issuecomment-1400043590 cc @lucasbru @guozhangwang @mjsax -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] ableegoldman opened a new pull request, #13147: MINOR: temporarily disable the 'false' parameter of SmokeTestDriverIntegrationTest

ableegoldman opened a new pull request, #13147: URL: https://github.com/apache/kafka/pull/13147 Need to get a clean build for 3.4 and this test has been extremely flaky. I'm looking into the failure as well, and want to pinpoint whether it's the `true` build that's broken or it's the parame

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

divijvaidya commented on code in PR #13135: URL: https://github.com/apache/kafka/pull/13135#discussion_r1083847838 ## clients/src/main/java/org/apache/kafka/common/compress/ZstdFactory.java: ## @@ -26,21 +26,25 @@ import org.apache.kafka.common.utils.ByteBufferInputStream; imp

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

divijvaidya commented on code in PR #13135: URL: https://github.com/apache/kafka/pull/13135#discussion_r1083831592 ## clients/src/main/java/org/apache/kafka/common/record/DefaultRecordBatch.java: ## @@ -273,20 +272,32 @@ public int partitionLeaderEpoch() { public DataInputS

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

divijvaidya commented on code in PR #13135: URL: https://github.com/apache/kafka/pull/13135#discussion_r1083852068 ## clients/src/main/java/org/apache/kafka/common/record/CompressionType.java: ## @@ -47,6 +47,11 @@ public OutputStream wrapForOutput(ByteBufferOutputStream buffer

[GitHub] [kafka] clolov commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

clolov commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1083906152 ## server-common/src/main/java/org/apache/kafka/server/util/CommandLineUtils.java: ## @@ -0,0 +1,201 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

[GitHub] [kafka] clolov commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

clolov commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1083909966 ## core/src/main/scala/kafka/tools/ConsoleConsumer.scala: ## @@ -352,9 +353,11 @@ object ConsoleConsumer extends Logging { } else if (options.has(offsetOpt))

[GitHub] [kafka] clolov commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

clolov commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1083920980 ## server-common/src/main/java/org/apache/kafka/server/util/CommandLineUtils.java: ## @@ -0,0 +1,201 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

[GitHub] [kafka] clolov commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

clolov commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1083920980 ## server-common/src/main/java/org/apache/kafka/server/util/CommandLineUtils.java: ## @@ -0,0 +1,201 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

[GitHub] [kafka] clolov commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

clolov commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1083920980 ## server-common/src/main/java/org/apache/kafka/server/util/CommandLineUtils.java: ## @@ -0,0 +1,201 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

[GitHub] [kafka] clolov commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

clolov commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1083906152 ## server-common/src/main/java/org/apache/kafka/server/util/CommandLineUtils.java: ## @@ -0,0 +1,201 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

[GitHub] [kafka] mimaison commented on a diff in pull request #11748: KAFKA-12635: Don't emit checkpoints for partitions without any offset…

mimaison commented on code in PR #11748: URL: https://github.com/apache/kafka/pull/11748#discussion_r1083928100 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorCheckpointTask.java: ## @@ -169,6 +172,7 @@ public String version() { return listConsume

[GitHub] [kafka] clolov commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

clolov commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1083908982 ## core/src/main/scala/kafka/admin/ZkSecurityMigrator.scala: ## @@ -100,6 +101,7 @@ object ZkSecurityMigrator extends Logging { false case _ =>

[GitHub] [kafka] clolov commented on pull request #10826: KAFKA-7632: Support Compression Level

clolov commented on PR #10826: URL: https://github.com/apache/kafka/pull/10826#issuecomment-1400189273 Hello @dongjinleekr! What is the current state of this pull request? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [kafka] mimaison merged pull request #13059: MINOR: KafkaConfig should not expose internal config when queried for non-internal values

mimaison merged PR #13059: URL: https://github.com/apache/kafka/pull/13059 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] mimaison merged pull request #13094: MINOR: Various cleanups in client tests

mimaison merged PR #13094: URL: https://github.com/apache/kafka/pull/13094 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] fvaleri commented on a diff in pull request #13131: KAFKA-14628: Move CommandLineUtils and CommandDefaultOptions shared classes

fvaleri commented on code in PR #13131: URL: https://github.com/apache/kafka/pull/13131#discussion_r1084030517 ## core/src/main/scala/kafka/admin/ZkSecurityMigrator.scala: ## @@ -100,6 +101,7 @@ object ZkSecurityMigrator extends Logging { false case _ =>

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

divijvaidya commented on code in PR #13135: URL: https://github.com/apache/kafka/pull/13135#discussion_r1084093810 ## clients/src/main/java/org/apache/kafka/common/compress/ZstdFactory.java: ## @@ -62,10 +68,11 @@ public void release(ByteBuffer buffer) { }

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

divijvaidya commented on code in PR #13135: URL: https://github.com/apache/kafka/pull/13135#discussion_r1084097512 ## clients/src/main/java/org/apache/kafka/common/compress/ZstdFactory.java: ## @@ -62,10 +68,11 @@ public void release(ByteBuffer buffer) { }

[GitHub] [kafka] divijvaidya commented on pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

divijvaidya commented on PR #13135: URL: https://github.com/apache/kafka/pull/13135#issuecomment-1400407941 TODO (will update PR in a short while) - 1. Add benchmark for case when batch contains single 10 byte message 2. Test consumer performance -- This is an automated mess

[GitHub] [kafka] C0urante commented on pull request #12561: KAFKA-12495: Exponential backoff retry to prevent rebalance storms when worker joins after revoking rebalance

C0urante commented on PR #12561: URL: https://github.com/apache/kafka/pull/12561#issuecomment-1400491466 @vamossagar12 I don't think it's necessary to call this out anywhere unless it's caused unexpected issues with our users. The intention behind the exponential backoff is to avoid rebalan

[GitHub] [kafka] C0urante opened a new pull request, #13148: KAFKA-14645: Use plugin classloader when retrieving connector plugin config definitions

C0urante opened a new pull request, #13148: URL: https://github.com/apache/kafka/pull/13148 [Jira](https://issues.apache.org/jira/browse/KAFKA-14645) If we don't switch to the classloader of a plugin before loading its `ConfigDef`, then classloading bugs can appear for, e.g., properti

[GitHub] [kafka] C0urante commented on a diff in pull request #13148: KAFKA-14645: Use plugin classloader when retrieving connector plugin config definitions

C0urante commented on code in PR #13148: URL: https://github.com/apache/kafka/pull/13148#discussion_r1084283540 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractHerder.java: ## @@ -821,13 +827,14 @@ public List connectorPluginConfig(String pluginName) {

[GitHub] [kafka] ijuma commented on a diff in pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

ijuma commented on code in PR #13135: URL: https://github.com/apache/kafka/pull/13135#discussion_r1084344515 ## clients/src/main/java/org/apache/kafka/common/record/DefaultRecordBatch.java: ## @@ -273,20 +272,32 @@ public int partitionLeaderEpoch() { public DataInputStream

[GitHub] [kafka] ijuma commented on a diff in pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

ijuma commented on code in PR #13135: URL: https://github.com/apache/kafka/pull/13135#discussion_r1084344515 ## clients/src/main/java/org/apache/kafka/common/record/DefaultRecordBatch.java: ## @@ -273,20 +272,32 @@ public int partitionLeaderEpoch() { public DataInputStream

[GitHub] [kafka] ijuma commented on a diff in pull request #12781: KAFKA-14132; Replace EasyMock with Mockito in KafkaBasedLogTest

ijuma commented on code in PR #12781: URL: https://github.com/apache/kafka/pull/12781#discussion_r1084369349 ## connect/runtime/src/main/java/org/apache/kafka/connect/util/KafkaBasedLog.java: ## @@ -98,7 +98,7 @@ private Optional> producer; private TopicAdmin admin;

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13126: KAFKA-14491: [1/N] Add segment value format for RocksDB versioned store

vcrfxia commented on code in PR #13126: URL: https://github.com/apache/kafka/pull/13126#discussion_r1082994727 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java: ## @@ -0,0 +1,523 @@ +/* + * Licensed to the Apache S

[GitHub] [kafka] erichaagdev opened a new pull request, #13149: Remove unnecessary asCollection causing eager dependency resolution

erichaagdev opened a new pull request, #13149: URL: https://github.com/apache/kafka/pull/13149 The call to `asCollection()` causes several configurations to be resolved eagerly, and potentially unnecessarily. Dropping `asCollection()` ensures the configurations are only resolved when they a

[GitHub] [kafka] cmccabe commented on a diff in pull request #13116: KAFKA-14351: Controller Mutation Quota for KRaft

cmccabe commented on code in PR #13116: URL: https://github.com/apache/kafka/pull/13116#discussion_r1084419497 ## core/src/main/scala/kafka/server/ControllerApis.scala: ## @@ -392,14 +394,36 @@ class ControllerApis(val requestChannel: RequestChannel, val describableTopicNam

[GitHub] [kafka] cmccabe commented on pull request #13116: KAFKA-14351: Controller Mutation Quota for KRaft

cmccabe commented on PR #13116: URL: https://github.com/apache/kafka/pull/13116#issuecomment-1400816232 > Also note that I didn't implement the full logic -- QuorumController has additional checks that could in fact cause a specific topic request to be rejected (e.g. explicit assignments no

[GitHub] [kafka] C0urante commented on a diff in pull request #13144: KAFKA-14463 Invoke of ConnectorClientConfigOverridePolicy#close

C0urante commented on code in PR #13144: URL: https://github.com/apache/kafka/pull/13144#discussion_r1084433485 ## connect/runtime/src/test/java/org/apache/kafka/connect/connector/policy/BaseConnectorClientConfigOverridePolicyTest.java: ## @@ -27,7 +27,7 @@ public abstract cl

[GitHub] [kafka] C0urante merged pull request #12802: KAFKA-14311: Connect Worker clean shutdown does not cleanly stop connectors/tasks

C0urante merged PR #12802: URL: https://github.com/apache/kafka/pull/12802 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13137: KAFKA-15086: Intra-cluster communication for Mirror Maker 2

gharris1727 commented on code in PR #13137: URL: https://github.com/apache/kafka/pull/13137#discussion_r1084435808 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -228,12 +229,17 @@ public class DistributedHerder extend

[GitHub] [kafka] C0urante commented on a diff in pull request #13137: KAFKA-15086: Intra-cluster communication for Mirror Maker 2

C0urante commented on code in PR #13137: URL: https://github.com/apache/kafka/pull/13137#discussion_r1084460160 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -228,12 +229,17 @@ public class DistributedHerder extends A

[GitHub] [kafka] guozhangwang commented on a diff in pull request #13138: MINOR: Small cleanups in refactored consumer implementation

guozhangwang commented on code in PR #13138: URL: https://github.com/apache/kafka/pull/13138#discussion_r1084463267 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CoordinatorRequestManager.java: ## @@ -149,11 +156,14 @@ private void onSuccessfulResponse(fin

[GitHub] [kafka] philipnee commented on pull request #13125: KAFKA-14626: Kafka Consumer Coordinator does not cleanup all metrics after shutdown

philipnee commented on PR #13125: URL: https://github.com/apache/kafka/pull/13125#issuecomment-1400882904 hey @cmccabe - would you have time to take a look at this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13126: KAFKA-14491: [1/N] Add segment value format for RocksDB versioned store

vcrfxia commented on code in PR #13126: URL: https://github.com/apache/kafka/pull/13126#discussion_r1084484753 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java: ## @@ -0,0 +1,523 @@ +/* + * Licensed to the Apache S

[GitHub] [kafka] nizhikov closed pull request #12574: KAFKA-13908 Rethrow ExecutionException to preserve original cause

nizhikov closed pull request #12574: KAFKA-13908 Rethrow ExecutionException to preserve original cause URL: https://github.com/apache/kafka/pull/12574 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13126: KAFKA-14491: [1/N] Add segment value format for RocksDB versioned store

vcrfxia commented on code in PR #13126: URL: https://github.com/apache/kafka/pull/13126#discussion_r1084488305 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java: ## @@ -0,0 +1,523 @@ +/* + * Licensed to the Apache S

[GitHub] [kafka] vcrfxia commented on pull request #13126: KAFKA-14491: [1/N] Add segment value format for RocksDB versioned store

vcrfxia commented on PR #13126: URL: https://github.com/apache/kafka/pull/13126#issuecomment-1400904752 Thanks for your reviews, @mjsax ! I pushed another commit just now to incorporate your latest suggestions (javadocs changes only) and responded inline to the main points of discussion. Th

[GitHub] [kafka] nizhikov commented on a diff in pull request #13144: KAFKA-14463 Invoke of ConnectorClientConfigOverridePolicy#close

nizhikov commented on code in PR #13144: URL: https://github.com/apache/kafka/pull/13144#discussion_r1084491140 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/distributed/DistributedHerderTest.java: ## @@ -227,9 +227,14 @@ public class DistributedHerderTest {

[GitHub] [kafka] nizhikov commented on pull request #13144: KAFKA-14463 Invoke of ConnectorClientConfigOverridePolicy#close

nizhikov commented on PR #13144: URL: https://github.com/apache/kafka/pull/13144#issuecomment-1400908925 @C0urante Thanks for the review. I've applied your suggestions. Please, take a look one more time. -- This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13120: MINOR: Connect Javadocs improvements

gharris1727 commented on code in PR #13120: URL: https://github.com/apache/kafka/pull/13120#discussion_r1084469223 ## connect/api/src/main/java/org/apache/kafka/connect/connector/policy/ConnectorClientConfigOverridePolicy.java: ## @@ -23,25 +23,25 @@ import java.util.List; /

  1   2   3   4   5   6   7   8   9   10   >