[GitHub] [kafka] anatasiavela commented on pull request #13078: KAFKA-13999: Add ProducerCount metrics (KIP-847)

2023-01-30 Thread via GitHub
anatasiavela commented on PR #13078: URL: https://github.com/apache/kafka/pull/13078#issuecomment-1409535758 @divijvaidya Since this PR is strictly focusing on the metric, I think it's reasonable for that change to be done in a separate PR. what do you think? -- This is an automated messa

[GitHub] [kafka] cmccabe commented on a diff in pull request #13116: KAFKA-14351: Controller Mutation Quota for KRaft

2023-01-30 Thread via GitHub
cmccabe commented on code in PR #13116: URL: https://github.com/apache/kafka/pull/13116#discussion_r1091311692 ## core/src/main/scala/kafka/server/ControllerApis.scala: ## @@ -322,15 +325,37 @@ class ControllerApis(val requestChannel: RequestChannel, } }

[GitHub] [kafka] beardt commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-01-30 Thread via GitHub
beardt commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1091360138 ## clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java: ## @@ -476,14 +480,34 @@ public List getConfiguredInstances(List classNames, Class t, M

[GitHub] [kafka] beardt commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-01-30 Thread via GitHub
beardt commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1091360138 ## clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java: ## @@ -476,14 +480,34 @@ public List getConfiguredInstances(List classNames, Class t, M

[GitHub] [kafka] beardt commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-01-30 Thread via GitHub
beardt commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1091360138 ## clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java: ## @@ -476,14 +480,34 @@ public List getConfiguredInstances(List classNames, Class t, M

[GitHub] [kafka] beardt commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-01-30 Thread via GitHub
beardt commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1091360138 ## clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java: ## @@ -476,14 +480,34 @@ public List getConfiguredInstances(List classNames, Class t, M

[GitHub] [kafka] Nayana-ibm commented on pull request #11929: MINOR: s390x Stage

2023-01-30 Thread via GitHub
Nayana-ibm commented on PR #11929: URL: https://github.com/apache/kafka/pull/11929#issuecomment-1409780907 Any update on this please? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] cmccabe merged pull request #13159: KAFKA-14656 Send UMR first during ZK migration

2023-01-30 Thread via GitHub
cmccabe merged PR #13159: URL: https://github.com/apache/kafka/pull/13159 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] Hangleton commented on pull request #13169: KAFKA-14658: Do not open broker ports until we are ready to accept traffic

2023-01-30 Thread via GitHub
Hangleton commented on PR #13169: URL: https://github.com/apache/kafka/pull/13169#issuecomment-1409850803 Taking a look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [kafka] mjsax merged pull request #13175: KAFAK-14660: Fix divide-by-zero vulnerability

2023-01-30 Thread via GitHub
mjsax merged PR #13175: URL: https://github.com/apache/kafka/pull/13175 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] mjsax commented on pull request #13143: KAFKA-14491: [3/N] Add logical key value segments

2023-01-31 Thread via GitHub
mjsax commented on PR #13143: URL: https://github.com/apache/kafka/pull/13143#issuecomment-1409929859 Merged the other PR -- can you rebase this one? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] mjsax merged pull request #13142: KAFKA-14491: [2/N] Refactor RocksDB store open iterator management

2023-01-31 Thread via GitHub
mjsax merged PR #13142: URL: https://github.com/apache/kafka/pull/13142 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] urbandan commented on a diff in pull request #13137: KAFKA-15086: Intra-cluster communication for Mirror Maker 2

2023-01-31 Thread via GitHub
urbandan commented on code in PR #13137: URL: https://github.com/apache/kafka/pull/13137#discussion_r1091577828 ## connect/mirror/src/test/java/org/apache/kafka/connect/mirror/integration/DedicatedMirrorIntegrationTest.java: ## @@ -0,0 +1,228 @@ +/* + * Licensed to the Apache So

[GitHub] [kafka] urbandan commented on a diff in pull request #13163: KAFKA-14653: MirrorMakerConfig using raw properties instead of post-r…

2023-01-31 Thread via GitHub
urbandan commented on code in PR #13163: URL: https://github.com/apache/kafka/pull/13163#discussion_r1091581987 ## clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java: ## @@ -246,13 +246,22 @@ public Map originals(Map configOverrides) { */ public

[GitHub] [kafka] Hangleton commented on a diff in pull request #13176: MINOR: some ZK migration code cleanups.

2023-01-31 Thread via GitHub
Hangleton commented on code in PR #13176: URL: https://github.com/apache/kafka/pull/13176#discussion_r1091621015 ## core/src/main/scala/kafka/migration/MigrationPropagator.scala: ## @@ -79,6 +81,18 @@ class MigrationPropagator( _image = image } + /** + * A very expe

[GitHub] [kafka] vamossagar12 commented on pull request #13095: KAFKA-14580: Moving EndToEndLatency from core to tools module

2023-01-31 Thread via GitHub
vamossagar12 commented on PR #13095: URL: https://github.com/apache/kafka/pull/13095#issuecomment-141003 @mimaison , I updated the system test to point to the new class. That one place seemed to be the only one relevant in this case. -- This is an automated message from the Apache Gi

[GitHub] [kafka] fvaleri commented on pull request #13095: KAFKA-14580: Moving EndToEndLatency from core to tools module

2023-01-31 Thread via GitHub
fvaleri commented on PR #13095: URL: https://github.com/apache/kafka/pull/13095#issuecomment-1410076642 > @mimaison , I updated the system test to point to the new class. That one place seemed to be the only one relevant in this case. Do you have a test run output that shows it works

[GitHub] [kafka] lucasbru commented on pull request #13164: MINOR: Fix scaladoc warnings

2023-01-31 Thread via GitHub
lucasbru commented on PR #13164: URL: https://github.com/apache/kafka/pull/13164#issuecomment-1410168606 @mjsax can we merge this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [kafka] dpcollins-google commented on a diff in pull request #13162: fix: replace an inefficient loop in kafka internals

2023-01-31 Thread via GitHub
dpcollins-google commented on code in PR #13162: URL: https://github.com/apache/kafka/pull/13162#discussion_r1091834852 ## clients/src/main/java/org/apache/kafka/common/utils/Utils.java: ## @@ -1225,13 +1226,11 @@ public static long tryWriteTo(TransferableChannel destChannel,

[GitHub] [kafka] beardt commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-01-31 Thread via GitHub
beardt commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1092017609 ## clients/src/test/java/org/apache/kafka/test/MockConsumerInterceptor.java: ## @@ -55,6 +58,11 @@ public void configure(Map configs) { Object clientIdValue = c

[GitHub] [kafka] Cerchie commented on a diff in pull request #13161: Kafka 14128

2023-01-31 Thread via GitHub
Cerchie commented on code in PR #13161: URL: https://github.com/apache/kafka/pull/13161#discussion_r1092026440 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java: ## @@ -521,7 +524,7 @@ protected Map getNumPartitions(final Set topics

[GitHub] [kafka] Cerchie commented on a diff in pull request #13161: Kafka 14128

2023-01-31 Thread via GitHub
Cerchie commented on code in PR #13161: URL: https://github.com/apache/kafka/pull/13161#discussion_r1092027837 ## clients/src/main/java/org/apache/kafka/common/internals/KafkaFutureImpl.java: ## @@ -160,7 +160,7 @@ private void maybeThrowCancellationException(Throwable cause) {

[GitHub] [kafka] Cerchie commented on a diff in pull request #13161: Kafka 14128

2023-01-31 Thread via GitHub
Cerchie commented on code in PR #13161: URL: https://github.com/apache/kafka/pull/13161#discussion_r1092034974 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java: ## @@ -466,7 +469,10 @@ public Set makeReady(final Map topics) {

[GitHub] [kafka] Cerchie commented on a diff in pull request #13161: Kafka 14128

2023-01-31 Thread via GitHub
Cerchie commented on code in PR #13161: URL: https://github.com/apache/kafka/pull/13161#discussion_r1092035905 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java: ## @@ -538,6 +544,8 @@ protected Map getNumPartitions(final Set topics

[GitHub] [kafka] divijvaidya commented on pull request #13078: KAFKA-13999: Add ProducerCount metrics (KIP-847)

2023-01-31 Thread via GitHub
divijvaidya commented on PR #13078: URL: https://github.com/apache/kafka/pull/13078#issuecomment-1410523645 @anatasiavela the proposal (using concurrentHashMap for producers) impacts this change. Let me try to explain why. Currently you are using `producerIdCount` and updating it on e

[GitHub] [kafka] jeqo opened a new pull request, #13177: [KAFKA-14441] Benchmark performance impact of metrics library

2023-01-31 Thread via GitHub
jeqo opened a new pull request, #13177: URL: https://github.com/apache/kafka/pull/13177 https://issues.apache.org/jira/browse/KAFKA-14441 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status

[GitHub] [kafka] yashmayya commented on a diff in pull request #13120: MINOR: Connect Javadocs improvements

2023-01-31 Thread via GitHub
yashmayya commented on code in PR #13120: URL: https://github.com/apache/kafka/pull/13120#discussion_r1092188566 ## connect/api/src/main/java/org/apache/kafka/connect/connector/policy/ConnectorClientConfigOverridePolicy.java: ## @@ -23,25 +23,25 @@ import java.util.List; /**

[GitHub] [kafka] guozhangwang merged pull request #13164: MINOR: Fix scaladoc warnings

2023-01-31 Thread via GitHub
guozhangwang merged PR #13164: URL: https://github.com/apache/kafka/pull/13164 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.ap

[GitHub] [kafka] guozhangwang commented on pull request #13164: MINOR: Fix scaladoc warnings

2023-01-31 Thread via GitHub
guozhangwang commented on PR #13164: URL: https://github.com/apache/kafka/pull/13164#issuecomment-1410744946 LGTM, Merged to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [kafka] mimaison commented on a diff in pull request #13136: KAFKA-14582: Move JmxTool to tools

2023-01-31 Thread via GitHub
mimaison commented on code in PR #13136: URL: https://github.com/apache/kafka/pull/13136#discussion_r1088957590 ## checkstyle/import-control.xml: ## @@ -407,7 +407,8 @@ - + + Review Comment: Can we keep the trailing spaces like all other entries

[GitHub] [kafka] lucasbru commented on pull request #13025: KAFKA-14299: Fix pause and resume with state updater

2023-01-31 Thread via GitHub
lucasbru commented on PR #13025: URL: https://github.com/apache/kafka/pull/13025#issuecomment-1410798829 @guozhangwang also wanted to have a look (note that I didn't address Brunos comments yet) -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] C0urante commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-01-31 Thread via GitHub
C0urante commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1092133488 ## clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java: ## @@ -476,14 +479,22 @@ public List getConfiguredInstances(List classNames, Class t, M

[GitHub] [kafka] guozhangwang commented on pull request #13025: KAFKA-14299: Fix pause and resume with state updater

2023-01-31 Thread via GitHub
guozhangwang commented on PR #13025: URL: https://github.com/apache/kafka/pull/13025#issuecomment-1410815034 ack, will take a look asap. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [kafka] dajac commented on a diff in pull request #12972: KAFKA-14391; Add ConsumerGroupHeartbeat API

2023-01-31 Thread via GitHub
dajac commented on code in PR #12972: URL: https://github.com/apache/kafka/pull/12972#discussion_r1092319133 ## clients/src/main/resources/common/message/ConsumerGroupHeartbeatResponse.json: ## @@ -0,0 +1,70 @@ +// Licensed to the Apache Software Foundation (ASF) under one or mo

[GitHub] [kafka] dajac commented on a diff in pull request #12972: KAFKA-14391; Add ConsumerGroupHeartbeat API

2023-01-31 Thread via GitHub
dajac commented on code in PR #12972: URL: https://github.com/apache/kafka/pull/12972#discussion_r1092320492 ## core/src/main/scala/kafka/server/ApiVersionManager.scala: ## @@ -86,14 +100,7 @@ class DefaultApiVersionManager( finalizedFeatures.features.map(kv => (kv._1,

[GitHub] [kafka] gharris1727 opened a new pull request, #13178: KAFKA-12468, KAFKA-14663, KAFKA-12566: Fix MM2 causing negative downstream lag

2023-01-31 Thread via GitHub
gharris1727 opened a new pull request, #13178: URL: https://github.com/apache/kafka/pull/13178 This PR addresses three distinct but closely related issues: 1. [KAFKA-12468](https://issues.apache.org/jira/browse/KAFKA-12468) "Initial offsets are copied from source to target cluster" "M

[GitHub] [kafka] guozhangwang commented on pull request #13167: KAFKA-14650: Synchronize access to tasks inside task manager

2023-01-31 Thread via GitHub
guozhangwang commented on PR #13167: URL: https://github.com/apache/kafka/pull/13167#issuecomment-1410914374 @lucasbru You're right! I think this issue exists even before #12397. After thinking that a bit, along with forward looking that IQ would need to access the Tasks plus the Stat

[GitHub] [kafka] guozhangwang commented on a diff in pull request #12654: KAFKA-10575: Add onRestoreSuspsnded to StateRestoreListener

2023-01-31 Thread via GitHub
guozhangwang commented on code in PR #12654: URL: https://github.com/apache/kafka/pull/12654#discussion_r1092383808 ## streams/src/main/java/org/apache/kafka/streams/processor/StateRestoreListener.java: ## @@ -37,6 +37,9 @@ * These two interfaces serve different restoration pu

[GitHub] [kafka] guozhangwang commented on a diff in pull request #12654: KAFKA-10575: Add onRestoreSuspsnded to StateRestoreListener

2023-01-31 Thread via GitHub
guozhangwang commented on code in PR #12654: URL: https://github.com/apache/kafka/pull/12654#discussion_r1092384524 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/StoreChangelogReaderTest.java: ## @@ -197,6 +198,49 @@ public void shouldNotRegisterStoreWith

[GitHub] [kafka] guozhangwang commented on pull request #12654: KAFKA-10575: Add onRestoreSuspsnded to StateRestoreListener

2023-01-31 Thread via GitHub
guozhangwang commented on PR #12654: URL: https://github.com/apache/kafka/pull/12654#issuecomment-1410947553 > Why has this PR 392 commits? I created this branch / PR around Oct. 2022, and then I rebased on top of trunk. My local branch is in right place but it seems github cannot rec

[GitHub] [kafka] C0urante commented on a diff in pull request #12984: KAFKA-14455: Kafka Connect create and update REST APIs should surface failures while writing to the config topic

2023-01-31 Thread via GitHub
C0urante commented on code in PR #12984: URL: https://github.com/apache/kafka/pull/12984#discussion_r1092418826 ## connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaConfigBackingStore.java: ## @@ -711,9 +752,32 @@ KafkaBasedLog setupAndCreateKafkaBasedLog(Stri

[GitHub] [kafka] guozhangwang closed pull request #12654: KAFKA-10575: Add onRestoreSuspsnded to StateRestoreListener

2023-01-31 Thread via GitHub
guozhangwang closed pull request #12654: KAFKA-10575: Add onRestoreSuspsnded to StateRestoreListener URL: https://github.com/apache/kafka/pull/12654 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] guozhangwang opened a new pull request, #13179: KAFKA-10575: Add onRestoreSuspsnded to StateRestoreListener

2023-01-31 Thread via GitHub
guozhangwang opened a new pull request, #13179: URL: https://github.com/apache/kafka/pull/13179 1. Add the new API (default impl is empty) to StateRestoreListener. 2. Update related unit tests ### Committer Checklist (excluded from commit message) - [ ] Verify design and implemen

[GitHub] [kafka] guozhangwang commented on pull request #12654: KAFKA-10575: Add onRestoreSuspsnded to StateRestoreListener

2023-01-31 Thread via GitHub
guozhangwang commented on PR #12654: URL: https://github.com/apache/kafka/pull/12654#issuecomment-1410999341 @mjsax tried to resolve the long commit history but it seems not possible, so I created a new PR (with the same branch name) here: https://github.com/apache/kafka/pull/13179 it's rea

[GitHub] [kafka] mumrah commented on a diff in pull request #13169: KAFKA-14658: Do not open broker ports until we are ready to accept traffic

2023-01-31 Thread via GitHub
mumrah commented on code in PR #13169: URL: https://github.com/apache/kafka/pull/13169#discussion_r1092435602 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -585,23 +616,35 @@ private[kafka] abstract class Acceptor(val socketServer: SocketServer, private var

[GitHub] [kafka] cmccabe commented on a diff in pull request #13169: KAFKA-14658: Do not open broker ports until we are ready to accept traffic

2023-01-31 Thread via GitHub
cmccabe commented on code in PR #13169: URL: https://github.com/apache/kafka/pull/13169#discussion_r1092445406 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -585,23 +616,35 @@ private[kafka] abstract class Acceptor(val socketServer: SocketServer, private var

[GitHub] [kafka] cmccabe commented on a diff in pull request #13169: KAFKA-14658: Do not open broker ports until we are ready to accept traffic

2023-01-31 Thread via GitHub
cmccabe commented on code in PR #13169: URL: https://github.com/apache/kafka/pull/13169#discussion_r1092446989 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -573,7 +591,20 @@ private[kafka] abstract class Acceptor(val socketServer: SocketServer, private val

[GitHub] [kafka] pprovenzano commented on pull request #13114: KAFKA-14084: SCRAM support in KRaft.

2023-01-31 Thread via GitHub
pprovenzano commented on PR #13114: URL: https://github.com/apache/kafka/pull/13114#issuecomment-1411037660 I just updated AlterUserScramCredentialsRequestTest to work in KRaft after finding it while looking into the tests you wanted. It covers a lot of what you want testing ScramControlMan

[GitHub] [kafka] beardt commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-01-31 Thread via GitHub
beardt commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1092513735 ## clients/src/test/java/org/apache/kafka/common/config/AbstractConfigTest.java: ## @@ -54,6 +58,12 @@ public void testConfiguredInstances() { testInvalidInput

[GitHub] [kafka] beardt commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-01-31 Thread via GitHub
beardt commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1092516181 ## clients/src/test/java/org/apache/kafka/clients/consumer/KafkaConsumerTest.java: ## @@ -168,6 +169,7 @@ public class KafkaConsumerTest { private final int default

[GitHub] [kafka] vamossagar12 commented on pull request #13158: KAFKA-14647: Moving TopicFilter to server-common/utils

2023-01-31 Thread via GitHub
vamossagar12 commented on PR #13158: URL: https://github.com/apache/kafka/pull/13158#issuecomment-146347 Thanks @fvaleri , I made the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] mumrah opened a new pull request, #13180: Add a summarizer for the metadata migration

2023-01-31 Thread via GitHub
mumrah opened a new pull request, #13180: URL: https://github.com/apache/kafka/pull/13180 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[GitHub] [kafka] vvcephei commented on pull request #12879: KAFKA-14409: Clean ProcessorParameters from casting

2023-01-31 Thread via GitHub
vvcephei commented on PR #12879: URL: https://github.com/apache/kafka/pull/12879#issuecomment-1411120341 There was only one test failure, which was in an unrelated component: `[Build / JDK 8 and Scala 2.12 / kafka.server.DynamicBrokerReconfigurationTest.testTrustStoreAlter(String).quorum=kr

[GitHub] [kafka] vvcephei merged pull request #12879: KAFKA-14409: Clean ProcessorParameters from casting

2023-01-31 Thread via GitHub
vvcephei merged PR #12879: URL: https://github.com/apache/kafka/pull/12879 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] guozhangwang commented on a diff in pull request #13025: KAFKA-14299: Fix pause and resume with state updater

2023-01-31 Thread via GitHub
guozhangwang commented on code in PR #13025: URL: https://github.com/apache/kafka/pull/13025#discussion_r1092598270 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/ReadOnlyTask.java: ## @@ -190,7 +190,7 @@ public void clearTaskTimeout() { @Override

[GitHub] [kafka] C0urante opened a new pull request, #13181: KAFKA-14610: Publish Mirror Maker 2 offset syncs in task commit() method

2023-01-31 Thread via GitHub
C0urante opened a new pull request, #13181: URL: https://github.com/apache/kafka/pull/13181 The [Jira](https://issues.apache.org/jira/browse/KAFKA-14610) contains a more detailed description of the motivation for this change. To summarize, if there are bursty topic partitions, offset sync s

[GitHub] [kafka] C0urante commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-01-31 Thread via GitHub
C0urante commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1092632585 ## clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java: ## @@ -414,34 +414,51 @@ private T getConfiguredInstance(Object klass, Class t, Map T ge

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13181: KAFKA-14610: Publish Mirror Maker 2 offset syncs in task commit() method

2023-01-31 Thread via GitHub
gharris1727 commented on code in PR #13181: URL: https://github.com/apache/kafka/pull/13181#discussion_r1092643889 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorSourceTask.java: ## @@ -87,6 +90,7 @@ public MirrorSourceTask() {} @Override public

[GitHub] [kafka] gharris1727 commented on pull request #13178: KAFKA-12468, KAFKA-14663, KAFKA-12566: Fix MM2 causing negative downstream lag

2023-01-31 Thread via GitHub
gharris1727 commented on PR #13178: URL: https://github.com/apache/kafka/pull/13178#issuecomment-1411301405 After #13181 is merged, I'll rebase and remove my fairness patch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [kafka] beardt commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-01-31 Thread via GitHub
beardt commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1092652543 ## clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java: ## @@ -414,34 +414,51 @@ private T getConfiguredInstance(Object klass, Class t, Map T getC

[GitHub] [kafka] beardt commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-01-31 Thread via GitHub
beardt commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1092652543 ## clients/src/main/java/org/apache/kafka/common/config/AbstractConfig.java: ## @@ -414,34 +414,51 @@ private T getConfiguredInstance(Object klass, Class t, Map T getC

[GitHub] [kafka] fvaleri commented on a diff in pull request #13136: KAFKA-14582: Move JmxTool to tools

2023-02-01 Thread via GitHub
fvaleri commented on code in PR #13136: URL: https://github.com/apache/kafka/pull/13136#discussion_r1092890819 ## core/src/main/scala/kafka/admin/ConsumerGroupCommand.scala: ## @@ -1033,7 +1033,7 @@ object ConsumerGroupCommand extends Logging { val describeOpt = parser.acce

[GitHub] [kafka] fvaleri commented on a diff in pull request #13136: KAFKA-14582: Move JmxTool to tools

2023-02-01 Thread via GitHub
fvaleri commented on code in PR #13136: URL: https://github.com/apache/kafka/pull/13136#discussion_r1092895124 ## tools/src/test/java/org/apache/kafka/tools/JmxToolTest.java: ## @@ -0,0 +1,284 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

[GitHub] [kafka] fvaleri commented on a diff in pull request #13136: KAFKA-14582: Move JmxTool to tools

2023-02-01 Thread via GitHub
fvaleri commented on code in PR #13136: URL: https://github.com/apache/kafka/pull/13136#discussion_r1092895124 ## tools/src/test/java/org/apache/kafka/tools/JmxToolTest.java: ## @@ -0,0 +1,284 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

[GitHub] [kafka] fvaleri commented on a diff in pull request #13136: KAFKA-14582: Move JmxTool to tools

2023-02-01 Thread via GitHub
fvaleri commented on code in PR #13136: URL: https://github.com/apache/kafka/pull/13136#discussion_r1092919815 ## tools/src/main/java/org/apache/kafka/tools/JmxTool.java: ## @@ -0,0 +1,434 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [kafka] fvaleri commented on a diff in pull request #13136: KAFKA-14582: Move JmxTool to tools

2023-02-01 Thread via GitHub
fvaleri commented on code in PR #13136: URL: https://github.com/apache/kafka/pull/13136#discussion_r1092919815 ## tools/src/main/java/org/apache/kafka/tools/JmxTool.java: ## @@ -0,0 +1,434 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [kafka] fvaleri commented on a diff in pull request #13136: KAFKA-14582: Move JmxTool to tools

2023-02-01 Thread via GitHub
fvaleri commented on code in PR #13136: URL: https://github.com/apache/kafka/pull/13136#discussion_r1092919815 ## tools/src/main/java/org/apache/kafka/tools/JmxTool.java: ## @@ -0,0 +1,434 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [kafka] fvaleri commented on a diff in pull request #13136: KAFKA-14582: Move JmxTool to tools

2023-02-01 Thread via GitHub
fvaleri commented on code in PR #13136: URL: https://github.com/apache/kafka/pull/13136#discussion_r1092919815 ## tools/src/main/java/org/apache/kafka/tools/JmxTool.java: ## @@ -0,0 +1,434 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [kafka] fvaleri commented on a diff in pull request #13136: KAFKA-14582: Move JmxTool to tools

2023-02-01 Thread via GitHub
fvaleri commented on code in PR #13136: URL: https://github.com/apache/kafka/pull/13136#discussion_r1092919815 ## tools/src/main/java/org/apache/kafka/tools/JmxTool.java: ## @@ -0,0 +1,434 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[GitHub] [kafka] yashmayya commented on a diff in pull request #12984: KAFKA-14455: Kafka Connect create and update REST APIs should surface failures while writing to the config topic

2023-02-01 Thread via GitHub
yashmayya commented on code in PR #12984: URL: https://github.com/apache/kafka/pull/12984#discussion_r1093090686 ## connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaConfigBackingStore.java: ## @@ -711,9 +752,32 @@ KafkaBasedLog setupAndCreateKafkaBasedLog(Str

[GitHub] [kafka] C0urante commented on a diff in pull request #13181: KAFKA-14610: Publish Mirror Maker 2 offset syncs in task commit() method

2023-02-01 Thread via GitHub
C0urante commented on code in PR #13181: URL: https://github.com/apache/kafka/pull/13181#discussion_r1093398759 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorSourceTask.java: ## @@ -87,6 +90,7 @@ public MirrorSourceTask() {} @Override public voi

[GitHub] [kafka] beardt commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-02-01 Thread via GitHub
beardt commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1093403718 ## clients/src/test/java/org/apache/kafka/common/config/AbstractConfigTest.java: ## @@ -54,6 +58,12 @@ public void testConfiguredInstances() { testInvalidInput

[GitHub] [kafka] C0urante commented on a diff in pull request #13181: KAFKA-14610: Publish Mirror Maker 2 offset syncs in task commit() method

2023-02-01 Thread via GitHub
C0urante commented on code in PR #13181: URL: https://github.com/apache/kafka/pull/13181#discussion_r1093398759 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorSourceTask.java: ## @@ -87,6 +90,7 @@ public MirrorSourceTask() {} @Override public voi

[GitHub] [kafka] mimaison commented on a diff in pull request #13136: KAFKA-14582: Move JmxTool to tools

2023-02-01 Thread via GitHub
mimaison commented on code in PR #13136: URL: https://github.com/apache/kafka/pull/13136#discussion_r1093412947 ## tools/src/test/java/org/apache/kafka/tools/JmxToolTest.java: ## @@ -0,0 +1,282 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * con

[GitHub] [kafka] beardt commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-02-01 Thread via GitHub
beardt commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1093403718 ## clients/src/test/java/org/apache/kafka/common/config/AbstractConfigTest.java: ## @@ -54,6 +58,12 @@ public void testConfiguredInstances() { testInvalidInput

[GitHub] [kafka] beardt commented on a diff in pull request #13168: Kafka 14565: Interceptor Resource Leak

2023-02-01 Thread via GitHub
beardt commented on code in PR #13168: URL: https://github.com/apache/kafka/pull/13168#discussion_r1093403718 ## clients/src/test/java/org/apache/kafka/common/config/AbstractConfigTest.java: ## @@ -54,6 +58,12 @@ public void testConfiguredInstances() { testInvalidInput

[GitHub] [kafka] fvaleri commented on a diff in pull request #13136: KAFKA-14582: Move JmxTool to tools

2023-02-01 Thread via GitHub
fvaleri commented on code in PR #13136: URL: https://github.com/apache/kafka/pull/13136#discussion_r1093445449 ## tools/src/test/java/org/apache/kafka/tools/JmxToolTest.java: ## @@ -0,0 +1,282 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

[GitHub] [kafka] C0urante commented on a diff in pull request #12984: KAFKA-14455: Kafka Connect create and update REST APIs should surface failures while writing to the config topic

2023-02-01 Thread via GitHub
C0urante commented on code in PR #12984: URL: https://github.com/apache/kafka/pull/12984#discussion_r1093433981 ## connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaConfigBackingStore.java: ## @@ -711,9 +768,35 @@ KafkaBasedLog setupAndCreateKafkaBasedLog(Stri

[GitHub] [kafka] C0urante commented on a diff in pull request #12984: KAFKA-14455: Kafka Connect create and update REST APIs should surface failures while writing to the config topic

2023-02-01 Thread via GitHub
C0urante commented on code in PR #12984: URL: https://github.com/apache/kafka/pull/12984#discussion_r1093416934 ## clients/src/main/java/org/apache/kafka/common/utils/Timer.java: ## @@ -161,7 +161,7 @@ public long remainingMs() { /** * Get the current time in millisec

[GitHub] [kafka] fvaleri commented on a diff in pull request #13136: KAFKA-14582: Move JmxTool to tools

2023-02-01 Thread via GitHub
fvaleri commented on code in PR #13136: URL: https://github.com/apache/kafka/pull/13136#discussion_r1093501099 ## tools/src/test/java/org/apache/kafka/tools/JmxToolTest.java: ## @@ -0,0 +1,282 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * cont

[GitHub] [kafka] cmccabe merged pull request #13169: KAFKA-14658: Do not open broker ports until we are ready to accept traffic

2023-02-01 Thread via GitHub
cmccabe merged PR #13169: URL: https://github.com/apache/kafka/pull/13169 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] cmccabe commented on a diff in pull request #13176: MINOR: some ZK migration code cleanups.

2023-02-01 Thread via GitHub
cmccabe commented on code in PR #13176: URL: https://github.com/apache/kafka/pull/13176#discussion_r1093550518 ## core/src/main/scala/kafka/migration/MigrationPropagator.scala: ## @@ -79,6 +81,18 @@ class MigrationPropagator( _image = image } + /** + * A very expens

[GitHub] [kafka] yashmayya commented on a diff in pull request #12984: KAFKA-14455: Kafka Connect create and update REST APIs should surface failures while writing to the config topic

2023-02-01 Thread via GitHub
yashmayya commented on code in PR #12984: URL: https://github.com/apache/kafka/pull/12984#discussion_r1093606834 ## connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaConfigBackingStore.java: ## @@ -711,9 +768,35 @@ KafkaBasedLog setupAndCreateKafkaBasedLog(Str

[GitHub] [kafka] yashmayya commented on a diff in pull request #12984: KAFKA-14455: Kafka Connect create and update REST APIs should surface failures while writing to the config topic

2023-02-01 Thread via GitHub
yashmayya commented on code in PR #12984: URL: https://github.com/apache/kafka/pull/12984#discussion_r1093620557 ## clients/src/main/java/org/apache/kafka/common/utils/Timer.java: ## @@ -161,7 +161,7 @@ public long remainingMs() { /** * Get the current time in millise

[GitHub] [kafka] cmccabe commented on a diff in pull request #13180: MINOR: Add a summary of the metadata migration

2023-02-01 Thread via GitHub
cmccabe commented on code in PR #13180: URL: https://github.com/apache/kafka/pull/13180#discussion_r1093639638 ## metadata/src/main/java/org/apache/kafka/metadata/migration/MigrationSummary.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] cmccabe commented on a diff in pull request #13180: MINOR: Add a summary of the metadata migration

2023-02-01 Thread via GitHub
cmccabe commented on code in PR #13180: URL: https://github.com/apache/kafka/pull/13180#discussion_r1093640509 ## metadata/src/main/java/org/apache/kafka/metadata/migration/MigrationSummary.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] cmccabe commented on pull request #13180: MINOR: Add a summary of the metadata migration

2023-02-01 Thread via GitHub
cmccabe commented on PR #13180: URL: https://github.com/apache/kafka/pull/13180#issuecomment-1412624413 Thanks for the PR. WIth regard to the formatting changes, can we use the style we've been doing elsewhere in the scala code of: ``` functionName( parameterA: typeA, paramet

[GitHub] [kafka] junrao commented on a diff in pull request #13046: KAFKA-14551 Move/Rewrite LeaderEpochFileCache and its dependencies to the storage module.

2023-02-01 Thread via GitHub
junrao commented on code in PR #13046: URL: https://github.com/apache/kafka/pull/13046#discussion_r1093672007 ## storage/src/main/java/org/apache/kafka/storage/internals/checkpoint/LeaderEpochCheckpoint.java: ## @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] gharris1727 opened a new pull request, #13182: KAFKA-14649: Isolate failures during plugin path scanning to single plugin classes

2023-02-01 Thread via GitHub
gharris1727 opened a new pull request, #13182: URL: https://github.com/apache/kafka/pull/13182 Plugin path scanning can fail for multiple reasons: 1. A developer incorrectly implements a plugin (e.g. by not providing a default constructor) 2. A packaging mistake happens, and some requi

[GitHub] [kafka] mjsax merged pull request #13126: KAFKA-14491: [1/N] Add segment value format for RocksDB versioned store

2023-02-01 Thread via GitHub
mjsax merged PR #13126: URL: https://github.com/apache/kafka/pull/13126 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] mjsax commented on pull request #13126: KAFKA-14491: [1/N] Add segment value format for RocksDB versioned store

2023-02-01 Thread via GitHub
mjsax commented on PR #13126: URL: https://github.com/apache/kafka/pull/13126#issuecomment-1412741570 Most of my comments are educational or nits. I merged this PR already, and if you want to address anything (feel free to ignore), let's do a follow up PR. Wanted to merge this to unblock ot

[GitHub] [kafka] mumrah opened a new pull request, #13183: KAFKA-14668 Avoid unnecessary UMR during ZK migration

2023-02-01 Thread via GitHub
mumrah opened a new pull request, #13183: URL: https://github.com/apache/kafka/pull/13183 Prior to this patch, when we handle a new MetadataDelta during dual write mode in the KRaft controller, we always send UMR to the ZK brokers. This patch changes the behavior to only send UMR if the bro

[GitHub] [kafka] gharris1727 opened a new pull request, #13184: KAFKA-14671: Refactor PredicatedTransformation to not implement Transformation

2023-02-01 Thread via GitHub
gharris1727 opened a new pull request, #13184: URL: https://github.com/apache/kafka/pull/13184 Signed-off-by: Greg Harris For several reasons, the PredicatedTransformation should not be a Transformation, see ticket for details. This refactor just naively wraps all transformations

[GitHub] [kafka] mjsax commented on a diff in pull request #13143: KAFKA-14491: [3/N] Add logical key value segments

2023-02-01 Thread via GitHub
mjsax commented on code in PR #13143: URL: https://github.com/apache/kafka/pull/13143#discussion_r1093764504 ## streams/src/test/java/org/apache/kafka/streams/state/internals/LogicalKeyValueSegmentTest.java: ## @@ -0,0 +1,270 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] cmccabe commented on a diff in pull request #13183: KAFKA-14668 Avoid unnecessary UMR during ZK migration

2023-02-01 Thread via GitHub
cmccabe commented on code in PR #13183: URL: https://github.com/apache/kafka/pull/13183#discussion_r1093787328 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -106,6 +109,13 @@ public void shutdown() throws InterruptedException {

[GitHub] [kafka] cmccabe commented on a diff in pull request #13183: KAFKA-14668 Avoid unnecessary UMR during ZK migration

2023-02-01 Thread via GitHub
cmccabe commented on code in PR #13183: URL: https://github.com/apache/kafka/pull/13183#discussion_r1093789522 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -474,6 +472,7 @@ class MetadataChangeEvent extends MigrationEvent {

[GitHub] [kafka] cmccabe commented on a diff in pull request #13183: KAFKA-14668 Avoid unnecessary UMR during ZK migration

2023-02-01 Thread via GitHub
cmccabe commented on code in PR #13183: URL: https://github.com/apache/kafka/pull/13183#discussion_r1093791731 ## server-common/src/main/java/org/apache/kafka/queue/KafkaEventQueue.java: ## @@ -250,7 +250,7 @@ private void handleEvents() throws InterruptedException {

[GitHub] [kafka] cmccabe commented on a diff in pull request #13183: KAFKA-14668 Avoid unnecessary UMR during ZK migration

2023-02-01 Thread via GitHub
cmccabe commented on code in PR #13183: URL: https://github.com/apache/kafka/pull/13183#discussion_r1093792394 ## server-common/src/main/java/org/apache/kafka/queue/KafkaEventQueue.java: ## @@ -250,7 +250,7 @@ private void handleEvents() throws InterruptedException {

[GitHub] [kafka] mjsax commented on a diff in pull request #10747: KAFKA-12446: Call subtractor before adder if key is the same

2023-02-01 Thread via GitHub
mjsax commented on code in PR #10747: URL: https://github.com/apache/kafka/pull/10747#discussion_r1093813869 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/ChangedSerializer.java: ## @@ -46,33 +45,40 @@ public void setIfUnset(final SerdeGetter getter) {

[GitHub] [kafka] mjsax commented on a diff in pull request #10747: KAFKA-12446: Call subtractor before adder if key is the same

2023-02-01 Thread via GitHub
mjsax commented on code in PR #10747: URL: https://github.com/apache/kafka/pull/10747#discussion_r1093821934 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/ChangedSerializer.java: ## @@ -45,34 +47,85 @@ public void setIfUnset(final SerdeGetter getter) {

<    1   2   3   4   5   6   7   8   9   10   >