[jira] [Created] (KAFKA-14431) TopologyTestDriver instantiation raised NPE in JUnit5

2022-12-01 Thread Mehrdad Karami (Jira)
Mehrdad Karami created KAFKA-14431: -- Summary: TopologyTestDriver instantiation raised NPE in JUnit5 Key: KAFKA-14431 URL: https://issues.apache.org/jira/browse/KAFKA-14431 Project: Kafka Iss

[GitHub] [kafka] vamossagar12 commented on a diff in pull request #12802: KAFKA-14311: Connect Worker clean shutdown does not cleanly stop connectors/tasks

2022-12-01 Thread GitBox
vamossagar12 commented on code in PR #12802: URL: https://github.com/apache/kafka/pull/12802#discussion_r1036857522 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -1645,6 +1646,8 @@ private void startAndStop(Collection

[jira] [Resolved] (KAFKA-14431) TopologyTestDriver instantiation raised NPE in JUnit5

2022-12-01 Thread Mehrdad Karami (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mehrdad Karami resolved KAFKA-14431. Resolution: Fixed Saw already fixed in truck branch > TopologyTestDriver instantiation ra

[GitHub] [kafka] omkreddy merged pull request #12924: try-finally around super call in http.py

2022-12-01 Thread GitBox
omkreddy merged PR #12924: URL: https://github.com/apache/kafka/pull/12924 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] mimaison commented on a diff in pull request #12913: KIP-866 Allow ZK brokers to register in KRaft

2022-12-01 Thread GitBox
mimaison commented on code in PR #12913: URL: https://github.com/apache/kafka/pull/12913#discussion_r1036885976 ## core/src/main/scala/kafka/server/KafkaServer.scala: ## @@ -490,25 +575,19 @@ class KafkaServer( private def initZkClient(time: Time): Unit = { info(s"Conn

[GitHub] [kafka] mimaison commented on pull request #12895: KAFKA-13881: Add package description javadocs

2022-12-01 Thread GitBox
mimaison commented on PR #12895: URL: https://github.com/apache/kafka/pull/12895#issuecomment-1333509571 Thanks for the update! I wonder if it would be better to have Connect and Streams in separate PRs especially as it's likely going to be different people reviewing them. -- This is an

[GitHub] [kafka] showuon commented on pull request #12847: KAFKA-14367; Add `SyncGroup` to the new `GroupCoordinator` interface

2022-12-01 Thread GitBox
showuon commented on PR #12847: URL: https://github.com/apache/kafka/pull/12847#issuecomment-1333528295 Sorry, was busy recently. I'll review it this week or next week. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [kafka] lucasbru commented on pull request #12925: Revert "[KAFKA-14324] Upgrade RocksDB to 7.1.2 (#12809)"

2022-12-01 Thread GitBox
lucasbru commented on PR #12925: URL: https://github.com/apache/kafka/pull/12925#issuecomment-1333536861 @cadonna Could you retrigger this? It is just one configuration (JDK 8 and Scala 2.12) that has lots of tests timing out, so that looks more like a node issue to me. -- This is an aut

[jira] [Commented] (KAFKA-10582) Mirror Maker 2 not replicating new topics until restart

2022-12-01 Thread sandeep (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641826#comment-17641826 ] sandeep commented on KAFKA-10582: -- I'm having similar issue with latest version of kaf

[GitHub] [kafka] cadonna opened a new pull request, #12931: MINOR: Fix log message used in version probing system test

2022-12-01 Thread GitBox
cadonna opened a new pull request, #12931: URL: https://github.com/apache/kafka/pull/12931 PR #12684 introduced a better format for timestamps in log messages. Unfortunately, we missed that one of the modified log messages is used by a system test for validation. This PR adapts the s

[GitHub] [kafka] cadonna commented on pull request #12931: MINOR: Fix log message used in version probing system test

2022-12-01 Thread GitBox
cadonna commented on PR #12931: URL: https://github.com/apache/kafka/pull/12931#issuecomment-1333724608 Locally the system test `kafkatest.tests.streams.streams_upgrade_test.StreamsUpgradeTest.test_version_probing_upgrade` passes with this PR. -- This is an automated message from the Apa

[GitHub] [kafka] cadonna commented on pull request #12925: Revert "[KAFKA-14324] Upgrade RocksDB to 7.1.2 (#12809)"

2022-12-01 Thread GitBox
cadonna commented on PR #12925: URL: https://github.com/apache/kafka/pull/12925#issuecomment-1333730811 I've already retriggered it once yesterday but I can retrigger again now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] ijuma commented on pull request #12912: MINOR: Fix some compiler warnings

2022-12-01 Thread GitBox
ijuma commented on PR #12912: URL: https://github.com/apache/kafka/pull/12912#issuecomment-1333773568 I don't think most of these are actual compiler warnings are they? They seem like lint type warnings. It would be good to clarify in the PR description. -- This is an automated message fr

[GitHub] [kafka] rajinisivaram commented on a diff in pull request #12914: KAFKA-14352: Rack-aware consumer partition assignment (KIP-881)

2022-12-01 Thread GitBox
rajinisivaram commented on code in PR #12914: URL: https://github.com/apache/kafka/pull/12914#discussion_r1037112256 ## clients/src/main/resources/common/message/ConsumerProtocolSubscription.json: ## @@ -21,7 +21,10 @@ // The current implementation assumes that future version

[GitHub] [kafka] ijuma commented on pull request #12280: MINOR: Reuse gradle daemon for scala compilation by default

2022-12-01 Thread GitBox
ijuma commented on PR #12280: URL: https://github.com/apache/kafka/pull/12280#issuecomment-1333777333 There is quite a lot of unit test flakiness, but it doesn't seem related to this PR (it's happening in trunk too). -- This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] ijuma merged pull request #12280: MINOR: Reuse gradle daemon for scala compilation by default

2022-12-01 Thread GitBox
ijuma merged PR #12280: URL: https://github.com/apache/kafka/pull/12280 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] reta commented on a diff in pull request #12898: optimize: -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT

2022-12-01 Thread GitBox
reta commented on code in PR #12898: URL: https://github.com/apache/kafka/pull/12898#discussion_r1037130900 ## bin/kafka-run-class.sh: ## @@ -212,7 +212,7 @@ fi # JMX port to use if [ $JMX_PORT ]; then - KAFKA_JMX_OPTS="$KAFKA_JMX_OPTS -Dcom.sun.management.jmxremote.port=

[jira] [Updated] (KAFKA-14425) The Kafka protocol should support nullable structs

2022-12-01 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot updated KAFKA-14425: Summary: The Kafka protocol should support nullable structs (was: Automated protocol should suppo

[GitHub] [kafka] C0urante commented on a diff in pull request #12898: optimize: -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT

2022-12-01 Thread GitBox
C0urante commented on code in PR #12898: URL: https://github.com/apache/kafka/pull/12898#discussion_r1037153804 ## bin/kafka-run-class.sh: ## @@ -212,7 +212,7 @@ fi # JMX port to use if [ $JMX_PORT ]; then - KAFKA_JMX_OPTS="$KAFKA_JMX_OPTS -Dcom.sun.management.jmxremote.p

[GitHub] [kafka] dajac opened a new pull request, #12932: [WIP] KAFKA-14425; The Kafka protocol should support nullable structs

2022-12-01 Thread GitBox
dajac opened a new pull request, #12932: URL: https://github.com/apache/kafka/pull/12932 WIP ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgr

[jira] [Commented] (KAFKA-14366) Kafka consumer rebalance issue, offsets points back to very old committed offset

2022-12-01 Thread Chetan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641951#comment-17641951 ] Chetan commented on KAFKA-14366: Hi [~pnee], # We have created a custom Listener from

[GitHub] [kafka] a364176773 commented on a diff in pull request #12898: optimize: -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT

2022-12-01 Thread GitBox
a364176773 commented on code in PR #12898: URL: https://github.com/apache/kafka/pull/12898#discussion_r1037170338 ## bin/kafka-run-class.sh: ## @@ -212,7 +212,7 @@ fi # JMX port to use if [ $JMX_PORT ]; then - KAFKA_JMX_OPTS="$KAFKA_JMX_OPTS -Dcom.sun.management.jmxremote

[GitHub] [kafka] dajac commented on a diff in pull request #12848: KAFKA-14367; Add `Heartbeat` to the new `GroupCoordinator` interface

2022-12-01 Thread GitBox
dajac commented on code in PR #12848: URL: https://github.com/apache/kafka/pull/12848#discussion_r1037177201 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -1764,42 +1764,38 @@ class KafkaApis(val requestChannel: RequestChannel, }) } - def handleHeartbeatR

[GitHub] [kafka] dajac commented on a diff in pull request #12848: KAFKA-14367; Add `Heartbeat` to the new `GroupCoordinator` interface

2022-12-01 Thread GitBox
dajac commented on code in PR #12848: URL: https://github.com/apache/kafka/pull/12848#discussion_r1037178096 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -1764,42 +1764,38 @@ class KafkaApis(val requestChannel: RequestChannel, }) } - def handleHeartbeatR

[GitHub] [kafka] divijvaidya commented on pull request #12821: KAFKA-14132: Replace PowerMock and EasyMock with Mockito in streams tests

2022-12-01 Thread GitBox
divijvaidya commented on PR #12821: URL: https://github.com/apache/kafka/pull/12821#issuecomment-1333861957 @clolov please fix the checkstyle failures in this PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [kafka] andymg3 commented on a diff in pull request #12892: KAFKA-14386: Change ReplicaPlacer place method to return a class instead of a list of list of integers

2022-12-01 Thread GitBox
andymg3 commented on code in PR #12892: URL: https://github.com/apache/kafka/pull/12892#discussion_r1037170517 ## metadata/src/main/java/org/apache/kafka/metadata/placement/StripedReplicaPlacer.java: ## @@ -440,6 +441,6 @@ public List> place( for (int partition = 0; par

[jira] [Updated] (KAFKA-14133) Remaining EasyMock to Mockito tests

2022-12-01 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-14133: - Description: {color:#de350b}There are tests which use both PowerMock and EasyMock. I have put t

[GitHub] [kafka] mimaison merged pull request #12343: MINOR: Update unit/integration tests to work with the IBM Semeru JDK

2022-12-01 Thread GitBox
mimaison merged PR #12343: URL: https://github.com/apache/kafka/pull/12343 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] divijvaidya opened a new pull request, #12933: MINOR: Optimize metric recording when quota check not required

2022-12-01 Thread GitBox
divijvaidya opened a new pull request, #12933: URL: https://github.com/apache/kafka/pull/12933 In the selector code path, we record some values for some sensors which do not have any metric associated with them that requires quota. Yet, a redundant check for quotas is made which consumes ~0

[GitHub] [kafka] andymg3 commented on pull request #12892: KAFKA-14386: Change ReplicaPlacer place method to return a class instead of a list of list of integers

2022-12-01 Thread GitBox
andymg3 commented on PR #12892: URL: https://github.com/apache/kafka/pull/12892#issuecomment-1333944951 > Thanks for the changes @andymg3 . > > Should we also improve the `PartitionRegistration` type to use a `PartitionAssignment` instead of `int[] replicas`? Thanks for the rev

[GitHub] [kafka] a364176773 commented on pull request #12898: optimize: -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT

2022-12-01 Thread GitBox
a364176773 commented on PR #12898: URL: https://github.com/apache/kafka/pull/12898#issuecomment-1333973530 > @a364176773 Thank you for your understanding, and thank you for the fix! LGTM > > Tested locally via Docker container; with this fix, all that was necessary was to set the `$J

[GitHub] [kafka] lucasbru commented on pull request #12925: Revert "[KAFKA-14324] Upgrade RocksDB to 7.1.2 (#12809)"

2022-12-01 Thread GitBox
lucasbru commented on PR #12925: URL: https://github.com/apache/kafka/pull/12925#issuecomment-1333981153 Tests look fine, now. However, in the meantime I have found the cause of the leak (or at least one) in trunk. We could consider waiting for this fix to go in and not revert this.

[jira] [Created] (KAFKA-14432) RocksDBStore relies on finalizers to not leak memory

2022-12-01 Thread Lucas Brutschy (Jira)
Lucas Brutschy created KAFKA-14432: -- Summary: RocksDBStore relies on finalizers to not leak memory Key: KAFKA-14432 URL: https://issues.apache.org/jira/browse/KAFKA-14432 Project: Kafka Issu

[jira] [Updated] (KAFKA-14432) RocksDBStore relies on finalizers to not leak memory

2022-12-01 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy updated KAFKA-14432: --- Description: Relying on finalizers in RocksDB has been deprecated for a long time, and star

[GitHub] [kafka] C0urante commented on a diff in pull request #12366: KAFKA-14021: Implement new KIP-618 APIs in MirrorSourceConnector

2022-12-01 Thread GitBox
C0urante commented on code in PR #12366: URL: https://github.com/apache/kafka/pull/12366#discussion_r1037300881 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorSourceConnector.java: ## @@ -185,11 +193,52 @@ public ConfigDef config() { return Mirror

[GitHub] [kafka] C0urante commented on pull request #12366: KAFKA-14021: Implement new KIP-618 APIs in MirrorSourceConnector

2022-12-01 Thread GitBox
C0urante commented on PR #12366: URL: https://github.com/apache/kafka/pull/12366#issuecomment-1333998703 Thanks @mimaison. I've rebased and addressed your comments -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [kafka] C0urante commented on pull request #12355: KAFKA-14017: Implement new KIP-618 APIs in FileStreamSourceConnector

2022-12-01 Thread GitBox
C0urante commented on PR #12355: URL: https://github.com/apache/kafka/pull/12355#issuecomment-1334019842 Thanks Mickael -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [kafka] C0urante merged pull request #12355: KAFKA-14017: Implement new KIP-618 APIs in FileStreamSourceConnector

2022-12-01 Thread GitBox
C0urante merged PR #12355: URL: https://github.com/apache/kafka/pull/12355 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] gharris1727 commented on a diff in pull request #12802: KAFKA-14311: Connect Worker clean shutdown does not cleanly stop connectors/tasks

2022-12-01 Thread GitBox
gharris1727 commented on code in PR #12802: URL: https://github.com/apache/kafka/pull/12802#discussion_r1037326575 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -1645,6 +1646,8 @@ private void startAndStop(Collection>

[GitHub] [kafka] lucasbru opened a new pull request, #12935: KAFKA-14432: RocksDBStore relies on finalizers to not leak memory

2022-12-01 Thread GitBox
lucasbru opened a new pull request, #12935: URL: https://github.com/apache/kafka/pull/12935 RocksDBStore relied on finalizers to not leak memory (and leaked memory after the upgrade to RocksDB 7). The problem was that every call to `options.statistics` creates a new wrapper object th

[jira] [Commented] (KAFKA-14366) Kafka consumer rebalance issue, offsets points back to very old committed offset

2022-12-01 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642049#comment-17642049 ] Philip Nee commented on KAFKA-14366: Hey [~Chetu]  Any specific reason why you are

[GitHub] [kafka] lucasbru commented on pull request #12355: KAFKA-14017: Implement new KIP-618 APIs in FileStreamSourceConnector

2022-12-01 Thread GitBox
lucasbru commented on PR #12355: URL: https://github.com/apache/kafka/pull/12355#issuecomment-1334068328 Are you sure this change compiles? Doesn't build for me -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [kafka] hachikuji commented on pull request #12837: MINOR: extract jointly owned parts of BrokerServer and ControllerServer

2022-12-01 Thread GitBox
hachikuji commented on PR #12837: URL: https://github.com/apache/kafka/pull/12837#issuecomment-1334076245 The test failures in the latest build are showing an NPE: ``` java.lang.NullPointerException: Cannot invoke "kafka.raft.KafkaRaftManager.client()" because the return value of "kaf

[GitHub] [kafka] ijuma commented on a diff in pull request #12837: MINOR: extract jointly owned parts of BrokerServer and ControllerServer

2022-12-01 Thread GitBox
ijuma commented on code in PR #12837: URL: https://github.com/apache/kafka/pull/12837#discussion_r1037363599 ## core/src/main/scala/kafka/server/JointServer.scala: ## @@ -0,0 +1,246 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor li

[GitHub] [kafka] cmccabe merged pull request #12930: MINOR: Implement EventQueue#size and EventQueue#empty

2022-12-01 Thread GitBox
cmccabe merged PR #12930: URL: https://github.com/apache/kafka/pull/12930 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] gharris1727 opened a new pull request, #12936: KAFKA-13881: Add Streams package infos

2022-12-01 Thread GitBox
gharris1727 opened a new pull request, #12936: URL: https://github.com/apache/kafka/pull/12936 Signed-off-by: Greg Harris Split off from #12895 to target only Streams packages. There are currently no descriptions of packages in the javadocs, which can be intimidating to new us

[GitHub] [kafka] gharris1727 opened a new pull request, #12937: KAFKA-13881: Add Connect package infos

2022-12-01 Thread GitBox
gharris1727 opened a new pull request, #12937: URL: https://github.com/apache/kafka/pull/12937 Signed-off-by: Greg Harris Split off from #12895 to target only Connect packages. There are currently no descriptions of packages in the javadocs, which can be intimidating to new us

[jira] [Updated] (KAFKA-14432) RocksDBStore relies on finalizers to not leak memory

2022-12-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14432: Component/s: streams > RocksDBStore relies on finalizers to not leak memory >

[GitHub] [kafka] gharris1727 commented on pull request #12895: KAFKA-13881: Add package description javadocs

2022-12-01 Thread GitBox
gharris1727 commented on PR #12895: URL: https://github.com/apache/kafka/pull/12895#issuecomment-1334084826 @mimaison I had some wishful thinking that putting everything together would mean that we're more motivated to get everything in, but I agree that the packages are logically separate

[GitHub] [kafka] alexec commented on pull request #12697: KAFKA-2526 : fix deserializer/serializer from properties for console …

2022-12-01 Thread GitBox
alexec commented on PR #12697: URL: https://github.com/apache/kafka/pull/12697#issuecomment-1334085360 We ended up building our own tool. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [kafka] alexec closed pull request #12697: KAFKA-2526 : fix deserializer/serializer from properties for console …

2022-12-01 Thread GitBox
alexec closed pull request #12697: KAFKA-2526 : fix deserializer/serializer from properties for console … URL: https://github.com/apache/kafka/pull/12697 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] mimaison opened a new pull request, #12938: MINOR: Fix compilation issue in FileStreamSourceConnector

2022-12-01 Thread GitBox
mimaison opened a new pull request, #12938: URL: https://github.com/apache/kafka/pull/12938 Fix compilation failure introduced in https://github.com/apache/kafka/pull/12355 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verif

[jira] [Commented] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2022-12-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642054#comment-17642054 ] Matthias J. Sax commented on KAFKA-12679: - [~lbrutschy] Can you link the corresp

[GitHub] [kafka] cmccabe commented on pull request #12837: MINOR: extract jointly owned parts of BrokerServer and ControllerServer

2022-12-01 Thread GitBox
cmccabe commented on PR #12837: URL: https://github.com/apache/kafka/pull/12837#issuecomment-1334090428 > The test failures in the latest build are showing an NPE: Yes. raftManager is initialized in `JointServer#start` so `BrokerServer#raftManager` needs to be a function, not a val.

[GitHub] [kafka] cmccabe commented on a diff in pull request #12837: MINOR: extract jointly owned parts of BrokerServer and ControllerServer

2022-12-01 Thread GitBox
cmccabe commented on code in PR #12837: URL: https://github.com/apache/kafka/pull/12837#discussion_r1037371708 ## core/src/main/scala/kafka/server/JointServer.scala: ## @@ -0,0 +1,246 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor

[GitHub] [kafka] mimaison commented on a diff in pull request #12366: KAFKA-14021: Implement new KIP-618 APIs in MirrorSourceConnector

2022-12-01 Thread GitBox
mimaison commented on code in PR #12366: URL: https://github.com/apache/kafka/pull/12366#discussion_r1037373964 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorSourceConnector.java: ## @@ -185,11 +193,52 @@ public ConfigDef config() { return Mirror

[GitHub] [kafka] mjsax commented on pull request #12934: add docs table entries for blocked time metrics

2022-12-01 Thread GitBox
mjsax commented on PR #12934: URL: https://github.com/apache/kafka/pull/12934#issuecomment-1334095663 Thanks for the PR -- what about the corresponding consumer/producer metrics that got added? cf https://cwiki.apache.org/confluence/display/KAFKA/KIP-761%3A+Add+Total+Blocked+Time+Met

[jira] [Commented] (KAFKA-12635) Mirrormaker 2 offset sync is incorrect if the target partition is empty

2022-12-01 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642058#comment-17642058 ] Mickael Maison commented on KAFKA-12635: [~chetan-pandey] Have these negative of

[jira] [Created] (KAFKA-14433) Clear all yammer metrics when test harnesses clean up

2022-12-01 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-14433: Summary: Clear all yammer metrics when test harnesses clean up Key: KAFKA-14433 URL: https://issues.apache.org/jira/browse/KAFKA-14433 Project: Kafka Issue T

[GitHub] [kafka] vamossagar12 commented on a diff in pull request #12802: KAFKA-14311: Connect Worker clean shutdown does not cleanly stop connectors/tasks

2022-12-01 Thread GitBox
vamossagar12 commented on code in PR #12802: URL: https://github.com/apache/kafka/pull/12802#discussion_r1037397259 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -1645,6 +1646,8 @@ private void startAndStop(Collection

[GitHub] [kafka] dajac opened a new pull request, #12939: ignore

2022-12-01 Thread GitBox
dajac opened a new pull request, #12939: URL: https://github.com/apache/kafka/pull/12939 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing s

[jira] [Updated] (KAFKA-14299) Benchmark and stabilize state updater

2022-12-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14299: Component/s: streams > Benchmark and stabilize state updater > ---

[jira] [Updated] (KAFKA-14412) Transactional semantics for StateStores

2022-12-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14412: Labels: kip (was: ) > Transactional semantics for StateStores > -

[GitHub] [kafka] jsancio merged pull request #12929: MINOR; Use Option to check for null values

2022-12-01 Thread GitBox
jsancio merged PR #12929: URL: https://github.com/apache/kafka/pull/12929 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] divijvaidya opened a new pull request, #12940: MINOR: Remove lock contention while adding sensors

2022-12-01 Thread GitBox
divijvaidya opened a new pull request, #12940: URL: https://github.com/apache/kafka/pull/12940 When we want to add a sensor to `Metrics.java`, we synchronize the entire method. However both, `sensors` and `childrenSensors` are concurrent hash maps and full method synchronization is not requ

[GitHub] [kafka] hachikuji commented on a diff in pull request #12837: MINOR: extract jointly owned parts of BrokerServer and ControllerServer

2022-12-01 Thread GitBox
hachikuji commented on code in PR #12837: URL: https://github.com/apache/kafka/pull/12837#discussion_r1037431796 ## core/src/main/scala/kafka/server/BrokerServer.scala: ## @@ -66,14 +67,14 @@ class BrokerSnapshotWriterBuilder(raftClient: RaftClient[ApiMessageAndVersion]) * A

[GitHub] [kafka] C0urante commented on pull request #12355: KAFKA-14017: Implement new KIP-618 APIs in FileStreamSourceConnector

2022-12-01 Thread GitBox
C0urante commented on PR #12355: URL: https://github.com/apache/kafka/pull/12355#issuecomment-1334169937 @lucasbru There's a compilation failure introduced by a change that landed on trunk between when the last commit for this PR was filed, and when it was merged. There is a fix PR already

[GitHub] [kafka] C0urante commented on pull request #12898: optimize: -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT

2022-12-01 Thread GitBox
C0urante commented on PR #12898: URL: https://github.com/apache/kafka/pull/12898#issuecomment-1334172537 Update: trunk is currently failing to build with a compilation error; as soon as a fix is merged and stabilized, I'll re-run the CI build on this one and merge if there don't appear to b

[GitHub] [kafka] hachikuji commented on a diff in pull request #12915: KAFKA-14417: Producer doesn't handle REQUEST_TIMED_OUT for InitProducerIdRequest, treats as fatal error

2022-12-01 Thread GitBox
hachikuji commented on code in PR #12915: URL: https://github.com/apache/kafka/pull/12915#discussion_r1036589456 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/TransactionManagerTest.java: ## @@ -1678,6 +1678,39 @@ public void testMultipleAddPartitionsPerFo

[GitHub] [kafka] C0urante opened a new pull request, #12941: KAFKA-13709: Add docs for exactly-once support in Connect

2022-12-01 Thread GitBox
C0urante opened a new pull request, #12941: URL: https://github.com/apache/kafka/pull/12941 [Jira](https://issues.apache.org/jira/browse/KAFKA-13709) Rebased on top of https://github.com/apache/kafka/pull/12938 to fix recently-introduced compilation failures on trunk. -- This is an

[jira] [Updated] (KAFKA-13709) Document exactly-once support for source connectors

2022-12-01 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-13709: -- Description: Add documentation for the support for exactly-once source connectors introduced

[jira] [Updated] (KAFKA-13709) Document exactly-once support for source connectors

2022-12-01 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-13709: -- Description: Add documentation for the support for exactly-once source connectors introduced

[jira] [Commented] (KAFKA-13709) Document exactly-once support for source connectors

2022-12-01 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642115#comment-17642115 ] Chris Egerton commented on KAFKA-13709: --- I've decided to keep the rollback instruc

[GitHub] [kafka] C0urante commented on pull request #12941: KAFKA-13709: Add docs for exactly-once support in Connect

2022-12-01 Thread GitBox
C0urante commented on PR #12941: URL: https://github.com/apache/kafka/pull/12941#issuecomment-1334192798 @joel-hamill would you be interested in taking a look at this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[jira] [Updated] (KAFKA-14433) Clear all yammer metrics when test harnesses clean up

2022-12-01 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-14433: - Attachment: image-2022-12-01-13-53-57-886.png > Clear all yammer metrics when test harnesses cle

[jira] [Updated] (KAFKA-14433) Clear all yammer metrics when test harnesses clean up

2022-12-01 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-14433: - Attachment: image-2022-12-01-13-55-35-488.png > Clear all yammer metrics when test harnesses cle

[jira] [Commented] (KAFKA-14433) Clear all yammer metrics when test harnesses clean up

2022-12-01 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642123#comment-17642123 ] David Arthur commented on KAFKA-14433: -- I did some investigation on this. Running {

[GitHub] [kafka] dajac merged pull request #12848: KAFKA-14367; Add `Heartbeat` to the new `GroupCoordinator` interface

2022-12-01 Thread GitBox
dajac merged PR #12848: URL: https://github.com/apache/kafka/pull/12848 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] jsancio commented on a diff in pull request #12892: KAFKA-14386: Change ReplicaPlacer place method to return a class instead of a list of list of integers

2022-12-01 Thread GitBox
jsancio commented on code in PR #12892: URL: https://github.com/apache/kafka/pull/12892#discussion_r1037457258 ## metadata/src/test/java/org/apache/kafka/controller/ClusterControlManagerTest.java: ## @@ -407,13 +405,8 @@ public void testPlaceReplicas(int numUsableBrokers) throws

[GitHub] [kafka] andymg3 commented on a diff in pull request #12892: KAFKA-14386: Change ReplicaPlacer place method to return a class instead of a list of list of integers

2022-12-01 Thread GitBox
andymg3 commented on code in PR #12892: URL: https://github.com/apache/kafka/pull/12892#discussion_r1037473300 ## metadata/src/test/java/org/apache/kafka/controller/ClusterControlManagerTest.java: ## @@ -407,13 +405,8 @@ public void testPlaceReplicas(int numUsableBrokers) throws

[GitHub] [kafka] andymg3 commented on a diff in pull request #12892: KAFKA-14386: Change ReplicaPlacer place method to return a class instead of a list of list of integers

2022-12-01 Thread GitBox
andymg3 commented on code in PR #12892: URL: https://github.com/apache/kafka/pull/12892#discussion_r1037475316 ## metadata/src/test/java/org/apache/kafka/controller/ClusterControlManagerTest.java: ## @@ -407,13 +405,8 @@ public void testPlaceReplicas(int numUsableBrokers) throws

[GitHub] [kafka] andymg3 commented on a diff in pull request #12892: KAFKA-14386: Change ReplicaPlacer place method to return a class instead of a list of list of integers

2022-12-01 Thread GitBox
andymg3 commented on code in PR #12892: URL: https://github.com/apache/kafka/pull/12892#discussion_r1037475316 ## metadata/src/test/java/org/apache/kafka/controller/ClusterControlManagerTest.java: ## @@ -407,13 +405,8 @@ public void testPlaceReplicas(int numUsableBrokers) throws

[GitHub] [kafka] andymg3 commented on a diff in pull request #12892: KAFKA-14386: Change ReplicaPlacer place method to return a class instead of a list of list of integers

2022-12-01 Thread GitBox
andymg3 commented on code in PR #12892: URL: https://github.com/apache/kafka/pull/12892#discussion_r1037477576 ## metadata/src/test/java/org/apache/kafka/metadata/placement/StripedReplicaPlacerTest.java: ## @@ -95,12 +94,7 @@ private List> place( PlacementSpec placement

[GitHub] [kafka] andymg3 commented on a diff in pull request #12892: KAFKA-14386: Change ReplicaPlacer place method to return a class instead of a list of list of integers

2022-12-01 Thread GitBox
andymg3 commented on code in PR #12892: URL: https://github.com/apache/kafka/pull/12892#discussion_r1037481276 ## metadata/src/main/java/org/apache/kafka/metadata/placement/TopicAssignment.java: ## @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

[jira] [Commented] (KAFKA-14374) Kafka streams losing messages in State Store during first launch of app

2022-12-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17642135#comment-17642135 ] Matthias J. Sax commented on KAFKA-14374: - Thanks for the details. – It would be

[GitHub] [kafka] andymg3 commented on a diff in pull request #12892: KAFKA-14386: Change ReplicaPlacer place method to return a class instead of a list of list of integers

2022-12-01 Thread GitBox
andymg3 commented on code in PR #12892: URL: https://github.com/apache/kafka/pull/12892#discussion_r1037485087 ## metadata/src/test/java/org/apache/kafka/metadata/placement/PartitionAssignmentTest.java: ## @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] philipnee commented on a diff in pull request #12862: Consumer refator find coordinator

2022-12-01 Thread GitBox
philipnee commented on code in PR #12862: URL: https://github.com/apache/kafka/pull/12862#discussion_r1037502209 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/DefaultBackgroundThread.java: ## @@ -145,19 +166,46 @@ void runOnce() { this.infligh

[GitHub] [kafka] mumrah opened a new pull request, #12942: KAFKA-14433 Clear Yammer metrics in QuorumTestHarness#tearDown

2022-12-01 Thread GitBox
mumrah opened a new pull request, #12942: URL: https://github.com/apache/kafka/pull/12942 The anonymous functions we create in classes like Partition will end up getting held by the singleton Yammer metrics registry. ``` newGauge("UnderReplicated", () => if (isUnderReplicated) 1

[GitHub] [kafka] mumrah commented on pull request #12942: KAFKA-14433 Clear Yammer metrics in QuorumTestHarness#tearDown

2022-12-01 Thread GitBox
mumrah commented on PR #12942: URL: https://github.com/apache/kafka/pull/12942#issuecomment-1334405072 Prior to this patch, I was seeing several KafkaRaftClient (and KafkaRaftManager) objects hanging around in memory after running the test suite for 30 minutes or so. https://user-im

[GitHub] [kafka] hachikuji commented on a diff in pull request #12942: KAFKA-14433 Clear Yammer metrics in QuorumTestHarness#tearDown

2022-12-01 Thread GitBox
hachikuji commented on code in PR #12942: URL: https://github.com/apache/kafka/pull/12942#discussion_r1037533828 ## core/src/test/scala/integration/kafka/server/QuorumTestHarness.scala: ## @@ -386,6 +386,7 @@ abstract class QuorumTestHarness extends Logging { def tearDown():

[GitHub] [kafka] gmcquillan commented on pull request #8066: KAFKA-4090: Validate SSL connection in client

2022-12-01 Thread GitBox
gmcquillan commented on PR #8066: URL: https://github.com/apache/kafka/pull/8066#issuecomment-1334412501 Hey -- has there been any movement on this? Is there anything I can do to help push this forward? This is a constant source of frustration for my teams. -- This is an automated message

[GitHub] [kafka] mumrah commented on a diff in pull request #12928: KAFKA-14304 Add RPC changes, records, and config from KIP-866

2022-12-01 Thread GitBox
mumrah commented on code in PR #12928: URL: https://github.com/apache/kafka/pull/12928#discussion_r1037537646 ## metadata/src/main/java/org/apache/kafka/migration/MigrationRecoveryState.java: ## @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] pprovenzano commented on a diff in pull request #12896: KAFKA-14398: Update EndToEndAuthorizationTest to test both ZK and KRAFT quorum servers

2022-12-01 Thread GitBox
pprovenzano commented on code in PR #12896: URL: https://github.com/apache/kafka/pull/12896#discussion_r1037538569 ## core/src/test/scala/integration/kafka/api/EndToEndAuthorizationTest.scala: ## @@ -36,15 +37,33 @@ import org.apache.kafka.common.errors.{GroupAuthorizationExcep

[GitHub] [kafka] hachikuji commented on a diff in pull request #12942: KAFKA-14433 Clear Yammer metrics in QuorumTestHarness#tearDown

2022-12-01 Thread GitBox
hachikuji commented on code in PR #12942: URL: https://github.com/apache/kafka/pull/12942#discussion_r1037533828 ## core/src/test/scala/integration/kafka/server/QuorumTestHarness.scala: ## @@ -386,6 +386,7 @@ abstract class QuorumTestHarness extends Logging { def tearDown():

[GitHub] [kafka] mumrah commented on a diff in pull request #12928: KAFKA-14304 Add RPC changes, records, and config from KIP-866

2022-12-01 Thread GitBox
mumrah commented on code in PR #12928: URL: https://github.com/apache/kafka/pull/12928#discussion_r1037541558 ## metadata/src/main/java/org/apache/kafka/migration/MigrationRecoveryState.java: ## @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] hachikuji commented on a diff in pull request #12942: KAFKA-14433 Clear Yammer metrics in QuorumTestHarness#tearDown

2022-12-01 Thread GitBox
hachikuji commented on code in PR #12942: URL: https://github.com/apache/kafka/pull/12942#discussion_r1037543086 ## core/src/test/scala/integration/kafka/server/QuorumTestHarness.scala: ## @@ -386,6 +386,7 @@ abstract class QuorumTestHarness extends Logging { def tearDown():

[GitHub] [kafka] vvcephei commented on a diff in pull request #12803: KAFKA-13602: Adding ability to multicast records.

2022-12-01 Thread GitBox
vvcephei commented on code in PR #12803: URL: https://github.com/apache/kafka/pull/12803#discussion_r1037554572 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsMetadataState.java: ## @@ -459,12 +461,23 @@ private List rebuildMetadataForSingleTopolog

[GitHub] [kafka] hachikuji merged pull request #12938: MINOR: Fix compilation issue in FileStreamSourceConnector

2022-12-01 Thread GitBox
hachikuji merged PR #12938: URL: https://github.com/apache/kafka/pull/12938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apach

[GitHub] [kafka] jsancio commented on a diff in pull request #12892: KAFKA-14386: Change ReplicaPlacer place method to return a class instead of a list of list of integers

2022-12-01 Thread GitBox
jsancio commented on code in PR #12892: URL: https://github.com/apache/kafka/pull/12892#discussion_r1037576847 ## metadata/src/test/java/org/apache/kafka/metadata/placement/PartitionAssignmentTest.java: ## @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] jsancio commented on a diff in pull request #12892: KAFKA-14386: Change ReplicaPlacer place method to return a class instead of a list of list of integers

2022-12-01 Thread GitBox
jsancio commented on code in PR #12892: URL: https://github.com/apache/kafka/pull/12892#discussion_r1037576847 ## metadata/src/test/java/org/apache/kafka/metadata/placement/PartitionAssignmentTest.java: ## @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

  1   2   >