[GitHub] [kafka] showuon opened a new pull request, #12334: KAFKA-13981: Adding new record type handler in metadata shell

2022-09-13 Thread GitBox
showuon opened a new pull request, #12334: URL: https://github.com/apache/kafka/pull/12334 New records handled in this PR: `FEATURE_LEVEL_RECORD`: featureLevels/FEATURE_NAME -> level `BROKER_REGISTRATION_CHANGE_RECORD`: -> brokers/BROKER_ID/isFenced -> true/false -> brokers

[GitHub] [kafka] showuon closed pull request #12334: KAFKA-13981: Adding new record type handler in metadata shell

2022-09-13 Thread GitBox
showuon closed pull request #12334: KAFKA-13981: Adding new record type handler in metadata shell URL: https://github.com/apache/kafka/pull/12334 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Commented] (KAFKA-14152) Add logic to fence kraft brokers which have fallen behind in replication

2022-09-13 Thread dengziming (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603403#comment-17603403 ] dengziming commented on KAFKA-14152: > it must catch up to the current metadata befo

[GitHub] [kafka] showuon opened a new pull request, #12631: [WIP] MINOR: building

2022-09-13 Thread GitBox
showuon opened a new pull request, #12631: URL: https://github.com/apache/kafka/pull/12631 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[jira] [Updated] (KAFKA-14115) Password configs are logged in plaintext in KRaft

2022-09-13 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Bentley updated KAFKA-14115: Fix Version/s: 3.2.3 (was: 3.2.2) > Password configs are logged in plaintex

[jira] [Updated] (KAFKA-14136) AlterConfigs in KRaft does not generate records for unchanged values

2022-09-13 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Bentley updated KAFKA-14136: Fix Version/s: 3.2.3 (was: 3.2.2) > AlterConfigs in KRaft does not generate

[jira] [Updated] (KAFKA-14111) Dynamic config update fails for "password" configs in KRaft

2022-09-13 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Bentley updated KAFKA-14111: Fix Version/s: 3.2.3 (was: 3.2.2) > Dynamic config update fails for "passwo

[jira] [Updated] (KAFKA-14107) Upgrade Jetty for CVE fixes

2022-09-13 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Bentley updated KAFKA-14107: Fix Version/s: 3.2.3 (was: 3.2.2) > Upgrade Jetty for CVE fixes > -

[jira] [Updated] (KAFKA-14149) Broken DynamicBrokerReconfigurationTest in 3.2 branch

2022-09-13 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Bentley updated KAFKA-14149: Fix Version/s: 3.2.3 (was: 3.2.2) > Broken DynamicBrokerReconfigurationTest

[jira] [Updated] (KAFKA-14196) Duplicated consumption during rebalance, causing OffsetValidationTest to act flaky

2022-09-13 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Bentley updated KAFKA-14196: Fix Version/s: 3.2.3 (was: 3.2.2) > Duplicated consumption during rebalance

[GitHub] [kafka] showuon commented on pull request #12626: KAFKA-14208; Do not raise wakeup in consumer during asynchronous offset commits

2022-09-13 Thread GitBox
showuon commented on PR #12626: URL: https://github.com/apache/kafka/pull/12626#issuecomment-1245023780 Failed tests are unrelated: ``` org.apache.kafka.common.network.SslTransportLayerTest.[1] tlsProtocol=TLSv1.2, useInlinePem=false kafka.test.ClusterTestExtensionsTest.[1]

[GitHub] [kafka] showuon merged pull request #12626: KAFKA-14208; Do not raise wakeup in consumer during asynchronous offset commits

2022-09-13 Thread GitBox
showuon merged PR #12626: URL: https://github.com/apache/kafka/pull/12626 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Updated] (KAFKA-14208) KafkaConsumer#commitAsync throws unexpected WakeupException

2022-09-13 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-14208: -- Fix Version/s: 3.2.2 > KafkaConsumer#commitAsync throws unexpected WakeupException > -

[jira] [Resolved] (KAFKA-14208) KafkaConsumer#commitAsync throws unexpected WakeupException

2022-09-13 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-14208. --- Resolution: Fixed > KafkaConsumer#commitAsync throws unexpected WakeupException > --

[jira] [Resolved] (KAFKA-14196) Duplicated consumption during rebalance, causing OffsetValidationTest to act flaky

2022-09-13 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-14196. --- Resolution: Fixed > Duplicated consumption during rebalance, causing OffsetValidationTest to act >

[jira] [Updated] (KAFKA-14196) Duplicated consumption during rebalance, causing OffsetValidationTest to act flaky

2022-09-13 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-14196: -- Fix Version/s: 3.2.2 (was: 3.2.3) > Duplicated consumption during rebalance, ca

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-13 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r969339721 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -1933,11 +1941,81 @@ private Map topicPartitionTags(TopicPartition tp) {

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-13 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r969340204 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -1933,11 +1941,81 @@ private Map topicPartitionTags(TopicPartition tp) {

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-13 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r969341737 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -1933,11 +1941,81 @@ private Map topicPartitionTags(TopicPartition tp) {

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969349691 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -356,6 +381,95 @@ private void mergeDuplicateSourceNodes() {

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969351766 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -270,17 +277,12 @@ private void maybeAddNodeForOptimizationMetada

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969353364 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -305,6 +307,36 @@ public void buildAndOptimizeTopology(final bool

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969353780 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamSelfJoin.java: ## @@ -0,0 +1,172 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969358545 ## streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java: ## @@ -1644,6 +1668,29 @@ private Map getClientCustomProps() { return props; } +

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969371482 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImplJoin.java: ## @@ -181,13 +181,22 @@ public KStream join(final KStream lhs,

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969371482 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImplJoin.java: ## @@ -181,13 +181,22 @@ public KStream join(final KStream lhs,

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969391230 ## streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java: ## @@ -663,6 +663,12 @@ public class StreamsConfig extends AbstractConfig { public static fin

[GitHub] [kafka] cadonna commented on a diff in pull request #12449: KAFKA-12947: Replace EasyMock and PowerMock with Mockito for StreamsMetricsImplTest

2022-09-13 Thread GitBox
cadonna commented on code in PR #12449: URL: https://github.com/apache/kafka/pull/12449#discussion_r969376097 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/metrics/StreamsMetricsImplTest.java: ## @@ -287,7 +279,7 @@ public void shouldGetExistingThreadLeve

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969484570 ## streams/src/test/java/org/apache/kafka/streams/StreamsConfigTest.java: ## @@ -1263,6 +1263,37 @@ public void testInvalidSecurityProtocol() { assertTrue(ce.

[jira] [Updated] (KAFKA-14220) "partition-count" not getting updated after revocation of partitions in case of Incremental Co-operative rebalancing.

2022-09-13 Thread Pritam Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pritam Kumar updated KAFKA-14220: - Description: Issue: 
In case of the revocation of partitions, the updation of "partition count"

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969520183 ## streams/src/test/java/org/apache/kafka/streams/integration/SelfJoinUpgradeIntegrationTest.java: ## @@ -0,0 +1,277 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-13 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r969358545 ## streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java: ## @@ -1644,6 +1668,29 @@ private Map getClientCustomProps() { return props; } +

[GitHub] [kafka] divijvaidya commented on pull request #12620: KAFKA-14206: upgrade zookeeper version to 3.7.1

2022-09-13 Thread GitBox
divijvaidya commented on PR #12620: URL: https://github.com/apache/kafka/pull/12620#issuecomment-1245356187 I did some analysis on what has changed and here is my summary: ZK 3.7.1 [contains CVE fixes](https://zookeeper.apache.org/doc/r3.7.1/releasenotes.html) for: 1. Jackson-datab

[GitHub] [kafka] ijuma commented on a diff in pull request #12630: Doc: explain how to set old default partitioner

2022-09-13 Thread GitBox
ijuma commented on code in PR #12630: URL: https://github.com/apache/kafka/pull/12630#discussion_r969605501 ## docs/upgrade.html: ## @@ -61,6 +61,11 @@ Upgrading to 3.3.0 from any vers +Note that upgrading client to 3.3.0 would introduce a new Sticky Partitioner, des

[GitHub] [kafka] ijuma commented on a diff in pull request #12630: Doc: explain how to set old default partitioner

2022-09-13 Thread GitBox
ijuma commented on code in PR #12630: URL: https://github.com/apache/kafka/pull/12630#discussion_r969605501 ## docs/upgrade.html: ## @@ -61,6 +61,11 @@ Upgrading to 3.3.0 from any vers +Note that upgrading client to 3.3.0 would introduce a new Sticky Partitioner, des

[GitHub] [kafka] ijuma commented on a diff in pull request #12630: Doc: explain how to set old default partitioner

2022-09-13 Thread GitBox
ijuma commented on code in PR #12630: URL: https://github.com/apache/kafka/pull/12630#discussion_r969605501 ## docs/upgrade.html: ## @@ -61,6 +61,11 @@ Upgrading to 3.3.0 from any vers +Note that upgrading client to 3.3.0 would introduce a new Sticky Partitioner, des

[GitHub] [kafka] ijuma commented on a diff in pull request #12630: Doc: explain how to set old default partitioner

2022-09-13 Thread GitBox
ijuma commented on code in PR #12630: URL: https://github.com/apache/kafka/pull/12630#discussion_r969605501 ## docs/upgrade.html: ## @@ -61,6 +61,11 @@ Upgrading to 3.3.0 from any vers +Note that upgrading client to 3.3.0 would introduce a new Sticky Partitioner, des

[GitHub] [kafka] C0urante commented on a diff in pull request #12602: KAFKA-13985: Skip committing MirrorSourceTask records without metadata

2022-09-13 Thread GitBox
C0urante commented on code in PR #12602: URL: https://github.com/apache/kafka/pull/12602#discussion_r969617914 ## connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorSourceTaskTest.java: ## @@ -160,6 +160,33 @@ public void testPoll() { } } +@Te

[GitHub] [kafka] C0urante commented on pull request #12622: Update WorkerSinkTask.java

2022-09-13 Thread GitBox
C0urante commented on PR #12622: URL: https://github.com/apache/kafka/pull/12622#issuecomment-1245423635 Thanks @kumarpritam863, this is an important fix. I did some research on the history behind incremental consumer rebalancing and how it relates to the rebalance listener API, and found t

[GitHub] [kafka] C0urante commented on pull request #12561: KAFKA-12495: Exponential backoff retry to prevent rebalance storms when worker joins after revoking rebalance

2022-09-13 Thread GitBox
C0urante commented on PR #12561: URL: https://github.com/apache/kafka/pull/12561#issuecomment-1245436111 @vamossagar12 there are failing unit tests, can you check those out and ping us after you've taken a look and fixed any issues? -- This is an automated message from the Apache Git Serv

[GitHub] [kafka] vamossagar12 commented on pull request #12561: KAFKA-12495: Exponential backoff retry to prevent rebalance storms when worker joins after revoking rebalance

2022-09-13 Thread GitBox
vamossagar12 commented on PR #12561: URL: https://github.com/apache/kafka/pull/12561#issuecomment-1245454894 > @vamossagar12 there are failing unit tests, can you check those out and ping us after you've taken a look and fixed any issues? Oops.. Sorry about that. I didn't check the te

[GitHub] [kafka] C0urante commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
C0urante commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r969668198 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerTaskTest.java: ## @@ -78,82 +78,41 @@ public void tearDown() { public void standardStartup(

[GitHub] [kafka] yashmayya commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
yashmayya commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r969737990 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerTaskTest.java: ## @@ -78,82 +78,41 @@ public void tearDown() { public void standardStartup

[GitHub] [kafka] jsancio commented on pull request #12625: KAFKA-14222; KRaft's memory pool should always allocate a buffer

2022-09-13 Thread GitBox
jsancio commented on PR #12625: URL: https://github.com/apache/kafka/pull/12625#issuecomment-1245546902 Thanks @hachikuji . Merging -- the failures look unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [kafka] jsancio merged pull request #12625: KAFKA-14222; KRaft's memory pool should always allocate a buffer

2022-09-13 Thread GitBox
jsancio merged PR #12625: URL: https://github.com/apache/kafka/pull/12625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] ijuma commented on pull request #12620: KAFKA-14206: upgrade zookeeper version to 3.7.1

2022-09-13 Thread GitBox
ijuma commented on PR #12620: URL: https://github.com/apache/kafka/pull/12620#issuecomment-1245588332 Exactly right @divijvaidya, we have generally fixed transitive dependency issues directly. -- This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [kafka] ijuma commented on pull request #12620: KAFKA-14206: upgrade zookeeper version to 3.7.1

2022-09-13 Thread GitBox
ijuma commented on PR #12620: URL: https://github.com/apache/kafka/pull/12620#issuecomment-1245590870 One more thing: ZK is going away in AK 4.0. We probably need to do one more upgrade before that. It may make sense to wait a bit and go straight to 3.8 so that we benefit from longer suppor

[GitHub] [kafka] mumrah commented on pull request #12627: KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads.

2022-09-13 Thread GitBox
mumrah commented on PR #12627: URL: https://github.com/apache/kafka/pull/12627#issuecomment-1245603207 This was closed in favor of #12628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [kafka] C0urante commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
C0urante commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r969819802 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerTaskTest.java: ## @@ -78,155 +74,80 @@ public void tearDown() { public void standardStartup

[GitHub] [kafka] yashmayya commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
yashmayya commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r969848670 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerTaskTest.java: ## @@ -78,155 +74,80 @@ public void tearDown() { public void standardStartu

[GitHub] [kafka] C0urante commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
C0urante commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r969850848 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerTaskTest.java: ## @@ -78,155 +74,80 @@ public void tearDown() { public void standardStartup

[jira] [Updated] (KAFKA-14220) "partition-count" not getting updated after revocation of partitions in case of Incremental Co-operative rebalancing.

2022-09-13 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-14220: -- Fix Version/s: (was: 3.0.1) > "partition-count" not getting updated after revocation of pa

[jira] [Updated] (KAFKA-14220) "partition-count" not getting updated after revocation of partitions in case of Incremental Co-operative rebalancing.

2022-09-13 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-14220: -- Reviewer: Chris Egerton (was: Konstantine Karantasis) > "partition-count" not getting updated

[GitHub] [kafka] C0urante closed pull request #12023: Create AbstractLogContext class with static and dynamic subclasses, remove DynamicPrefixLogger

2022-09-13 Thread GitBox
C0urante closed pull request #12023: Create AbstractLogContext class with static and dynamic subclasses, remove DynamicPrefixLogger URL: https://github.com/apache/kafka/pull/12023 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [kafka] C0urante commented on a diff in pull request #12020: KAFKA-13818: Add generation to consumer assignor logs

2022-09-13 Thread GitBox
C0urante commented on code in PR #12020: URL: https://github.com/apache/kafka/pull/12020#discussion_r969868569 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -164,13 +166,23 @@ public ConsumerCoordinator(GroupRebalanceConfig

[GitHub] [kafka] C0urante commented on a diff in pull request #12020: KAFKA-13818: Add generation to consumer assignor logs

2022-09-13 Thread GitBox
C0urante commented on code in PR #12020: URL: https://github.com/apache/kafka/pull/12020#discussion_r969868569 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -164,13 +166,23 @@ public ConsumerCoordinator(GroupRebalanceConfig

[GitHub] [kafka] C0urante commented on a diff in pull request #12020: KAFKA-13818: Add generation to consumer assignor logs

2022-09-13 Thread GitBox
C0urante commented on code in PR #12020: URL: https://github.com/apache/kafka/pull/12020#discussion_r969868569 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -164,13 +166,23 @@ public ConsumerCoordinator(GroupRebalanceConfig

[GitHub] [kafka] C0urante commented on a diff in pull request #12020: KAFKA-13818: Add generation to consumer assignor logs

2022-09-13 Thread GitBox
C0urante commented on code in PR #12020: URL: https://github.com/apache/kafka/pull/12020#discussion_r969868569 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -164,13 +166,23 @@ public ConsumerCoordinator(GroupRebalanceConfig

[GitHub] [kafka] jsancio commented on pull request #12610: MINOR; Update documentation for printing dependencies

2022-09-13 Thread GitBox
jsancio commented on PR #12610: URL: https://github.com/apache/kafka/pull/12610#issuecomment-1245683056 @ijuma What do you think? Can we merge this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [kafka] jsancio merged pull request #12414: KAFKA-14073; Log the reason for snapshot

2022-09-13 Thread GitBox
jsancio merged PR #12414: URL: https://github.com/apache/kafka/pull/12414 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Resolved] (KAFKA-14073) Logging the reason for creating a snapshot

2022-09-13 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio resolved KAFKA-14073. Resolution: Fixed > Logging the reason for creating a snapshot > -

[jira] [Updated] (KAFKA-14206) Upgrade zookeeper to 3.7.1 to address security vulnerabilities

2022-09-13 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio updated KAFKA-14206: --- Fix Version/s: 3.4.0 > Upgrade zookeeper to 3.7.1 to address securit

[GitHub] [kafka] littlehorse-eng commented on a diff in pull request #12392: KAFKA-14053: Transactional producer should bump the epoch and skip ab…

2022-09-13 Thread GitBox
littlehorse-eng commented on code in PR #12392: URL: https://github.com/apache/kafka/pull/12392#discussion_r969894336 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/TransactionManagerTest.java: ## @@ -2594,27 +2596,20 @@ public void testDropCommitOnBatchExp

[GitHub] [kafka] nizhikov opened a new pull request, #12632: KAFKA-12878 Support --bootstrap-server in kafka-streams-application-reset tool

2022-09-13 Thread GitBox
nizhikov opened a new pull request, #12632: URL: https://github.com/apache/kafka/pull/12632 Implementation of [KIP-865](https://cwiki.apache.org/confluence/display/KAFKA/KIP-865%3A+Support+--bootstrap-server+in+kafka-streams-application-reset) Support of `--bootstrap-server` option fo

[GitHub] [kafka] philipnee opened a new pull request, #12633: [Consumer Refactor] Background thread skeleton

2022-09-13 Thread GitBox
philipnee opened a new pull request, #12633: URL: https://github.com/apache/kafka/pull/12633 * This is part of the effort for the consumer refactor project. This PR is the skeleton of the background thread that runs in the background. *Testing strategies Tests will be add once im

[GitHub] [kafka] philipnee commented on pull request #12633: [Consumer Refactor] Background thread skeleton

2022-09-13 Thread GitBox
philipnee commented on PR #12633: URL: https://github.com/apache/kafka/pull/12633#issuecomment-1245764396 @hachikuji @guozhangwang - This is the skeleton of the background thread. Please feel free to pull in reviewers. -- This is an automated message from the Apache Git Service. To r

[GitHub] [kafka] philipnee commented on pull request #12633: [Consumer Refactor] Background thread skeleton

2022-09-13 Thread GitBox
philipnee commented on PR #12633: URL: https://github.com/apache/kafka/pull/12633#issuecomment-1245767412 @showuon - Seems like you are quite familiar with the kafka consumer. So, cc you here. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] ijuma commented on a diff in pull request #12392: KAFKA-14053: Transactional producer should bump the epoch and skip ab…

2022-09-13 Thread GitBox
ijuma commented on code in PR #12392: URL: https://github.com/apache/kafka/pull/12392#discussion_r969939389 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/TransactionManagerTest.java: ## @@ -2594,27 +2596,20 @@ public void testDropCommitOnBatchExpiry() thro

[jira] [Created] (KAFKA-14225) lazy val exemptSensor Could Cause Deadlock

2022-09-13 Thread Huilin Shi (Jira)
Huilin Shi created KAFKA-14225: -- Summary: lazy val exemptSensor Could Cause Deadlock Key: KAFKA-14225 URL: https://issues.apache.org/jira/browse/KAFKA-14225 Project: Kafka Issue Type: Bug

[GitHub] [kafka] hachikuji merged pull request #12629: MINOR: Clean up FetcherTest

2022-09-13 Thread GitBox
hachikuji merged PR #12629: URL: https://github.com/apache/kafka/pull/12629 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apach

[jira] [Updated] (KAFKA-14225) lazy val exemptSensor Could Cause Deadlock

2022-09-13 Thread Huilin Shi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huilin Shi updated KAFKA-14225: --- Description: There is a chance to cause deadlock when multiple threads access ClientRequestQuotaMan

[jira] [Updated] (KAFKA-14225) lazy val exemptSensor Could Cause Deadlock

2022-09-13 Thread Huilin Shi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huilin Shi updated KAFKA-14225: --- Description: There is a chance to cause deadlock when multiple threads access ClientRequestQuotaMan

[GitHub] [kafka] hshi2022 opened a new pull request, #12634: KAFKA-14225: Fix deadlock caused by lazy val exemptSensor

2022-09-13 Thread GitBox
hshi2022 opened a new pull request, #12634: URL: https://github.com/apache/kafka/pull/12634 …ntRequestQuotaManager. In the version Scala 2.12, the lazy val initialization is under the object lock. The deadlock could happen in the following condition: In thread a, when ClientRequestQu

[GitHub] [kafka] artemlivshits commented on a diff in pull request #12630: MINOR; Explain how to set old default partitioner

2022-09-13 Thread GitBox
artemlivshits commented on code in PR #12630: URL: https://github.com/apache/kafka/pull/12630#discussion_r969977220 ## docs/upgrade.html: ## @@ -61,6 +61,11 @@ Upgrading to 3.3.0 from any vers +Note that upgrading client to 3.3.0 would introduce a new Sticky Partitio

[GitHub] [kafka] ijuma merged pull request #12630: MINOR; Explain how to set old default partitioner

2022-09-13 Thread GitBox
ijuma merged PR #12630: URL: https://github.com/apache/kafka/pull/12630 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] ijuma opened a new pull request, #12635: MINOR: Mention that kraft is production ready in upgrade notes

2022-09-13 Thread GitBox
ijuma opened a new pull request, #12635: URL: https://github.com/apache/kafka/pull/12635 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes)

[jira] [Updated] (KAFKA-14224) Consumer should auto-commit prior to cooperative partition revocation

2022-09-13 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-14224: -- Labels: new-consumer-threading-should-fix (was: ) > Consumer should auto-commit prior to coop

[jira] [Commented] (KAFKA-14224) Consumer should auto-commit prior to cooperative partition revocation

2022-09-13 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603743#comment-17603743 ] Guozhang Wang commented on KAFKA-14224: --- cc [~pnee] I think we can address this by

[GitHub] [kafka] junrao commented on a diff in pull request #12570: KAFKA-14156: Built-in partitioner may create suboptimal batches

2022-09-13 Thread GitBox
junrao commented on code in PR #12570: URL: https://github.com/apache/kafka/pull/12570#discussion_r970071437 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/RecordAccumulatorTest.java: ## @@ -1137,23 +1137,26 @@ public void onCompletion(RecordMetadata metada

[GitHub] [kafka] guozhangwang commented on a diff in pull request #12600: KAFKA-10199: Suspend tasks in the state updater on revocation

2022-09-13 Thread GitBox
guozhangwang commented on code in PR #12600: URL: https://github.com/apache/kafka/pull/12600#discussion_r970078065 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/Tasks.java: ## @@ -103,42 +101,75 @@ public void addPendingStandbyTasksToCreate(final Map

[GitHub] [kafka] C0urante commented on a diff in pull request #12615: KAFKA-14132: Migrate some Connect tests from EasyMock/PowerMock to Mockito

2022-09-13 Thread GitBox
C0urante commented on code in PR #12615: URL: https://github.com/apache/kafka/pull/12615#discussion_r970065964 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/errors/RetryWithToleranceOperatorTest.java: ## @@ -222,25 +224,31 @@ public void testExecAndHandleRet

[GitHub] [kafka] jsancio commented on a diff in pull request #12635: MINOR: Mention that kraft is production ready in upgrade notes

2022-09-13 Thread GitBox
jsancio commented on code in PR #12635: URL: https://github.com/apache/kafka/pull/12635#discussion_r970105135 ## docs/upgrade.html: ## @@ -81,6 +81,8 @@ Upgrading a KRaft-based cluster Notable changes in 3.3.0 +KRaft mode is production ready for new clusters.

[GitHub] [kafka] ijuma commented on a diff in pull request #12635: MINOR: Mention that kraft is production ready in upgrade notes

2022-09-13 Thread GitBox
ijuma commented on code in PR #12635: URL: https://github.com/apache/kafka/pull/12635#discussion_r970107576 ## docs/upgrade.html: ## @@ -81,6 +81,8 @@ Upgrading a KRaft-based cluster Notable changes in 3.3.0 +KRaft mode is production ready for new clusters. Se

[GitHub] [kafka] C0urante commented on a diff in pull request #12632: KAFKA-12878 Support --bootstrap-server in kafka-streams-application-reset tool

2022-09-13 Thread GitBox
C0urante commented on code in PR #12632: URL: https://github.com/apache/kafka/pull/12632#discussion_r970105520 ## core/src/main/scala/kafka/tools/StreamsResetter.java: ## @@ -213,8 +217,12 @@ private void parseArguments(final String[] args) { .ofType(String.class)

[GitHub] [kafka] C0urante commented on pull request #12632: KAFKA-12878 Support --bootstrap-server in kafka-streams-application-reset tool

2022-09-13 Thread GitBox
C0urante commented on PR #12632: URL: https://github.com/apache/kafka/pull/12632#issuecomment-1245985132 Should we also update the system test [here](https://github.com/apache/kafka/blob/7b49f175b964c468d5ca7ac9bd70dfd72e6a2e4e/tests/kafkatest/services/streams.py#L547)? -- This is an auto

[GitHub] [kafka] jsancio commented on a diff in pull request #12635: MINOR: Mention that kraft is production ready in upgrade notes

2022-09-13 Thread GitBox
jsancio commented on code in PR #12635: URL: https://github.com/apache/kafka/pull/12635#discussion_r970110217 ## docs/upgrade.html: ## @@ -81,6 +81,8 @@ Upgrading a KRaft-based cluster Notable changes in 3.3.0 +KRaft mode is production ready for new clusters.

[GitHub] [kafka] jsancio commented on a diff in pull request #12635: MINOR: Mention that kraft is production ready in upgrade notes

2022-09-13 Thread GitBox
jsancio commented on code in PR #12635: URL: https://github.com/apache/kafka/pull/12635#discussion_r970110815 ## docs/upgrade.html: ## @@ -81,6 +81,8 @@ Upgrading a KRaft-based cluster Notable changes in 3.3.0 +KRaft mode is production ready for new clusters.

[jira] [Commented] (KAFKA-10635) Streams application fails with OutOfOrderSequenceException after rolling restarts of brokers

2022-09-13 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603789#comment-17603789 ] Guozhang Wang commented on KAFKA-10635: --- Hi [~nicktelford], regarding the broker's

[GitHub] [kafka] cmccabe opened a new pull request, #12636: KAFKA-14214: Convert StandardAuthorizer to copy-on-write

2022-09-13 Thread GitBox
cmccabe opened a new pull request, #12636: URL: https://github.com/apache/kafka/pull/12636 Convert StandardAuthorizer to use a copy-on-write sorted array rather than a ConcurrentSkipList. The issue with the skiplist was that because it was modified while in use by StandardAuthorizer#authori

[jira] [Created] (KAFKA-14226) Introduce support for nested structures

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14226: Summary: Introduce support for nested structures Key: KAFKA-14226 URL: https://issues.apache.org/jira/browse/KAFKA-14226 Project: Kafka

[jira] [Created] (KAFKA-14227) Support for nested structures: MaskField

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14227: Summary: Support for nested structures: MaskField Key: KAFKA-14227 URL: https://issues.apache.org/jira/browse/KAFKA-14227 Project: Kafk

[jira] [Created] (KAFKA-14229) Support for nested structures: HoistValue

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14229: Summary: Support for nested structures: HoistValue Key: KAFKA-14229 URL: https://issues.apache.org/jira/browse/KAFKA-14229 Project: Kaf

[jira] [Created] (KAFKA-14228) Support for nested structures: ValueToKey

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14228: Summary: Support for nested structures: ValueToKey Key: KAFKA-14228 URL: https://issues.apache.org/jira/browse/KAFKA-14228 Project: Kaf

[jira] [Created] (KAFKA-14232) Support for nested structures: InsertField

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14232: Summary: Support for nested structures: InsertField Key: KAFKA-14232 URL: https://issues.apache.org/jira/browse/KAFKA-14232 Project: Ka

[jira] [Created] (KAFKA-14230) Support for nested structures: Cast

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14230: Summary: Support for nested structures: Cast Key: KAFKA-14230 URL: https://issues.apache.org/jira/browse/KAFKA-14230 Project: Kafka

[jira] [Created] (KAFKA-14231) Support for nested structures: ReplaceField

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
Jorge Esteban Quilcate Otoya created KAFKA-14231: Summary: Support for nested structures: ReplaceField Key: KAFKA-14231 URL: https://issues.apache.org/jira/browse/KAFKA-14231 Project: K

[GitHub] [kafka] jeqo opened a new pull request, #12637: KAFKA-14226: [connect:transform] Introduce support for nested structures

2022-09-13 Thread GitBox
jeqo opened a new pull request, #12637: URL: https://github.com/apache/kafka/pull/12637 KIP-821 initial implementation. *Summary of testing strategy (including rationale) for the feature or bug fix. Unit and/or integration tests are expected for any behaviour change and system

[jira] [Assigned] (KAFKA-13656) Connect Transforms support for nested structures

2022-09-13 Thread Jorge Esteban Quilcate Otoya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Esteban Quilcate Otoya reassigned KAFKA-13656: Assignee: Jorge Esteban Quilcate Otoya > Connect Transfor

[GitHub] [kafka] artemlivshits commented on a diff in pull request #12570: KAFKA-14156: Built-in partitioner may create suboptimal batches

2022-09-13 Thread GitBox
artemlivshits commented on code in PR #12570: URL: https://github.com/apache/kafka/pull/12570#discussion_r970145425 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/RecordAccumulatorTest.java: ## @@ -1137,23 +1137,26 @@ public void onCompletion(RecordMetadata

[GitHub] [kafka] artemlivshits commented on a diff in pull request #12570: KAFKA-14156: Built-in partitioner may create suboptimal batches

2022-09-13 Thread GitBox
artemlivshits commented on code in PR #12570: URL: https://github.com/apache/kafka/pull/12570#discussion_r970145425 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/RecordAccumulatorTest.java: ## @@ -1137,23 +1137,26 @@ public void onCompletion(RecordMetadata

  1   2   >