[GitHub] [kafka] showuon opened a new pull request, #13684: MINOR: fix compilation failure

2023-05-08 Thread via GitHub
showuon opened a new pull request, #13684: URL: https://github.com/apache/kafka/pull/13684 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[GitHub] [kafka] showuon opened a new pull request, #13685: test

2023-05-08 Thread via GitHub
showuon opened a new pull request, #13685: URL: https://github.com/apache/kafka/pull/13685 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[GitHub] [kafka] showuon closed pull request #13685: test

2023-05-08 Thread via GitHub
showuon closed pull request #13685: test URL: https://github.com/apache/kafka/pull/13685 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr..

[GitHub] [kafka] cmccabe opened a new pull request, #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
cmccabe opened a new pull request, #13686: URL: https://github.com/apache/kafka/pull/13686 Metadata image classes such as MetadataImage, ClusterImage, FeaturesImage, and so forth contain numerous sub-images. This PR adds a structured way of traversing those sub-images. This is useful for th

[GitHub] [kafka] showuon opened a new pull request, #13687: MINOR: test

2023-05-08 Thread via GitHub
showuon opened a new pull request, #13687: URL: https://github.com/apache/kafka/pull/13687 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[GitHub] [kafka] cadonna commented on a diff in pull request #13656: KAFKA-14911: Add system tests for rolling upgrade path of KIP-904

2023-05-08 Thread via GitHub
cadonna commented on code in PR #13656: URL: https://github.com/apache/kafka/pull/13656#discussion_r1187159604 ## tests/kafkatest/tests/streams/streams_upgrade_test.py: ## @@ -236,6 +237,96 @@ def test_rolling_upgrade_with_2_bounces(self, from_version, to_version): s

[GitHub] [kafka] cadonna commented on a diff in pull request #13656: KAFKA-14911: Add system tests for rolling upgrade path of KIP-904

2023-05-08 Thread via GitHub
cadonna commented on code in PR #13656: URL: https://github.com/apache/kafka/pull/13656#discussion_r1187176080 ## tests/kafkatest/tests/streams/streams_upgrade_test.py: ## @@ -236,6 +237,96 @@ def test_rolling_upgrade_with_2_bounces(self, from_version, to_version): s

[GitHub] [kafka] cadonna commented on a diff in pull request #13656: KAFKA-14911: Add system tests for rolling upgrade path of KIP-904

2023-05-08 Thread via GitHub
cadonna commented on code in PR #13656: URL: https://github.com/apache/kafka/pull/13656#discussion_r1187176080 ## tests/kafkatest/tests/streams/streams_upgrade_test.py: ## @@ -236,6 +237,96 @@ def test_rolling_upgrade_with_2_bounces(self, from_version, to_version): s

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13681: KAFKA-14133: Migrate ActiveTaskCreator mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
divijvaidya commented on code in PR #13681: URL: https://github.com/apache/kafka/pull/13681#discussion_r1187177289 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## Review Comment: we still seem to have easymock imports. Can you

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13681: KAFKA-14133: Migrate ActiveTaskCreator mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
divijvaidya commented on code in PR #13681: URL: https://github.com/apache/kafka/pull/13681#discussion_r1187177289 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## Review Comment: we still seem to have easymock imports. Can you

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13681: KAFKA-14133: Migrate ActiveTaskCreator mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
divijvaidya commented on code in PR #13681: URL: https://github.com/apache/kafka/pull/13681#discussion_r1187180990 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -291,16 +292,16 @@ public void shouldPrepareActiveTaskInStateUpda

[GitHub] [kafka] showuon commented on pull request #13684: MINOR: fix compilation failure

2023-05-08 Thread via GitHub
showuon commented on PR #13684: URL: https://github.com/apache/kafka/pull/13684#issuecomment-1538084603 Failed tests are unrelated: ``` Build / JDK 8 and Scala 2.12 / org.apache.kafka.clients.consumer.internals.EagerConsumerCoordinatorTest.testPrepareJoinAndRejoinAfterFailedRebalan

[GitHub] [kafka] showuon commented on pull request #13684: MINOR: fix compilation failure

2023-05-08 Thread via GitHub
showuon commented on PR #13684: URL: https://github.com/apache/kafka/pull/13684#issuecomment-1538085446 @mimaison , call for review to fix the build in v3.4 branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[jira] [Created] (KAFKA-14973) Inconsistent treatment of invalid config when creating or altering a topic

2023-05-08 Thread James Shaw (Jira)
James Shaw created KAFKA-14973: -- Summary: Inconsistent treatment of invalid config when creating or altering a topic Key: KAFKA-14973 URL: https://issues.apache.org/jira/browse/KAFKA-14973 Project: Kafka

[GitHub] [kafka] showuon commented on pull request #13684: MINOR: fix compilation failure

2023-05-08 Thread via GitHub
showuon commented on PR #13684: URL: https://github.com/apache/kafka/pull/13684#issuecomment-1538178676 It's fine sine it's for 3.4 branch, not for trunk branch. Thanks @divijvaidya -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [kafka] satishd commented on a diff in pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
satishd commented on code in PR #13535: URL: https://github.com/apache/kafka/pull/13535#discussion_r1187412666 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1083,48 +1095,100 @@ class ReplicaManager(val config: KafkaConfig, fetchPartitionStatus += (

[GitHub] [kafka] satishd commented on pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
satishd commented on PR #13535: URL: https://github.com/apache/kafka/pull/13535#issuecomment-1538311434 Thanks @junrao for the updated review comments. Addressed the comments with inline replies or with the latest commits. -- This is an automated message from the Apache Git Service. To r

[GitHub] [kafka] dajac commented on a diff in pull request #13644: KAFKA-14500; [1/N] Rewrite MemberMetadata in Java

2023-05-08 Thread via GitHub
dajac commented on code in PR #13644: URL: https://github.com/apache/kafka/pull/13644#discussion_r1187462145 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/generic/GenericGroupMember.java: ## @@ -0,0 +1,499 @@ +/** + * Licensed to the Apache Software Found

[GitHub] [kafka] vamossagar12 commented on pull request #13594: KAFKA-14913: Using ThreadUtils.shutdownExecutorServiceQuietly to close executors in Connect Runtime

2023-05-08 Thread via GitHub
vamossagar12 commented on PR #13594: URL: https://github.com/apache/kafka/pull/13594#issuecomment-1538450687 Thanks @viktorsomogyi . These are the ones we could find so far. We can create follow up tickets if needed for any other executors within connect which need to be closed this way and

[GitHub] [kafka] cadonna merged pull request #13621: KAFKA-14133: Migrate ChangeLogReader mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
cadonna merged PR #13621: URL: https://github.com/apache/kafka/pull/13621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Created] (KAFKA-14974) Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-14974: -- Summary: Restore backward compatibility in KafkaBasedLog Key: KAFKA-14974 URL: https://issues.apache.org/jira/browse/KAFKA-14974 Project: Kafka Issue Type: Task

[GitHub] [kafka] viktorsomogyi merged pull request #13594: KAFKA-14913: Using ThreadUtils.shutdownExecutorServiceQuietly to close executors in Connect Runtime

2023-05-08 Thread via GitHub
viktorsomogyi merged PR #13594: URL: https://github.com/apache/kafka/pull/13594 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.a

[GitHub] [kafka] viktorsomogyi commented on pull request #13594: KAFKA-14913: Using ThreadUtils.shutdownExecutorServiceQuietly to close executors in Connect Runtime

2023-05-08 Thread via GitHub
viktorsomogyi commented on PR #13594: URL: https://github.com/apache/kafka/pull/13594#issuecomment-1538474921 Merged it, thank you @vamossagar12 for the contribution! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Created] (KAFKA-14975) Make TopicBasedRemoteLogMetadataManager methods wait for initialize to complete

2023-05-08 Thread Matthew de Detrich (Jira)
Matthew de Detrich created KAFKA-14975: -- Summary: Make TopicBasedRemoteLogMetadataManager methods wait for initialize to complete Key: KAFKA-14975 URL: https://issues.apache.org/jira/browse/KAFKA-14975

[jira] [Updated] (KAFKA-14975) Make TopicBasedRemoteLogMetadataManager methods wait for initialize to complete

2023-05-08 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew de Detrich updated KAFKA-14975: --- Description: In the current implementation of  > Make TopicBasedRemoteLogMetadataMan

[jira] [Assigned] (KAFKA-14975) Make TopicBasedRemoteLogMetadataManager methods wait for initialize to complete

2023-05-08 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew de Detrich reassigned KAFKA-14975: -- Assignee: Matthew de Detrich > Make TopicBasedRemoteLogMetadataManager method

[jira] [Updated] (KAFKA-14975) Make TopicBasedRemoteLogMetadataManager methods wait for initialize to complete

2023-05-08 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew de Detrich updated KAFKA-14975: --- Description: In the current implementation of TopicBasedRemoteLogMetadataManager var

[GitHub] [kafka] yashmayya opened a new pull request, #13688: KAFKA-14974: Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread via GitHub
yashmayya opened a new pull request, #13688: URL: https://github.com/apache/kafka/pull/13688 From https://issues.apache.org/jira/browse/KAFKA-14974: > `KafkaBasedLog` is a widely used utility class that provides a generic implementation of a shared, compacted log of records in a Kafka

[GitHub] [kafka] yashmayya commented on pull request #13688: KAFKA-14974: Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread via GitHub
yashmayya commented on PR #13688: URL: https://github.com/apache/kafka/pull/13688#issuecomment-1538509105 @rhauch could you please take a look whenever you get a chance? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Commented] (KAFKA-14974) Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720546#comment-17720546 ] Randall Hauch commented on KAFKA-14974: --- Thanks for catching this and providing a

[GitHub] [kafka] mdedetrich opened a new pull request, #13689: KAFKA-14975: Wait for TopicBasedRemoteLogMetadataManager.ensureInitializedAndNotClosed to initialize rather than throwing

2023-05-08 Thread via GitHub
mdedetrich opened a new pull request, #13689: URL: https://github.com/apache/kafka/pull/13689 Current implementation of TopicBasedRemoteLogMetadataManager.ensureInitializedAndNotClosed will immediately throw an exception if TopicBasedRemoteLogMetadataManager is not initialized. Rather we s

[GitHub] [kafka] mdedetrich commented on pull request #13689: KAFKA-14975: Wait for TopicBasedRemoteLogMetadataManager.ensureInitializedAndNotClosed to initialize rather than throwing

2023-05-08 Thread via GitHub
mdedetrich commented on PR #13689: URL: https://github.com/apache/kafka/pull/13689#issuecomment-1538539482 @satishd @junrao Pinging you because you done previous PR's regarding Tiered Storage -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [kafka] mdedetrich commented on a diff in pull request #13689: KAFKA-14975: Wait for TopicBasedRemoteLogMetadataManager.ensureInitializedAndNotClosed to initialize rather than throwing

2023-05-08 Thread via GitHub
mdedetrich commented on code in PR #13689: URL: https://github.com/apache/kafka/pull/13689#discussion_r1187572164 ## storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerHarness.java: ## @@ -104,27 +103,10 @@ public void on

[jira] [Updated] (KAFKA-14974) Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-14974: -- Component/s: KafkaConnect > Restore backward compatibility in KafkaBasedLog >

[jira] [Updated] (KAFKA-14974) Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya updated KAFKA-14974: --- Affects Version/s: 3.5.0 3.4.1 3.3.3 > Restore backwar

[jira] [Updated] (KAFKA-14455) Kafka Connect create and update REST APIs should surface failures while writing to the config topic

2023-05-08 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya updated KAFKA-14455: --- Fix Version/s: 3.5.0 3.4.1 3.3.3 > Kafka Connect create and up

[GitHub] [kafka] jolshan commented on a diff in pull request #13638: KAFKA-14462; [11/N] Add CurrentAssignmentBuilder

2023-05-08 Thread via GitHub
jolshan commented on code in PR #13638: URL: https://github.com/apache/kafka/pull/13638#discussion_r1187582387 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -0,0 +1,445 @@ +/* + * Licensed to the Apache Softwa

[jira] [Commented] (KAFKA-14974) Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720553#comment-17720553 ] Randall Hauch commented on KAFKA-14974: --- [~yash.mayya], just to clarify: # #12984

[GitHub] [kafka] reta commented on a diff in pull request #13689: KAFKA-14975: Wait for TopicBasedRemoteLogMetadataManager.ensureInitializedAndNotClosed to initialize rather than throwing

2023-05-08 Thread via GitHub
reta commented on code in PR #13689: URL: https://github.com/apache/kafka/pull/13689#discussion_r1187596223 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManager.java: ## @@ -76,6 +78,11 @@ public class TopicBasedRemoteLo

[GitHub] [kafka] yashmayya commented on pull request #13688: KAFKA-14974: Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread via GitHub
yashmayya commented on PR #13688: URL: https://github.com/apache/kafka/pull/13688#issuecomment-1538608185 Yep, that's right. We should backport this to `3.3`, `3.4` and `3.5` as well (before the `3.5.0` release ideally, if possible). -- This is an automated message from the Apache Git Ser

[jira] [Commented] (KAFKA-14974) Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720566#comment-17720566 ] Yash Mayya commented on KAFKA-14974: Thanks [~rhauch], your understanding here is co

[GitHub] [kafka] mehbey commented on a diff in pull request #13681: KAFKA-14133: Migrate ActiveTaskCreator mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
mehbey commented on code in PR #13681: URL: https://github.com/apache/kafka/pull/13681#discussion_r1187611572 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -2579,7 +2563,9 @@ public void shouldUpdateInputPartitionsAfterRebalanc

[GitHub] [kafka] yashmayya commented on pull request #13688: KAFKA-14974: Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread via GitHub
yashmayya commented on PR #13688: URL: https://github.com/apache/kafka/pull/13688#issuecomment-1538658594 Thanks for taking a look @rhauch and your understanding here is correct. We should try to backport this to `3.3`, `3.4` and `3.5` as well (before the `3.5.0` release ideally, if possibl

[GitHub] [kafka] dajac commented on a diff in pull request #13638: KAFKA-14462; [11/N] Add CurrentAssignmentBuilder

2023-05-08 Thread via GitHub
dajac commented on code in PR #13638: URL: https://github.com/apache/kafka/pull/13638#discussion_r1187624902 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -0,0 +1,445 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] philipnee commented on pull request #13455: KAFKA-14841 Handle callbacks to ConsumerRebalanceListener in MockConsumer

2023-05-08 Thread via GitHub
philipnee commented on PR #13455: URL: https://github.com/apache/kafka/pull/13455#issuecomment-1538664591 Thanks @vvcephei - Think this is the failure. ZK related so I think the PR is good to go. `Build / JDK 17 and Scala 2.13 / [1] Type=ZK, Name=testDualWrite, MetadataVersion=3.4-I

[GitHub] [kafka] philipnee commented on pull request #13664: KAFKA-14961: harden DefaultBackgroundThreadTest.testStartupAndTearDown test

2023-05-08 Thread via GitHub
philipnee commented on PR #13664: URL: https://github.com/apache/kafka/pull/13664#issuecomment-1538666147 Thanks @vvcephei - I think this is a flaky test? `Build / JDK 17 and Scala 2.13 / testDescribeStateOfExistingGroupWithRoundRobinAssignor() – kafka.admin.DescribeConsumerGroupTest

[GitHub] [kafka] jolshan commented on a diff in pull request #13638: KAFKA-14462; [11/N] Add CurrentAssignmentBuilder

2023-05-08 Thread via GitHub
jolshan commented on code in PR #13638: URL: https://github.com/apache/kafka/pull/13638#discussion_r1187640544 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -0,0 +1,445 @@ +/* + * Licensed to the Apache Softwa

[GitHub] [kafka] hachikuji commented on a diff in pull request #13608: KAFKA-14884: Include check transaction is still ongoing right before append

2023-05-08 Thread via GitHub
hachikuji commented on code in PR #13608: URL: https://github.com/apache/kafka/pull/13608#discussion_r1187662801 ## storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateEntry.java: ## @@ -103,28 +121,44 @@ public boolean maybeUpdateProducerEpoch(short produ

[GitHub] [kafka] kirktrue commented on pull request #13640: KAFKA-14937: Refactoring for client code to reduce boilerplate

2023-05-08 Thread via GitHub
kirktrue commented on PR #13640: URL: https://github.com/apache/kafka/pull/13640#issuecomment-1538766507 Closing pull request and moving to a fork where we're doing integration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] kirktrue closed pull request #13640: KAFKA-14937: Refactoring for client code to reduce boilerplate

2023-05-08 Thread via GitHub
kirktrue closed pull request #13640: KAFKA-14937: Refactoring for client code to reduce boilerplate URL: https://github.com/apache/kafka/pull/13640 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [kafka] jolshan commented on a diff in pull request #13608: KAFKA-14884: Include check transaction is still ongoing right before append

2023-05-08 Thread via GitHub
jolshan commented on code in PR #13608: URL: https://github.com/apache/kafka/pull/13608#discussion_r1187719600 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -579,9 +579,33 @@ class UnifiedLog(@volatile var logStartOffset: Long, result } - def hasOngoingTran

[GitHub] [kafka] jolshan commented on pull request #13608: KAFKA-14884: Include check transaction is still ongoing right before append

2023-05-08 Thread via GitHub
jolshan commented on PR #13608: URL: https://github.com/apache/kafka/pull/13608#issuecomment-1538787228 Thanks @hachikuji! Working on the tests as well, so I will push those when they are ready. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] mdedetrich commented on a diff in pull request #13689: KAFKA-14975: Wait for TopicBasedRemoteLogMetadataManager.ensureInitializedAndNotClosed to initialize rather than throwing

2023-05-08 Thread via GitHub
mdedetrich commented on code in PR #13689: URL: https://github.com/apache/kafka/pull/13689#discussion_r1187745268 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManager.java: ## @@ -76,6 +78,11 @@ public class TopicBasedRe

[GitHub] [kafka] Scanteianu commented on pull request #13455: KAFKA-14841 Handle callbacks to ConsumerRebalanceListener in MockConsumer

2023-05-08 Thread via GitHub
Scanteianu commented on PR #13455: URL: https://github.com/apache/kafka/pull/13455#issuecomment-1538836327 > Thanks @vvcephei - Think this is the failure. ZK related so I think the PR is good to go. > > `Build / JDK 17 and Scala 2.13 / [1] Type=ZK, Name=testDualWrite, MetadataVersion

[GitHub] [kafka] dajac merged pull request #13638: KAFKA-14462; [11/N] Add CurrentAssignmentBuilder

2023-05-08 Thread via GitHub
dajac merged PR #13638: URL: https://github.com/apache/kafka/pull/13638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] vepo commented on pull request #12809: KAFKA-14324: Upgrade RocksDB to 7.1.2

2023-05-08 Thread via GitHub
vepo commented on PR #12809: URL: https://github.com/apache/kafka/pull/12809#issuecomment-1538926727 Is there any fix for this memory leak? How can we reproduce this memory leak for testing? Some of our environment is facing a memory leak but we cannot reproduce it, as we use **_org.

[GitHub] [kafka] mehbey commented on pull request #13681: KAFKA-14133: Migrate ActiveTaskCreator mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
mehbey commented on PR #13681: URL: https://github.com/apache/kafka/pull/13681#issuecomment-1538991259 Addressed Divij's comment and re-based with the latest changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Updated] (KAFKA-14976) Left/outer stream-stream joins create KV stores that aren't customizable

2023-05-08 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-14976: --- Labels: needs-kip (was: ) > Left/outer stream-stream joins create KV stores

[jira] [Created] (KAFKA-14976) Left/outer stream-stream joins create KV stores that aren't customizable

2023-05-08 Thread A. Sophie Blee-Goldman (Jira)
A. Sophie Blee-Goldman created KAFKA-14976: -- Summary: Left/outer stream-stream joins create KV stores that aren't customizable Key: KAFKA-14976 URL: https://issues.apache.org/jira/browse/KAFKA-14976

[GitHub] [kafka] mumrah commented on a diff in pull request #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
mumrah commented on code in PR #13686: URL: https://github.com/apache/kafka/pull/13686#discussion_r1187879810 ## shell/src/main/java/org/apache/kafka/shell/node/printer/ShellNodePrinter.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[jira] [Resolved] (KAFKA-14698) Received request api key LEADER_AND_ISR which is not enabled

2023-05-08 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14698. -- Fix Version/s: (was: 3.4.1) Resolution: Duplicate > Received request api key LEADER

[jira] [Resolved] (KAFKA-14918) KRaft controller sending ZK controller RPCs to KRaft brokers

2023-05-08 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14918. -- Resolution: Fixed > KRaft controller sending ZK controller RPCs to KRaft brokers > ---

[jira] [Commented] (KAFKA-14918) KRaft controller sending ZK controller RPCs to KRaft brokers

2023-05-08 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720672#comment-17720672 ] Colin McCabe commented on KAFKA-14918: -- This is fixed, sorry for the delay in resol

[GitHub] [kafka] mjsax commented on a diff in pull request #13682: MINOR: improved exception/warn logging for stream-stream join store settings

2023-05-08 Thread via GitHub
mjsax commented on code in PR #13682: URL: https://github.com/apache/kafka/pull/13682#discussion_r1187897510 ## streams/src/main/java/org/apache/kafka/streams/state/internals/VersionedKeyValueToBytesStoreAdapter.java: ## @@ -115,7 +115,7 @@ public boolean persistent() { @

[GitHub] [kafka] cmccabe commented on a diff in pull request #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
cmccabe commented on code in PR #13686: URL: https://github.com/apache/kafka/pull/13686#discussion_r1187897908 ## shell/src/main/java/org/apache/kafka/shell/node/printer/ShellNodePrinter.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] cmccabe commented on a diff in pull request #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
cmccabe commented on code in PR #13686: URL: https://github.com/apache/kafka/pull/13686#discussion_r1187898447 ## metadata/src/main/java/org/apache/kafka/image/node/printer/MetadataNodeRedactionCriteria.java: ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] cmccabe commented on a diff in pull request #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
cmccabe commented on code in PR #13686: URL: https://github.com/apache/kafka/pull/13686#discussion_r1187898816 ## shell/src/main/java/org/apache/kafka/shell/state/MetadataShellState.java: ## @@ -0,0 +1,57 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or m

[GitHub] [kafka] cmccabe commented on a diff in pull request #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
cmccabe commented on code in PR #13686: URL: https://github.com/apache/kafka/pull/13686#discussion_r1187899780 ## shell/src/main/java/org/apache/kafka/shell/command/FindCommandHandler.java: ## @@ -80,28 +84,28 @@ public FindCommandHandler(List paths) { } @Override -

[GitHub] [kafka] cmccabe commented on a diff in pull request #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
cmccabe commented on code in PR #13686: URL: https://github.com/apache/kafka/pull/13686#discussion_r1187901148 ## shell/src/main/java/org/apache/kafka/shell/glob/GlobVisitor.java: ## @@ -93,32 +96,33 @@ public String toString() { } @Override -public void accept(M

[GitHub] [kafka] junrao commented on a diff in pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
junrao commented on code in PR #13535: URL: https://github.com/apache/kafka/pull/13535#discussion_r1187906662 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1083,48 +1095,100 @@ class ReplicaManager(val config: KafkaConfig, fetchPartitionStatus += (t

[GitHub] [kafka] mjsax merged pull request #13654: HOTFIX: fix broken Streams upgrade system test

2023-05-08 Thread via GitHub
mjsax merged PR #13654: URL: https://github.com/apache/kafka/pull/13654 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] mjsax commented on pull request #13654: HOTFIX: fix broken Streams upgrade system test

2023-05-08 Thread via GitHub
mjsax commented on PR #13654: URL: https://github.com/apache/kafka/pull/13654#issuecomment-1539079411 Merged to `trunk` and cherry-picked to `3.5` branch. \cc @mimaison Seems some other streams system tests are still flaky (well, I hope it's not a real issue). I hope to find time to

[GitHub] [kafka] vvcephei commented on pull request #13664: KAFKA-14961: harden DefaultBackgroundThreadTest.testStartupAndTearDown test

2023-05-08 Thread via GitHub
vvcephei commented on PR #13664: URL: https://github.com/apache/kafka/pull/13664#issuecomment-1539151873 Thanks, @philipnee ! I agree it's probably flaky, but I don't see a ticket filed for it. We haven't been very strict recently about this, but I'm reluctant to just forge ahead at

[GitHub] [kafka] satishd commented on a diff in pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
satishd commented on code in PR #13535: URL: https://github.com/apache/kafka/pull/13535#discussion_r1187999169 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1083,48 +1095,100 @@ class ReplicaManager(val config: KafkaConfig, fetchPartitionStatus += (

[GitHub] [kafka] satishd commented on pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
satishd commented on PR #13535: URL: https://github.com/apache/kafka/pull/13535#issuecomment-1539215636 Thanks @junrao , updated the [comment](https://github.com/apache/kafka/pull/13535#discussion_r1184274133) with the latest commit. -- This is an automated message from the Apache Git S

[GitHub] [kafka] junrao commented on a diff in pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
junrao commented on code in PR #13535: URL: https://github.com/apache/kafka/pull/13535#discussion_r1187994355 ## core/src/test/scala/integration/kafka/server/DelayedRemoteFetchTest.scala: ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

[jira] [Updated] (KAFKA-14561) Improve transactions experience for older clients by ensuring ongoing transaction

2023-05-08 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-14561: Fix Version/s: 3.6.0 (was: 3.5.0) The PR was reverted in 3.5 branch. Updated the fi

[jira] [Comment Edited] (KAFKA-14561) Improve transactions experience for older clients by ensuring ongoing transaction

2023-05-08 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720717#comment-17720717 ] Jun Rao edited comment on KAFKA-14561 at 5/9/23 12:24 AM: -- The

[GitHub] [kafka] mehbey commented on pull request #13681: KAFKA-14133: Migrate ActiveTaskCreator mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
mehbey commented on PR #13681: URL: https://github.com/apache/kafka/pull/13681#issuecomment-1539231000 Verified that failing Testing are not related to this change ``` [2023-05-08T22:22:09.340Z] 1: Task failed with an exception. [2023-05-08T22:22:09.340Z] --- [2023

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13443: KAFKA-14514: Add Range Assignor on the Server (KIP-848)

2023-05-08 Thread via GitHub
rreddy-22 commented on code in PR #13443: URL: https://github.com/apache/kafka/pull/13443#discussion_r1188009097 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/assignor/RangeAssignorTest.java: ## @@ -0,0 +1,574 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13443: KAFKA-14514: Add Range Assignor on the Server (KIP-848)

2023-05-08 Thread via GitHub
rreddy-22 commented on code in PR #13443: URL: https://github.com/apache/kafka/pull/13443#discussion_r1188009762 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/assignor/RangeAssignorTest.java: ## @@ -0,0 +1,574 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] mjsax commented on a diff in pull request #11433: KAFKA-13295: Avoiding Transation timeouts arising due to long restora…

2023-05-08 Thread via GitHub
mjsax commented on code in PR #11433: URL: https://github.com/apache/kafka/pull/11433#discussion_r1188018045 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -342,7 +342,18 @@ public void handleAssignment(final Map> activeTasks,

[GitHub] [kafka] mjsax commented on a diff in pull request #11433: KAFKA-13295: Avoiding Transation timeouts arising due to long restora…

2023-05-08 Thread via GitHub
mjsax commented on code in PR #11433: URL: https://github.com/apache/kafka/pull/11433#discussion_r1180626928 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -342,7 +342,18 @@ public void handleAssignment(final Map> activeTasks,

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-08 Thread via GitHub
jeffkbkim commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1187976451 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -0,0 +1,865 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] satishd commented on a diff in pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
satishd commented on code in PR #13535: URL: https://github.com/apache/kafka/pull/13535#discussion_r1187999169 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1083,48 +1095,100 @@ class ReplicaManager(val config: KafkaConfig, fetchPartitionStatus += (

[GitHub] [kafka] satishd commented on a diff in pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
satishd commented on code in PR #13535: URL: https://github.com/apache/kafka/pull/13535#discussion_r1187999169 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1083,48 +1095,100 @@ class ReplicaManager(val config: KafkaConfig, fetchPartitionStatus += (

[GitHub] [kafka] yashmayya commented on pull request #13688: KAFKA-14974: Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread via GitHub
yashmayya commented on PR #13688: URL: https://github.com/apache/kafka/pull/13688#issuecomment-1539275733 @rhauch looks like there are 4 test failures - none of them are new failures or related to this change. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] showuon commented on a diff in pull request #13584: MINOR: Add log segment unit tests, If the maximum offset beyond index, appen…

2023-05-08 Thread via GitHub
showuon commented on code in PR #13584: URL: https://github.com/apache/kafka/pull/13584#discussion_r1188050289 ## core/src/test/scala/unit/kafka/log/LogSegmentTest.scala: ## @@ -65,6 +68,29 @@ class LogSegmentTest { Utils.delete(logDir) } + /** + * If the maximum of

[GitHub] [kafka] showuon commented on a diff in pull request #13584: MINOR: Add log segment unit tests, If the maximum offset beyond index, appen…

2023-05-08 Thread via GitHub
showuon commented on code in PR #13584: URL: https://github.com/apache/kafka/pull/13584#discussion_r1188050289 ## core/src/test/scala/unit/kafka/log/LogSegmentTest.scala: ## @@ -65,6 +68,29 @@ class LogSegmentTest { Utils.delete(logDir) } + /** + * If the maximum of

[GitHub] [kafka] showuon merged pull request #13684: MINOR: fix compilation failure

2023-05-08 Thread via GitHub
showuon merged PR #13684: URL: https://github.com/apache/kafka/pull/13684 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Commented] (KAFKA-8170) To add kafka data at rest encryption

2023-05-08 Thread kaiyangzhang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720759#comment-17720759 ] kaiyangzhang commented on KAFKA-8170: - [~sliebau] Hello, do you have any new progress

[jira] [Comment Edited] (KAFKA-8170) To add kafka data at rest encryption

2023-05-08 Thread kaiyangzhang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720759#comment-17720759 ] kaiyangzhang edited comment on KAFKA-8170 at 5/9/23 3:28 AM: -

[jira] [Created] (KAFKA-14977) testDescribeStateOfExistingGroupWithRoundRobinAssignor is flaky

2023-05-08 Thread Philip Nee (Jira)
Philip Nee created KAFKA-14977: -- Summary: testDescribeStateOfExistingGroupWithRoundRobinAssignor is flaky Key: KAFKA-14977 URL: https://issues.apache.org/jira/browse/KAFKA-14977 Project: Kafka

[GitHub] [kafka] philipnee commented on pull request #13664: KAFKA-14961: harden DefaultBackgroundThreadTest.testStartupAndTearDown test

2023-05-08 Thread via GitHub
philipnee commented on PR #13664: URL: https://github.com/apache/kafka/pull/13664#issuecomment-1539356020 Thank you @vvcephei - https://issues.apache.org/jira/browse/KAFKA-14977 is filed 💘 -- This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Created] (KAFKA-14978) ExactlyOnceWorkerSourceTask does not remove parent metrics

2023-05-08 Thread Daniel Urban (Jira)
Daniel Urban created KAFKA-14978: Summary: ExactlyOnceWorkerSourceTask does not remove parent metrics Key: KAFKA-14978 URL: https://issues.apache.org/jira/browse/KAFKA-14978 Project: Kafka Is

[GitHub] [kafka] urbandan opened a new pull request, #13690: KAFKA-14978 ExactlyOnceWorkerSourceTask should remove parent metrics

2023-05-08 Thread via GitHub
urbandan opened a new pull request, #13690: URL: https://github.com/apache/kafka/pull/13690 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testin

[GitHub] [kafka] urbandan commented on pull request #13690: KAFKA-14978 ExactlyOnceWorkerSourceTask should remove parent metrics

2023-05-08 Thread via GitHub
urbandan commented on PR #13690: URL: https://github.com/apache/kafka/pull/13690#issuecomment-1539486889 @viktorsomogyi can you please take a look at this small fix? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] cadonna commented on pull request #12809: KAFKA-14324: Upgrade RocksDB to 7.1.2

2023-05-08 Thread via GitHub
cadonna commented on PR #12809: URL: https://github.com/apache/kafka/pull/12809#issuecomment-1539552693 @vepo The memory leaks are fixed in the following PRs: https://github.com/apache/kafka/pull/12935 https://github.com/apache/kafka/pull/12959 Without those fixes we would have r