[GitHub] [kafka] cadonna commented on pull request #12809: KAFKA-14324: Upgrade RocksDB to 7.1.2

2023-05-08 Thread via GitHub
cadonna commented on PR #12809: URL: https://github.com/apache/kafka/pull/12809#issuecomment-1539552693 @vepo The memory leaks are fixed in the following PRs: https://github.com/apache/kafka/pull/12935 https://github.com/apache/kafka/pull/12959 Without those fixes we would have r

[GitHub] [kafka] urbandan commented on pull request #13690: KAFKA-14978 ExactlyOnceWorkerSourceTask should remove parent metrics

2023-05-08 Thread via GitHub
urbandan commented on PR #13690: URL: https://github.com/apache/kafka/pull/13690#issuecomment-1539486889 @viktorsomogyi can you please take a look at this small fix? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] urbandan opened a new pull request, #13690: KAFKA-14978 ExactlyOnceWorkerSourceTask should remove parent metrics

2023-05-08 Thread via GitHub
urbandan opened a new pull request, #13690: URL: https://github.com/apache/kafka/pull/13690 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testin

[jira] [Created] (KAFKA-14978) ExactlyOnceWorkerSourceTask does not remove parent metrics

2023-05-08 Thread Daniel Urban (Jira)
Daniel Urban created KAFKA-14978: Summary: ExactlyOnceWorkerSourceTask does not remove parent metrics Key: KAFKA-14978 URL: https://issues.apache.org/jira/browse/KAFKA-14978 Project: Kafka Is

[GitHub] [kafka] philipnee commented on pull request #13664: KAFKA-14961: harden DefaultBackgroundThreadTest.testStartupAndTearDown test

2023-05-08 Thread via GitHub
philipnee commented on PR #13664: URL: https://github.com/apache/kafka/pull/13664#issuecomment-1539356020 Thank you @vvcephei - https://issues.apache.org/jira/browse/KAFKA-14977 is filed 💘 -- This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Created] (KAFKA-14977) testDescribeStateOfExistingGroupWithRoundRobinAssignor is flaky

2023-05-08 Thread Philip Nee (Jira)
Philip Nee created KAFKA-14977: -- Summary: testDescribeStateOfExistingGroupWithRoundRobinAssignor is flaky Key: KAFKA-14977 URL: https://issues.apache.org/jira/browse/KAFKA-14977 Project: Kafka

[jira] [Comment Edited] (KAFKA-8170) To add kafka data at rest encryption

2023-05-08 Thread kaiyangzhang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720759#comment-17720759 ] kaiyangzhang edited comment on KAFKA-8170 at 5/9/23 3:28 AM: -

[jira] [Commented] (KAFKA-8170) To add kafka data at rest encryption

2023-05-08 Thread kaiyangzhang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720759#comment-17720759 ] kaiyangzhang commented on KAFKA-8170: - [~sliebau] Hello, do you have any new progress

[GitHub] [kafka] showuon merged pull request #13684: MINOR: fix compilation failure

2023-05-08 Thread via GitHub
showuon merged PR #13684: URL: https://github.com/apache/kafka/pull/13684 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] showuon commented on a diff in pull request #13584: MINOR: Add log segment unit tests, If the maximum offset beyond index, appen…

2023-05-08 Thread via GitHub
showuon commented on code in PR #13584: URL: https://github.com/apache/kafka/pull/13584#discussion_r1188050289 ## core/src/test/scala/unit/kafka/log/LogSegmentTest.scala: ## @@ -65,6 +68,29 @@ class LogSegmentTest { Utils.delete(logDir) } + /** + * If the maximum of

[GitHub] [kafka] showuon commented on a diff in pull request #13584: MINOR: Add log segment unit tests, If the maximum offset beyond index, appen…

2023-05-08 Thread via GitHub
showuon commented on code in PR #13584: URL: https://github.com/apache/kafka/pull/13584#discussion_r1188050289 ## core/src/test/scala/unit/kafka/log/LogSegmentTest.scala: ## @@ -65,6 +68,29 @@ class LogSegmentTest { Utils.delete(logDir) } + /** + * If the maximum of

[GitHub] [kafka] yashmayya commented on pull request #13688: KAFKA-14974: Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread via GitHub
yashmayya commented on PR #13688: URL: https://github.com/apache/kafka/pull/13688#issuecomment-1539275733 @rhauch looks like there are 4 test failures - none of them are new failures or related to this change. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] satishd commented on a diff in pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
satishd commented on code in PR #13535: URL: https://github.com/apache/kafka/pull/13535#discussion_r1187999169 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1083,48 +1095,100 @@ class ReplicaManager(val config: KafkaConfig, fetchPartitionStatus += (

[GitHub] [kafka] satishd commented on a diff in pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
satishd commented on code in PR #13535: URL: https://github.com/apache/kafka/pull/13535#discussion_r1187999169 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1083,48 +1095,100 @@ class ReplicaManager(val config: KafkaConfig, fetchPartitionStatus += (

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-08 Thread via GitHub
jeffkbkim commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1187976451 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -0,0 +1,865 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] mjsax commented on a diff in pull request #11433: KAFKA-13295: Avoiding Transation timeouts arising due to long restora…

2023-05-08 Thread via GitHub
mjsax commented on code in PR #11433: URL: https://github.com/apache/kafka/pull/11433#discussion_r1180626928 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -342,7 +342,18 @@ public void handleAssignment(final Map> activeTasks,

[GitHub] [kafka] mjsax commented on a diff in pull request #11433: KAFKA-13295: Avoiding Transation timeouts arising due to long restora…

2023-05-08 Thread via GitHub
mjsax commented on code in PR #11433: URL: https://github.com/apache/kafka/pull/11433#discussion_r1188018045 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -342,7 +342,18 @@ public void handleAssignment(final Map> activeTasks,

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13443: KAFKA-14514: Add Range Assignor on the Server (KIP-848)

2023-05-08 Thread via GitHub
rreddy-22 commented on code in PR #13443: URL: https://github.com/apache/kafka/pull/13443#discussion_r1188009762 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/assignor/RangeAssignorTest.java: ## @@ -0,0 +1,574 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13443: KAFKA-14514: Add Range Assignor on the Server (KIP-848)

2023-05-08 Thread via GitHub
rreddy-22 commented on code in PR #13443: URL: https://github.com/apache/kafka/pull/13443#discussion_r1188009097 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/assignor/RangeAssignorTest.java: ## @@ -0,0 +1,574 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] mehbey commented on pull request #13681: KAFKA-14133: Migrate ActiveTaskCreator mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
mehbey commented on PR #13681: URL: https://github.com/apache/kafka/pull/13681#issuecomment-1539231000 Verified that failing Testing are not related to this change ``` [2023-05-08T22:22:09.340Z] 1: Task failed with an exception. [2023-05-08T22:22:09.340Z] --- [2023

[jira] [Comment Edited] (KAFKA-14561) Improve transactions experience for older clients by ensuring ongoing transaction

2023-05-08 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720717#comment-17720717 ] Jun Rao edited comment on KAFKA-14561 at 5/9/23 12:24 AM: -- The

[jira] [Updated] (KAFKA-14561) Improve transactions experience for older clients by ensuring ongoing transaction

2023-05-08 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-14561: Fix Version/s: 3.6.0 (was: 3.5.0) The PR was reverted in 3.5 branch. Updated the fi

[GitHub] [kafka] junrao commented on a diff in pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
junrao commented on code in PR #13535: URL: https://github.com/apache/kafka/pull/13535#discussion_r1187994355 ## core/src/test/scala/integration/kafka/server/DelayedRemoteFetchTest.scala: ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] satishd commented on pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
satishd commented on PR #13535: URL: https://github.com/apache/kafka/pull/13535#issuecomment-1539215636 Thanks @junrao , updated the [comment](https://github.com/apache/kafka/pull/13535#discussion_r1184274133) with the latest commit. -- This is an automated message from the Apache Git S

[GitHub] [kafka] satishd commented on a diff in pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
satishd commented on code in PR #13535: URL: https://github.com/apache/kafka/pull/13535#discussion_r1187999169 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1083,48 +1095,100 @@ class ReplicaManager(val config: KafkaConfig, fetchPartitionStatus += (

[GitHub] [kafka] vvcephei commented on pull request #13664: KAFKA-14961: harden DefaultBackgroundThreadTest.testStartupAndTearDown test

2023-05-08 Thread via GitHub
vvcephei commented on PR #13664: URL: https://github.com/apache/kafka/pull/13664#issuecomment-1539151873 Thanks, @philipnee ! I agree it's probably flaky, but I don't see a ticket filed for it. We haven't been very strict recently about this, but I'm reluctant to just forge ahead at

[GitHub] [kafka] mjsax commented on pull request #13654: HOTFIX: fix broken Streams upgrade system test

2023-05-08 Thread via GitHub
mjsax commented on PR #13654: URL: https://github.com/apache/kafka/pull/13654#issuecomment-1539079411 Merged to `trunk` and cherry-picked to `3.5` branch. \cc @mimaison Seems some other streams system tests are still flaky (well, I hope it's not a real issue). I hope to find time to

[GitHub] [kafka] mjsax merged pull request #13654: HOTFIX: fix broken Streams upgrade system test

2023-05-08 Thread via GitHub
mjsax merged PR #13654: URL: https://github.com/apache/kafka/pull/13654 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] junrao commented on a diff in pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
junrao commented on code in PR #13535: URL: https://github.com/apache/kafka/pull/13535#discussion_r1187906662 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1083,48 +1095,100 @@ class ReplicaManager(val config: KafkaConfig, fetchPartitionStatus += (t

[GitHub] [kafka] cmccabe commented on a diff in pull request #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
cmccabe commented on code in PR #13686: URL: https://github.com/apache/kafka/pull/13686#discussion_r1187901148 ## shell/src/main/java/org/apache/kafka/shell/glob/GlobVisitor.java: ## @@ -93,32 +96,33 @@ public String toString() { } @Override -public void accept(M

[GitHub] [kafka] cmccabe commented on a diff in pull request #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
cmccabe commented on code in PR #13686: URL: https://github.com/apache/kafka/pull/13686#discussion_r1187899780 ## shell/src/main/java/org/apache/kafka/shell/command/FindCommandHandler.java: ## @@ -80,28 +84,28 @@ public FindCommandHandler(List paths) { } @Override -

[GitHub] [kafka] cmccabe commented on a diff in pull request #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
cmccabe commented on code in PR #13686: URL: https://github.com/apache/kafka/pull/13686#discussion_r1187898816 ## shell/src/main/java/org/apache/kafka/shell/state/MetadataShellState.java: ## @@ -0,0 +1,57 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or m

[GitHub] [kafka] cmccabe commented on a diff in pull request #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
cmccabe commented on code in PR #13686: URL: https://github.com/apache/kafka/pull/13686#discussion_r1187898447 ## metadata/src/main/java/org/apache/kafka/image/node/printer/MetadataNodeRedactionCriteria.java: ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] cmccabe commented on a diff in pull request #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
cmccabe commented on code in PR #13686: URL: https://github.com/apache/kafka/pull/13686#discussion_r1187897908 ## shell/src/main/java/org/apache/kafka/shell/node/printer/ShellNodePrinter.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] mjsax commented on a diff in pull request #13682: MINOR: improved exception/warn logging for stream-stream join store settings

2023-05-08 Thread via GitHub
mjsax commented on code in PR #13682: URL: https://github.com/apache/kafka/pull/13682#discussion_r1187897510 ## streams/src/main/java/org/apache/kafka/streams/state/internals/VersionedKeyValueToBytesStoreAdapter.java: ## @@ -115,7 +115,7 @@ public boolean persistent() { @

[jira] [Commented] (KAFKA-14918) KRaft controller sending ZK controller RPCs to KRaft brokers

2023-05-08 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720672#comment-17720672 ] Colin McCabe commented on KAFKA-14918: -- This is fixed, sorry for the delay in resol

[jira] [Resolved] (KAFKA-14918) KRaft controller sending ZK controller RPCs to KRaft brokers

2023-05-08 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14918. -- Resolution: Fixed > KRaft controller sending ZK controller RPCs to KRaft brokers > ---

[jira] [Resolved] (KAFKA-14698) Received request api key LEADER_AND_ISR which is not enabled

2023-05-08 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14698. -- Fix Version/s: (was: 3.4.1) Resolution: Duplicate > Received request api key LEADER

[GitHub] [kafka] mumrah commented on a diff in pull request #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
mumrah commented on code in PR #13686: URL: https://github.com/apache/kafka/pull/13686#discussion_r1187879810 ## shell/src/main/java/org/apache/kafka/shell/node/printer/ShellNodePrinter.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[jira] [Created] (KAFKA-14976) Left/outer stream-stream joins create KV stores that aren't customizable

2023-05-08 Thread A. Sophie Blee-Goldman (Jira)
A. Sophie Blee-Goldman created KAFKA-14976: -- Summary: Left/outer stream-stream joins create KV stores that aren't customizable Key: KAFKA-14976 URL: https://issues.apache.org/jira/browse/KAFKA-14976

[jira] [Updated] (KAFKA-14976) Left/outer stream-stream joins create KV stores that aren't customizable

2023-05-08 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-14976: --- Labels: needs-kip (was: ) > Left/outer stream-stream joins create KV stores

[GitHub] [kafka] mehbey commented on pull request #13681: KAFKA-14133: Migrate ActiveTaskCreator mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
mehbey commented on PR #13681: URL: https://github.com/apache/kafka/pull/13681#issuecomment-1538991259 Addressed Divij's comment and re-based with the latest changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] vepo commented on pull request #12809: KAFKA-14324: Upgrade RocksDB to 7.1.2

2023-05-08 Thread via GitHub
vepo commented on PR #12809: URL: https://github.com/apache/kafka/pull/12809#issuecomment-1538926727 Is there any fix for this memory leak? How can we reproduce this memory leak for testing? Some of our environment is facing a memory leak but we cannot reproduce it, as we use **_org.

[GitHub] [kafka] dajac merged pull request #13638: KAFKA-14462; [11/N] Add CurrentAssignmentBuilder

2023-05-08 Thread via GitHub
dajac merged PR #13638: URL: https://github.com/apache/kafka/pull/13638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] Scanteianu commented on pull request #13455: KAFKA-14841 Handle callbacks to ConsumerRebalanceListener in MockConsumer

2023-05-08 Thread via GitHub
Scanteianu commented on PR #13455: URL: https://github.com/apache/kafka/pull/13455#issuecomment-1538836327 > Thanks @vvcephei - Think this is the failure. ZK related so I think the PR is good to go. > > `Build / JDK 17 and Scala 2.13 / [1] Type=ZK, Name=testDualWrite, MetadataVersion

[GitHub] [kafka] mdedetrich commented on a diff in pull request #13689: KAFKA-14975: Wait for TopicBasedRemoteLogMetadataManager.ensureInitializedAndNotClosed to initialize rather than throwing

2023-05-08 Thread via GitHub
mdedetrich commented on code in PR #13689: URL: https://github.com/apache/kafka/pull/13689#discussion_r1187745268 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManager.java: ## @@ -76,6 +78,11 @@ public class TopicBasedRe

[GitHub] [kafka] jolshan commented on pull request #13608: KAFKA-14884: Include check transaction is still ongoing right before append

2023-05-08 Thread via GitHub
jolshan commented on PR #13608: URL: https://github.com/apache/kafka/pull/13608#issuecomment-1538787228 Thanks @hachikuji! Working on the tests as well, so I will push those when they are ready. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] jolshan commented on a diff in pull request #13608: KAFKA-14884: Include check transaction is still ongoing right before append

2023-05-08 Thread via GitHub
jolshan commented on code in PR #13608: URL: https://github.com/apache/kafka/pull/13608#discussion_r1187719600 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -579,9 +579,33 @@ class UnifiedLog(@volatile var logStartOffset: Long, result } - def hasOngoingTran

[GitHub] [kafka] kirktrue closed pull request #13640: KAFKA-14937: Refactoring for client code to reduce boilerplate

2023-05-08 Thread via GitHub
kirktrue closed pull request #13640: KAFKA-14937: Refactoring for client code to reduce boilerplate URL: https://github.com/apache/kafka/pull/13640 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [kafka] kirktrue commented on pull request #13640: KAFKA-14937: Refactoring for client code to reduce boilerplate

2023-05-08 Thread via GitHub
kirktrue commented on PR #13640: URL: https://github.com/apache/kafka/pull/13640#issuecomment-1538766507 Closing pull request and moving to a fork where we're doing integration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] hachikuji commented on a diff in pull request #13608: KAFKA-14884: Include check transaction is still ongoing right before append

2023-05-08 Thread via GitHub
hachikuji commented on code in PR #13608: URL: https://github.com/apache/kafka/pull/13608#discussion_r1187662801 ## storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateEntry.java: ## @@ -103,28 +121,44 @@ public boolean maybeUpdateProducerEpoch(short produ

[GitHub] [kafka] jolshan commented on a diff in pull request #13638: KAFKA-14462; [11/N] Add CurrentAssignmentBuilder

2023-05-08 Thread via GitHub
jolshan commented on code in PR #13638: URL: https://github.com/apache/kafka/pull/13638#discussion_r1187640544 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -0,0 +1,445 @@ +/* + * Licensed to the Apache Softwa

[GitHub] [kafka] philipnee commented on pull request #13664: KAFKA-14961: harden DefaultBackgroundThreadTest.testStartupAndTearDown test

2023-05-08 Thread via GitHub
philipnee commented on PR #13664: URL: https://github.com/apache/kafka/pull/13664#issuecomment-1538666147 Thanks @vvcephei - I think this is a flaky test? `Build / JDK 17 and Scala 2.13 / testDescribeStateOfExistingGroupWithRoundRobinAssignor() – kafka.admin.DescribeConsumerGroupTest

[GitHub] [kafka] philipnee commented on pull request #13455: KAFKA-14841 Handle callbacks to ConsumerRebalanceListener in MockConsumer

2023-05-08 Thread via GitHub
philipnee commented on PR #13455: URL: https://github.com/apache/kafka/pull/13455#issuecomment-1538664591 Thanks @vvcephei - Think this is the failure. ZK related so I think the PR is good to go. `Build / JDK 17 and Scala 2.13 / [1] Type=ZK, Name=testDualWrite, MetadataVersion=3.4-I

[GitHub] [kafka] dajac commented on a diff in pull request #13638: KAFKA-14462; [11/N] Add CurrentAssignmentBuilder

2023-05-08 Thread via GitHub
dajac commented on code in PR #13638: URL: https://github.com/apache/kafka/pull/13638#discussion_r1187624902 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -0,0 +1,445 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] yashmayya commented on pull request #13688: KAFKA-14974: Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread via GitHub
yashmayya commented on PR #13688: URL: https://github.com/apache/kafka/pull/13688#issuecomment-1538658594 Thanks for taking a look @rhauch and your understanding here is correct. We should try to backport this to `3.3`, `3.4` and `3.5` as well (before the `3.5.0` release ideally, if possibl

[GitHub] [kafka] mehbey commented on a diff in pull request #13681: KAFKA-14133: Migrate ActiveTaskCreator mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
mehbey commented on code in PR #13681: URL: https://github.com/apache/kafka/pull/13681#discussion_r1187611572 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -2579,7 +2563,9 @@ public void shouldUpdateInputPartitionsAfterRebalanc

[jira] [Commented] (KAFKA-14974) Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720566#comment-17720566 ] Yash Mayya commented on KAFKA-14974: Thanks [~rhauch], your understanding here is co

[GitHub] [kafka] yashmayya commented on pull request #13688: KAFKA-14974: Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread via GitHub
yashmayya commented on PR #13688: URL: https://github.com/apache/kafka/pull/13688#issuecomment-1538608185 Yep, that's right. We should backport this to `3.3`, `3.4` and `3.5` as well (before the `3.5.0` release ideally, if possible). -- This is an automated message from the Apache Git Ser

[GitHub] [kafka] reta commented on a diff in pull request #13689: KAFKA-14975: Wait for TopicBasedRemoteLogMetadataManager.ensureInitializedAndNotClosed to initialize rather than throwing

2023-05-08 Thread via GitHub
reta commented on code in PR #13689: URL: https://github.com/apache/kafka/pull/13689#discussion_r1187596223 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManager.java: ## @@ -76,6 +78,11 @@ public class TopicBasedRemoteLo

[jira] [Commented] (KAFKA-14974) Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720553#comment-17720553 ] Randall Hauch commented on KAFKA-14974: --- [~yash.mayya], just to clarify: # #12984

[GitHub] [kafka] jolshan commented on a diff in pull request #13638: KAFKA-14462; [11/N] Add CurrentAssignmentBuilder

2023-05-08 Thread via GitHub
jolshan commented on code in PR #13638: URL: https://github.com/apache/kafka/pull/13638#discussion_r1187582387 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/CurrentAssignmentBuilder.java: ## @@ -0,0 +1,445 @@ +/* + * Licensed to the Apache Softwa

[jira] [Updated] (KAFKA-14455) Kafka Connect create and update REST APIs should surface failures while writing to the config topic

2023-05-08 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya updated KAFKA-14455: --- Fix Version/s: 3.5.0 3.4.1 3.3.3 > Kafka Connect create and up

[jira] [Updated] (KAFKA-14974) Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya updated KAFKA-14974: --- Affects Version/s: 3.5.0 3.4.1 3.3.3 > Restore backwar

[jira] [Updated] (KAFKA-14974) Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-14974: -- Component/s: KafkaConnect > Restore backward compatibility in KafkaBasedLog >

[GitHub] [kafka] mdedetrich commented on a diff in pull request #13689: KAFKA-14975: Wait for TopicBasedRemoteLogMetadataManager.ensureInitializedAndNotClosed to initialize rather than throwing

2023-05-08 Thread via GitHub
mdedetrich commented on code in PR #13689: URL: https://github.com/apache/kafka/pull/13689#discussion_r1187572164 ## storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerHarness.java: ## @@ -104,27 +103,10 @@ public void on

[GitHub] [kafka] mdedetrich commented on pull request #13689: KAFKA-14975: Wait for TopicBasedRemoteLogMetadataManager.ensureInitializedAndNotClosed to initialize rather than throwing

2023-05-08 Thread via GitHub
mdedetrich commented on PR #13689: URL: https://github.com/apache/kafka/pull/13689#issuecomment-1538539482 @satishd @junrao Pinging you because you done previous PR's regarding Tiered Storage -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [kafka] mdedetrich opened a new pull request, #13689: KAFKA-14975: Wait for TopicBasedRemoteLogMetadataManager.ensureInitializedAndNotClosed to initialize rather than throwing

2023-05-08 Thread via GitHub
mdedetrich opened a new pull request, #13689: URL: https://github.com/apache/kafka/pull/13689 Current implementation of TopicBasedRemoteLogMetadataManager.ensureInitializedAndNotClosed will immediately throw an exception if TopicBasedRemoteLogMetadataManager is not initialized. Rather we s

[jira] [Commented] (KAFKA-14974) Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720546#comment-17720546 ] Randall Hauch commented on KAFKA-14974: --- Thanks for catching this and providing a

[GitHub] [kafka] yashmayya commented on pull request #13688: KAFKA-14974: Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread via GitHub
yashmayya commented on PR #13688: URL: https://github.com/apache/kafka/pull/13688#issuecomment-1538509105 @rhauch could you please take a look whenever you get a chance? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [kafka] yashmayya opened a new pull request, #13688: KAFKA-14974: Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread via GitHub
yashmayya opened a new pull request, #13688: URL: https://github.com/apache/kafka/pull/13688 From https://issues.apache.org/jira/browse/KAFKA-14974: > `KafkaBasedLog` is a widely used utility class that provides a generic implementation of a shared, compacted log of records in a Kafka

[jira] [Updated] (KAFKA-14975) Make TopicBasedRemoteLogMetadataManager methods wait for initialize to complete

2023-05-08 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew de Detrich updated KAFKA-14975: --- Description: In the current implementation of TopicBasedRemoteLogMetadataManager var

[jira] [Assigned] (KAFKA-14975) Make TopicBasedRemoteLogMetadataManager methods wait for initialize to complete

2023-05-08 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew de Detrich reassigned KAFKA-14975: -- Assignee: Matthew de Detrich > Make TopicBasedRemoteLogMetadataManager method

[jira] [Updated] (KAFKA-14975) Make TopicBasedRemoteLogMetadataManager methods wait for initialize to complete

2023-05-08 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew de Detrich updated KAFKA-14975: --- Description: In the current implementation of  > Make TopicBasedRemoteLogMetadataMan

[jira] [Created] (KAFKA-14975) Make TopicBasedRemoteLogMetadataManager methods wait for initialize to complete

2023-05-08 Thread Matthew de Detrich (Jira)
Matthew de Detrich created KAFKA-14975: -- Summary: Make TopicBasedRemoteLogMetadataManager methods wait for initialize to complete Key: KAFKA-14975 URL: https://issues.apache.org/jira/browse/KAFKA-14975

[GitHub] [kafka] viktorsomogyi commented on pull request #13594: KAFKA-14913: Using ThreadUtils.shutdownExecutorServiceQuietly to close executors in Connect Runtime

2023-05-08 Thread via GitHub
viktorsomogyi commented on PR #13594: URL: https://github.com/apache/kafka/pull/13594#issuecomment-1538474921 Merged it, thank you @vamossagar12 for the contribution! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] viktorsomogyi merged pull request #13594: KAFKA-14913: Using ThreadUtils.shutdownExecutorServiceQuietly to close executors in Connect Runtime

2023-05-08 Thread via GitHub
viktorsomogyi merged PR #13594: URL: https://github.com/apache/kafka/pull/13594 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.a

[jira] [Created] (KAFKA-14974) Restore backward compatibility in KafkaBasedLog

2023-05-08 Thread Yash Mayya (Jira)
Yash Mayya created KAFKA-14974: -- Summary: Restore backward compatibility in KafkaBasedLog Key: KAFKA-14974 URL: https://issues.apache.org/jira/browse/KAFKA-14974 Project: Kafka Issue Type: Task

[GitHub] [kafka] cadonna merged pull request #13621: KAFKA-14133: Migrate ChangeLogReader mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
cadonna merged PR #13621: URL: https://github.com/apache/kafka/pull/13621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] vamossagar12 commented on pull request #13594: KAFKA-14913: Using ThreadUtils.shutdownExecutorServiceQuietly to close executors in Connect Runtime

2023-05-08 Thread via GitHub
vamossagar12 commented on PR #13594: URL: https://github.com/apache/kafka/pull/13594#issuecomment-1538450687 Thanks @viktorsomogyi . These are the ones we could find so far. We can create follow up tickets if needed for any other executors within connect which need to be closed this way and

[GitHub] [kafka] dajac commented on a diff in pull request #13644: KAFKA-14500; [1/N] Rewrite MemberMetadata in Java

2023-05-08 Thread via GitHub
dajac commented on code in PR #13644: URL: https://github.com/apache/kafka/pull/13644#discussion_r1187462145 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/generic/GenericGroupMember.java: ## @@ -0,0 +1,499 @@ +/** + * Licensed to the Apache Software Found

[GitHub] [kafka] satishd commented on pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
satishd commented on PR #13535: URL: https://github.com/apache/kafka/pull/13535#issuecomment-1538311434 Thanks @junrao for the updated review comments. Addressed the comments with inline replies or with the latest commits. -- This is an automated message from the Apache Git Service. To r

[GitHub] [kafka] satishd commented on a diff in pull request #13535: KAFKA-9579 Fetch implementation for records in the remote storage through a specific purgatory.

2023-05-08 Thread via GitHub
satishd commented on code in PR #13535: URL: https://github.com/apache/kafka/pull/13535#discussion_r1187412666 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1083,48 +1095,100 @@ class ReplicaManager(val config: KafkaConfig, fetchPartitionStatus += (

[GitHub] [kafka] showuon commented on pull request #13684: MINOR: fix compilation failure

2023-05-08 Thread via GitHub
showuon commented on PR #13684: URL: https://github.com/apache/kafka/pull/13684#issuecomment-1538178676 It's fine sine it's for 3.4 branch, not for trunk branch. Thanks @divijvaidya -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[jira] [Created] (KAFKA-14973) Inconsistent treatment of invalid config when creating or altering a topic

2023-05-08 Thread James Shaw (Jira)
James Shaw created KAFKA-14973: -- Summary: Inconsistent treatment of invalid config when creating or altering a topic Key: KAFKA-14973 URL: https://issues.apache.org/jira/browse/KAFKA-14973 Project: Kafka

[GitHub] [kafka] showuon commented on pull request #13684: MINOR: fix compilation failure

2023-05-08 Thread via GitHub
showuon commented on PR #13684: URL: https://github.com/apache/kafka/pull/13684#issuecomment-1538085446 @mimaison , call for review to fix the build in v3.4 branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [kafka] showuon commented on pull request #13684: MINOR: fix compilation failure

2023-05-08 Thread via GitHub
showuon commented on PR #13684: URL: https://github.com/apache/kafka/pull/13684#issuecomment-1538084603 Failed tests are unrelated: ``` Build / JDK 8 and Scala 2.12 / org.apache.kafka.clients.consumer.internals.EagerConsumerCoordinatorTest.testPrepareJoinAndRejoinAfterFailedRebalan

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13681: KAFKA-14133: Migrate ActiveTaskCreator mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
divijvaidya commented on code in PR #13681: URL: https://github.com/apache/kafka/pull/13681#discussion_r1187180990 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -291,16 +292,16 @@ public void shouldPrepareActiveTaskInStateUpda

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13681: KAFKA-14133: Migrate ActiveTaskCreator mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
divijvaidya commented on code in PR #13681: URL: https://github.com/apache/kafka/pull/13681#discussion_r1187177289 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## Review Comment: we still seem to have easymock imports. Can you

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13681: KAFKA-14133: Migrate ActiveTaskCreator mock in TaskManagerTest to Mockito

2023-05-08 Thread via GitHub
divijvaidya commented on code in PR #13681: URL: https://github.com/apache/kafka/pull/13681#discussion_r1187177289 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## Review Comment: we still seem to have easymock imports. Can you

[GitHub] [kafka] cadonna commented on a diff in pull request #13656: KAFKA-14911: Add system tests for rolling upgrade path of KIP-904

2023-05-08 Thread via GitHub
cadonna commented on code in PR #13656: URL: https://github.com/apache/kafka/pull/13656#discussion_r1187176080 ## tests/kafkatest/tests/streams/streams_upgrade_test.py: ## @@ -236,6 +237,96 @@ def test_rolling_upgrade_with_2_bounces(self, from_version, to_version): s

[GitHub] [kafka] cadonna commented on a diff in pull request #13656: KAFKA-14911: Add system tests for rolling upgrade path of KIP-904

2023-05-08 Thread via GitHub
cadonna commented on code in PR #13656: URL: https://github.com/apache/kafka/pull/13656#discussion_r1187176080 ## tests/kafkatest/tests/streams/streams_upgrade_test.py: ## @@ -236,6 +237,96 @@ def test_rolling_upgrade_with_2_bounces(self, from_version, to_version): s

[GitHub] [kafka] cadonna commented on a diff in pull request #13656: KAFKA-14911: Add system tests for rolling upgrade path of KIP-904

2023-05-08 Thread via GitHub
cadonna commented on code in PR #13656: URL: https://github.com/apache/kafka/pull/13656#discussion_r1187159604 ## tests/kafkatest/tests/streams/streams_upgrade_test.py: ## @@ -236,6 +237,96 @@ def test_rolling_upgrade_with_2_bounces(self, from_version, to_version): s

[GitHub] [kafka] showuon opened a new pull request, #13687: MINOR: test

2023-05-08 Thread via GitHub
showuon opened a new pull request, #13687: URL: https://github.com/apache/kafka/pull/13687 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[GitHub] [kafka] cmccabe opened a new pull request, #13686: MINOR: Create the MetadataNode classes to introspect MetadataImage

2023-05-08 Thread via GitHub
cmccabe opened a new pull request, #13686: URL: https://github.com/apache/kafka/pull/13686 Metadata image classes such as MetadataImage, ClusterImage, FeaturesImage, and so forth contain numerous sub-images. This PR adds a structured way of traversing those sub-images. This is useful for th

[GitHub] [kafka] showuon closed pull request #13685: test

2023-05-08 Thread via GitHub
showuon closed pull request #13685: test URL: https://github.com/apache/kafka/pull/13685 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr..

[GitHub] [kafka] showuon opened a new pull request, #13685: test

2023-05-08 Thread via GitHub
showuon opened a new pull request, #13685: URL: https://github.com/apache/kafka/pull/13685 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[GitHub] [kafka] showuon opened a new pull request, #13684: MINOR: fix compilation failure

2023-05-08 Thread via GitHub
showuon opened a new pull request, #13684: URL: https://github.com/apache/kafka/pull/13684 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing