[GitHub] [kafka] srishti-saraswat opened a new pull request, #12621: CC-17475 Migrate connect system tests to KRaft

2022-09-12 Thread GitBox
srishti-saraswat opened a new pull request, #12621: URL: https://github.com/apache/kafka/pull/12621 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary o

[jira] [Commented] (KAFKA-14218) replace temp file handler with JUnit 5 Temporary Directory Support

2022-09-12 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603003#comment-17603003 ] Divij Vaidya commented on KAFKA-14218: -- While working on this task, the owner must

[GitHub] [kafka] clolov commented on a diff in pull request #12607: KAFKA-14133: Replace EasyMock with Mockito in streams tests

2022-09-12 Thread GitBox
clolov commented on code in PR #12607: URL: https://github.com/apache/kafka/pull/12607#discussion_r968200658 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -325,25 +318,21 @@ public void shouldHandleRemovedTasksToRecycleFromSta

[GitHub] [kafka] clolov commented on a diff in pull request #12607: KAFKA-14133: Replace EasyMock with Mockito in streams tests

2022-09-12 Thread GitBox
clolov commented on code in PR #12607: URL: https://github.com/apache/kafka/pull/12607#discussion_r968204971 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -1300,28 +1229,16 @@ public void shouldCloseActiveTasksWhenHandlingLost

[GitHub] [kafka] clolov commented on a diff in pull request #12607: KAFKA-14133: Replace EasyMock with Mockito in streams tests

2022-09-12 Thread GitBox
clolov commented on code in PR #12607: URL: https://github.com/apache/kafka/pull/12607#discussion_r968206590 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -4091,21 +3815,20 @@ public void shouldListNotPausedTasks() { t

[GitHub] [kafka] clolov commented on a diff in pull request #12607: KAFKA-14133: Replace EasyMock with Mockito in streams tests

2022-09-12 Thread GitBox
clolov commented on code in PR #12607: URL: https://github.com/apache/kafka/pull/12607#discussion_r968206590 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -4091,21 +3815,20 @@ public void shouldListNotPausedTasks() { t

[GitHub] [kafka] cadonna commented on a diff in pull request #12600: KAFKA-10199: Suspend tasks in the state updater on revocation

2022-09-12 Thread GitBox
cadonna commented on code in PR #12600: URL: https://github.com/apache/kafka/pull/12600#discussion_r968207039 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -960,7 +1011,7 @@ private void removeRevokedTasksFromStateUpdater(final Se

[GitHub] [kafka] clolov commented on a diff in pull request #12607: KAFKA-14133: Replace EasyMock with Mockito in streams tests

2022-09-12 Thread GitBox
clolov commented on code in PR #12607: URL: https://github.com/apache/kafka/pull/12607#discussion_r968207227 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -271,11 +265,13 @@ public void shouldClassifyExistingTasksWithStateUpdat

[GitHub] [kafka] clolov commented on a diff in pull request #12607: KAFKA-14133: Replace EasyMock with Mockito in streams tests

2022-09-12 Thread GitBox
clolov commented on code in PR #12607: URL: https://github.com/apache/kafka/pull/12607#discussion_r968208668 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -2180,29 +2035,16 @@ public void shouldCommitAllActiveTasksThatNeedComm

[GitHub] [kafka] clolov commented on a diff in pull request #12607: KAFKA-14133: Replace EasyMock with Mockito in streams tests

2022-09-12 Thread GitBox
clolov commented on code in PR #12607: URL: https://github.com/apache/kafka/pull/12607#discussion_r968209811 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -4068,18 +3796,14 @@ public void shouldConvertStandbyTaskToActiveTask()

[jira] [Assigned] (KAFKA-14206) Upgrade zookeeper to 3.7.1 to address security vulnerabilities

2022-09-12 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen reassigned KAFKA-14206: - Assignee: Luke Chen > Upgrade zookeeper to 3.7.1 to address security vulnerabilities >

[GitHub] [kafka] mdedetrich commented on pull request #12284: KAFKA-13980: Upgrade from Scala 2.12.15 to 2.12.16

2022-09-12 Thread GitBox
mdedetrich commented on PR #12284: URL: https://github.com/apache/kafka/pull/12284#issuecomment-1243496402 @ijuma @mimaison Any chance of a review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] mdedetrich commented on pull request #12524: KAFKA-14133: Replace EasyMock with Mockito in streams test

2022-09-12 Thread GitBox
mdedetrich commented on PR #12524: URL: https://github.com/apache/kafka/pull/12524#issuecomment-1243525214 @cadonna I have just rebased the PR with the latest streams changes that got merged into. trunk. I also did some more improvements for `StreamThreadTest` with the usage of `MockitoJUni

[GitHub] [kafka] srishti-saraswat closed pull request #12567: Migrate connect system tests to KRaft

2022-09-12 Thread GitBox
srishti-saraswat closed pull request #12567: Migrate connect system tests to KRaft URL: https://github.com/apache/kafka/pull/12567 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [kafka] Kaiserchen commented on pull request #12570: KAFKA-14156: Built-in partitioner may create suboptimal batches

2022-09-12 Thread GitBox
Kaiserchen commented on PR #12570: URL: https://github.com/apache/kafka/pull/12570#issuecomment-1243614928 Just stumbled over this. Our company basically runs similar code. Without this optimization MM is non functional for us :) -- This is an automated message from the Apache Git Se

[jira] [Created] (KAFKA-14220) "partition-count" not getting updated after revocation of partitions in case of Incremental Co-operative rebalancing.

2022-09-12 Thread Pritam Kumar (Jira)
Pritam Kumar created KAFKA-14220: Summary: "partition-count" not getting updated after revocation of partitions in case of Incremental Co-operative rebalancing. Key: KAFKA-14220 URL: https://issues.apache.org/jira

[GitHub] [kafka] kumarpritam863 opened a new pull request, #12622: Update WorkerSinkTask.java

2022-09-12 Thread GitBox
kumarpritam863 opened a new pull request, #12622: URL: https://github.com/apache/kafka/pull/12622 In case of the revocation of partitions, the updation of "**partition count**" metrics is being done before updating the new set of assignments. "**invokePartitionsRevoked**" method of "**onJoi

[jira] [Commented] (KAFKA-14220) "partition-count" not getting updated after revocation of partitions in case of Incremental Co-operative rebalancing.

2022-09-12 Thread Pritam Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603056#comment-17603056 ] Pritam Kumar commented on KAFKA-14220: -- The following is the patch made and verifie

[GitHub] [kafka] C0urante commented on a diff in pull request #12616: KAFKA-14198: Define separate swagger configuration in Gradle build

2022-09-12 Thread GitBox
C0urante commented on code in PR #12616: URL: https://github.com/apache/kafka/pull/12616#discussion_r968359263 ## build.gradle: ## @@ -2564,7 +2568,9 @@ project(':connect:runtime') { implementation libs.mavenArtifact implementation libs.swaggerAnnotations Review Comme

[GitHub] [kafka] dajac commented on pull request #12603: KAFKA-14196: Pausing partition to prevent duplication when autocommit is enabled

2022-09-12 Thread GitBox
dajac commented on PR #12603: URL: https://github.com/apache/kafka/pull/12603#issuecomment-1243677992 @philipnee Thanks for the PR. Could you please update the description to explain the change? -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] C0urante commented on a diff in pull request #12616: KAFKA-14198: Define separate swagger configuration in Gradle build

2022-09-12 Thread GitBox
C0urante commented on code in PR #12616: URL: https://github.com/apache/kafka/pull/12616#discussion_r968366903 ## build.gradle: ## @@ -2653,9 +2659,9 @@ project(':connect:runtime') { } task genConnectOpenAPIDocs(type: io.swagger.v3.plugins.gradle.tasks.ResolveTask, depe

[GitHub] [kafka] C0urante commented on pull request #12616: KAFKA-14198: Define separate swagger configuration in Gradle build

2022-09-12 Thread GitBox
C0urante commented on PR #12616: URL: https://github.com/apache/kafka/pull/12616#issuecomment-1243683309 Thanks @ijuma. Yes, I verified both, and updated the PR description with the steps I took to do so. -- This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [kafka] divijvaidya opened a new pull request, #12623: MINOR: Fix ConsumerPerformanceTest to work with non-default locales

2022-09-12 Thread GitBox
divijvaidya opened a new pull request, #12623: URL: https://github.com/apache/kafka/pull/12623 **Problem** `ConsumerPerformanceTest` fails when running on a machine with a Locale where decimal is represented by a comma. E.g. in locale of Germany, one point 2 is written as `1,2` and not `

[GitHub] [kafka] cadonna commented on a diff in pull request #12600: KAFKA-10199: Suspend tasks in the state updater on revocation

2022-09-12 Thread GitBox
cadonna commented on code in PR #12600: URL: https://github.com/apache/kafka/pull/12600#discussion_r968378813 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/Tasks.java: ## @@ -52,11 +53,8 @@ class Tasks implements TasksRegistry { // we receive a new a

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-12 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r968381069 ## clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java: ## @@ -2403,17 +2404,40 @@ private ClusterResourceListeners configureClusterResourceL

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-12 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r968386276 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -446,6 +429,31 @@ private RequestFuture sendMetadataRequest(MetadataRequest

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-12 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r968387276 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -446,6 +429,31 @@ private RequestFuture sendMetadataRequest(MetadataRequest

[GitHub] [kafka] ijuma commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-12 Thread GitBox
ijuma commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r968392833 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -446,6 +429,31 @@ private RequestFuture sendMetadataRequest(MetadataRequest.Build

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-12 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r968407998 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -1933,11 +1941,76 @@ private Map topicPartitionTags(TopicPartition tp) {

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-12 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r968439103 ## clients/src/test/java/org/apache/kafka/common/security/authenticator/ClientAuthenticationFailureTest.java: ## @@ -90,12 +90,12 @@ public void testConsumerWithInva

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-12 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r968439965 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -1933,11 +1941,76 @@ private Map topicPartitionTags(TopicPartition tp) {

[GitHub] [kafka] cadonna commented on a diff in pull request #12600: KAFKA-10199: Suspend tasks in the state updater on revocation

2022-09-12 Thread GitBox
cadonna commented on code in PR #12600: URL: https://github.com/apache/kafka/pull/12600#discussion_r968441157 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -421,73 +421,96 @@ private void classifyTasksWithoutStateUpdater(final Map

[GitHub] [kafka] cadonna commented on a diff in pull request #12600: KAFKA-10199: Suspend tasks in the state updater on revocation

2022-09-12 Thread GitBox
cadonna commented on code in PR #12600: URL: https://github.com/apache/kafka/pull/12600#discussion_r968443807 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -960,7 +1011,7 @@ private void removeRevokedTasksFromStateUpdater(final Se

[GitHub] [kafka] ijuma commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-12 Thread GitBox
ijuma commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r968463099 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -1933,11 +1941,76 @@ private Map topicPartitionTags(TopicPartition tp) {

[jira] [Commented] (KAFKA-14184) Kafka streams application crashes due to "UnsupportedOperationException: this should not happen: timestamp() is not supported in standby tasks."

2022-09-12 Thread Suresh Rukmangathan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603102#comment-17603102 ] Suresh Rukmangathan commented on KAFKA-14184: - [~cadonna] - We are yet to tr

[GitHub] [kafka] C0urante merged pull request #12475: MINOR; Update scalafmt to latest version

2022-09-12 Thread GitBox
C0urante merged PR #12475: URL: https://github.com/apache/kafka/pull/12475 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] C0urante commented on pull request #12616: KAFKA-14198: Define separate swagger configuration in Gradle build

2022-09-12 Thread GitBox
C0urante commented on PR #12616: URL: https://github.com/apache/kafka/pull/12616#issuecomment-1243801973 Thanks Ismael. Kicked off another Jenkins build since the previous one failed with a disk space error; will merge if everything looks alright. -- This is an automated message from the

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-12 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r968480496 ## clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java: ## @@ -2403,17 +2404,40 @@ private ClusterResourceListeners configureClusterResourceL

[GitHub] [kafka] divijvaidya commented on a diff in pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2022-09-12 Thread GitBox
divijvaidya commented on code in PR #12590: URL: https://github.com/apache/kafka/pull/12590#discussion_r968491064 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -1933,11 +1941,76 @@ private Map topicPartitionTags(TopicPartition tp) {

[GitHub] [kafka] junrao commented on pull request #12570: KAFKA-14156: Built-in partitioner may create suboptimal batches

2022-09-12 Thread GitBox
junrao commented on PR #12570: URL: https://github.com/apache/kafka/pull/12570#issuecomment-1243846932 Thanks for the feedback @Kaiserchen . Could you run your MM test by changing `last == null || last.isFull()` to `deque.size() > 1 || last == null || last.isFull()` and see if it's still fu

[GitHub] [kafka] divijvaidya commented on pull request #12591: MINOR: Replace usage of File.createTempFile() with TestUtils.tempFile()

2022-09-12 Thread GitBox
divijvaidya commented on PR #12591: URL: https://github.com/apache/kafka/pull/12591#issuecomment-1243849390 Thank you both for your review. Since, we agree up to pick up this incremental change, I am assuming no action is required from my side before you approve & merge? Note t

[GitHub] [kafka] cadonna commented on a diff in pull request #12524: KAFKA-14133: Replace EasyMock with Mockito in streams test

2022-09-12 Thread GitBox
cadonna commented on code in PR #12524: URL: https://github.com/apache/kafka/pull/12524#discussion_r968474491 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/ProcessorStateManagerTest.java: ## @@ -308,29 +300,21 @@ public void shouldRestoreTimestampedStoreW

[jira] [Created] (KAFKA-14221) Update Apache Kafka JVM version in CI to latest versions

2022-09-12 Thread Matthew de Detrich (Jira)
Matthew de Detrich created KAFKA-14221: -- Summary: Update Apache Kafka JVM version in CI to latest versions Key: KAFKA-14221 URL: https://issues.apache.org/jira/browse/KAFKA-14221 Project: Kafka

[jira] [Updated] (KAFKA-14221) Update Apache Kafka JVM version in CI to latest versions

2022-09-12 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew de Detrich updated KAFKA-14221: --- Description: In a recent test (see [https://ci-builds.apache.org/blue/organizations

[jira] [Commented] (KAFKA-14221) Update Apache Kafka JVM version in CI to latest versions

2022-09-12 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603151#comment-17603151 ] Matthew de Detrich commented on KAFKA-14221: [~ijuma] Maybe you have some id

[jira] [Updated] (KAFKA-14221) Update Apache Kafka JVM version in CI to latest versions

2022-09-12 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew de Detrich updated KAFKA-14221: --- Description: In a recent test (see [https://ci-builds.apache.org/blue/organizations

[GitHub] [kafka] cmccabe merged pull request #12618: KAFKA-14217: app-reset-tool.html should remove reference to --zookeeper flag that no longer exists

2022-09-12 Thread GitBox
cmccabe merged PR #12618: URL: https://github.com/apache/kafka/pull/12618 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] cmccabe merged pull request #12617: KAFKA-14216: Remove ZK reference from org.apache.kafka.server.quota.ClientQuotaCallback javadoc

2022-09-12 Thread GitBox
cmccabe merged PR #12617: URL: https://github.com/apache/kafka/pull/12617 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Resolved] (KAFKA-14217) app-reset-tool.html should remove reference to --zookeeper flag that no longer exists

2022-09-12 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14217. -- Fix Version/s: 3.3 Reviewer: Luke Chen Resolution: Fixed > app-reset-tool.html

[jira] [Resolved] (KAFKA-14216) Remove ZK reference from org.apache.kafka.server.quota.ClientQuotaCallback javadoc

2022-09-12 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14216. -- Fix Version/s: 3.3 Reviewer: Luke Chen Resolution: Fixed > Remove ZK reference

[GitHub] [kafka] mdedetrich commented on pull request #12524: KAFKA-14133: Replace EasyMock with Mockito in streams test

2022-09-12 Thread GitBox
mdedetrich commented on PR #12524: URL: https://github.com/apache/kafka/pull/12524#issuecomment-1243928456 Thanks for the comments, I am still get my head around the tests and what precisely is being tested. I forgot to mention earlier that `StreamTaskTest` needs more work (doing thi

[GitHub] [kafka] jsancio commented on a diff in pull request #12596: KAFKA-14203 Don't make snapshots on broker after metadata errors

2022-09-12 Thread GitBox
jsancio commented on code in PR #12596: URL: https://github.com/apache/kafka/pull/12596#discussion_r968614919 ## core/src/main/scala/kafka/server/metadata/BrokerMetadataListener.scala: ## @@ -325,7 +325,7 @@ class BrokerMetadataListener( try { _image = _delta.apply()

[jira] [Commented] (KAFKA-14221) Update Apache Kafka JVM version in CI to latest versions

2022-09-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603170#comment-17603170 ] Ismael Juma commented on KAFKA-14221: - We are running the latest JDK 17 build availa

[jira] [Commented] (KAFKA-14221) Update Apache Kafka JVM version in CI to latest versions

2022-09-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603173#comment-17603173 ] Ismael Juma commented on KAFKA-14221: - Actually, it seems that we may want to switch

[jira] [Comment Edited] (KAFKA-14221) Update Apache Kafka JVM version in CI to latest versions

2022-09-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603173#comment-17603173 ] Ismael Juma edited comment on KAFKA-14221 at 9/12/22 4:21 PM:

[GitHub] [kafka] C0urante commented on pull request #12616: KAFKA-14198: Define separate swagger configuration in Gradle build

2022-09-12 Thread GitBox
C0urante commented on PR #12616: URL: https://github.com/apache/kafka/pull/12616#issuecomment-1243994114 Failures appear unrelated, merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [kafka] C0urante merged pull request #12616: KAFKA-14198: Define separate swagger configuration in Gradle build

2022-09-12 Thread GitBox
C0urante merged PR #12616: URL: https://github.com/apache/kafka/pull/12616 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] C0urante commented on pull request #12616: KAFKA-14198: Define separate swagger configuration in Gradle build

2022-09-12 Thread GitBox
C0urante commented on PR #12616: URL: https://github.com/apache/kafka/pull/12616#issuecomment-1243995392 @ijuma @jsancio I'm assuming we don't need to backport this to 3.3 since the immediate issues with Swagger deps and Connect OpenAPI docs generation have already been fixed on that branch

[GitHub] [kafka] jolshan commented on a diff in pull request #12392: KAFKA-14053: Transactional producer should bump the epoch and skip ab…

2022-09-12 Thread GitBox
jolshan commented on code in PR #12392: URL: https://github.com/apache/kafka/pull/12392#discussion_r968682161 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/TransactionManagerTest.java: ## @@ -2594,27 +2596,20 @@ public void testDropCommitOnBatchExpiry() th

[GitHub] [kafka] hachikuji opened a new pull request, #12624: KAFKA-14215; Ensure forwarded requests are applied to broker request quota

2022-09-12 Thread GitBox
hachikuji opened a new pull request, #12624: URL: https://github.com/apache/kafka/pull/12624 Currently forwarded requests are not applied to any quotas on either the controller or the broker. The controller-side throttling requires the controller to apply the quota changes from the log to t

[jira] [Created] (KAFKA-14222) Exhausted BatchMemoryPool

2022-09-12 Thread Jose Armando Garcia Sancio (Jira)
Jose Armando Garcia Sancio created KAFKA-14222: -- Summary: Exhausted BatchMemoryPool Key: KAFKA-14222 URL: https://issues.apache.org/jira/browse/KAFKA-14222 Project: Kafka Issue T

[GitHub] [kafka] hachikuji commented on a diff in pull request #12603: KAFKA-14196: Pausing partition to prevent duplication when autocommit is enabled

2022-09-12 Thread GitBox
hachikuji commented on code in PR #12603: URL: https://github.com/apache/kafka/pull/12603#discussion_r968754388 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java: ## @@ -769,6 +773,7 @@ private static class TopicPartitionState {

[GitHub] [kafka] hachikuji commented on a diff in pull request #12611: KAFKA-14208: Should not wake-up with non-blocking coordinator discovery

2022-09-12 Thread GitBox
hachikuji commented on code in PR #12611: URL: https://github.com/apache/kafka/pull/12611#discussion_r968778503 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java: ## @@ -249,7 +249,14 @@ protected synchronized boolean ensureCoordinato

[GitHub] [kafka] hachikuji commented on a diff in pull request #12611: KAFKA-14208: Should not wake-up with non-blocking coordinator discovery

2022-09-12 Thread GitBox
hachikuji commented on code in PR #12611: URL: https://github.com/apache/kafka/pull/12611#discussion_r968778503 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java: ## @@ -249,7 +249,14 @@ protected synchronized boolean ensureCoordinato

[GitHub] [kafka] philipnee commented on a diff in pull request #12603: KAFKA-14196: Pausing partition to prevent duplication when autocommit is enabled

2022-09-12 Thread GitBox
philipnee commented on code in PR #12603: URL: https://github.com/apache/kafka/pull/12603#discussion_r968822443 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -799,64 +804,95 @@ protected boolean onJoinPrepare(Timer timer, in

[GitHub] [kafka] jsancio opened a new pull request, #12625: KAFKA-14222; KRaft's memory pool should always allocate a buffer

2022-09-12 Thread GitBox
jsancio opened a new pull request, #12625: URL: https://github.com/apache/kafka/pull/12625 Because the snapshot writer sets a linger ms of Integer.MAX_VALUE it is possible for the memory pool to run out of memory if the snapshot is greater than 5 * 8MB. This change allows the BatchMe

[GitHub] [kafka] jsancio commented on a diff in pull request #12596: KAFKA-14203 Don't make snapshots on broker after metadata errors

2022-09-12 Thread GitBox
jsancio commented on code in PR #12596: URL: https://github.com/apache/kafka/pull/12596#discussion_r968872243 ## core/src/main/scala/kafka/server/metadata/BrokerMetadataListener.scala: ## @@ -325,7 +325,7 @@ class BrokerMetadataListener( try { _image = _delta.apply()

[GitHub] [kafka] hachikuji opened a new pull request, #12626: KAFKA-14208; Do not raise wakeup in consumer during asynchronous offset commits

2022-09-12 Thread GitBox
hachikuji opened a new pull request, #12626: URL: https://github.com/apache/kafka/pull/12626 Asynchronous offset commits may throw an unexpected `WakeupException` following https://github.com/apache/kafka/pull/11631 and https://github.com/apache/kafka/pull/12244. This patch fixes the proble

[GitHub] [kafka] hachikuji commented on pull request #12611: KAFKA-14208: Should not wake-up with non-blocking coordinator discovery

2022-09-12 Thread GitBox
hachikuji commented on PR #12611: URL: https://github.com/apache/kafka/pull/12611#issuecomment-1244352156 @showuon Since Guozhang is out sick, I raised https://github.com/apache/kafka/pull/12626 which addresses your (and my) comments. Please take a look. I'll leave this open for now. --

[GitHub] [kafka] hachikuji commented on a diff in pull request #12603: KAFKA-14196: Pausing partition to prevent duplication when autocommit is enabled

2022-09-12 Thread GitBox
hachikuji commented on code in PR #12603: URL: https://github.com/apache/kafka/pull/12603#discussion_r968897177 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -799,64 +804,95 @@ protected boolean onJoinPrepare(Timer timer, in

[GitHub] [kafka] mumrah commented on a diff in pull request #12596: KAFKA-14203 Don't make snapshots on broker after metadata errors

2022-09-12 Thread GitBox
mumrah commented on code in PR #12596: URL: https://github.com/apache/kafka/pull/12596#discussion_r968903902 ## core/src/main/scala/kafka/server/metadata/BrokerMetadataListener.scala: ## @@ -325,7 +325,7 @@ class BrokerMetadataListener( try { _image = _delta.apply()

[GitHub] [kafka] philipnee commented on a diff in pull request #12603: KAFKA-14196: Pausing partition to prevent duplication when autocommit is enabled

2022-09-12 Thread GitBox
philipnee commented on code in PR #12603: URL: https://github.com/apache/kafka/pull/12603#discussion_r968910809 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -756,10 +756,15 @@ protected boolean onJoinPrepare(Timer timer, in

[jira] [Created] (KAFKA-14223) Update jdk_17_latest to adoptium 17.0.4+1

2022-09-12 Thread Matthew de Detrich (Jira)
Matthew de Detrich created KAFKA-14223: -- Summary: Update jdk_17_latest to adoptium 17.0.4+1 Key: KAFKA-14223 URL: https://issues.apache.org/jira/browse/KAFKA-14223 Project: Kafka Issue T

[jira] [Commented] (KAFKA-14221) Update Apache Kafka JVM version in CI to latest versions

2022-09-12 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603260#comment-17603260 ] Matthew de Detrich commented on KAFKA-14221: Thanks, INFRA ticket created >

[GitHub] [kafka] mumrah merged pull request #12596: KAFKA-14203 Don't make snapshots on broker after metadata errors

2022-09-12 Thread GitBox
mumrah merged PR #12596: URL: https://github.com/apache/kafka/pull/12596 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.o

[jira] [Resolved] (KAFKA-14223) Update jdk_17_latest to adoptium 17.0.4+1

2022-09-12 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew de Detrich resolved KAFKA-14223. Resolution: Invalid Created in wrong project > Update jdk_17_latest to adoptium 1

[jira] [Comment Edited] (KAFKA-14221) Update Apache Kafka JVM version in CI to latest versions

2022-09-12 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603260#comment-17603260 ] Matthew de Detrich edited comment on KAFKA-14221 at 9/12/22 9:49 PM: -

[GitHub] [kafka] mumrah commented on a diff in pull request #12624: KAFKA-14215; Ensure forwarded requests are applied to broker request quota

2022-09-12 Thread GitBox
mumrah commented on code in PR #12624: URL: https://github.com/apache/kafka/pull/12624#discussion_r968961932 ## clients/src/main/java/org/apache/kafka/common/requests/AbstractResponse.java: ## @@ -268,6 +268,8 @@ public ApiKeys apiKey() { public abstract int throttleTimeM

[jira] [Comment Edited] (KAFKA-14221) Update Apache Kafka JVM version in CI to latest versions

2022-09-12 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603260#comment-17603260 ] Matthew de Detrich edited comment on KAFKA-14221 at 9/12/22 9:52 PM: -

[GitHub] [kafka] hachikuji commented on a diff in pull request #12603: KAFKA-14196: Pausing partition to prevent duplication when autocommit is enabled

2022-09-12 Thread GitBox
hachikuji commented on code in PR #12603: URL: https://github.com/apache/kafka/pull/12603#discussion_r968922937 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -799,64 +804,95 @@ protected boolean onJoinPrepare(Timer timer, in

[GitHub] [kafka] hachikuji commented on a diff in pull request #12624: KAFKA-14215; Ensure forwarded requests are applied to broker request quota

2022-09-12 Thread GitBox
hachikuji commented on code in PR #12624: URL: https://github.com/apache/kafka/pull/12624#discussion_r968967605 ## clients/src/main/java/org/apache/kafka/common/requests/AbstractResponse.java: ## @@ -268,6 +268,8 @@ public ApiKeys apiKey() { public abstract int throttleTi

[GitHub] [kafka] hachikuji commented on a diff in pull request #12603: KAFKA-14196: Pausing partition to prevent duplication when autocommit is enabled

2022-09-12 Thread GitBox
hachikuji commented on code in PR #12603: URL: https://github.com/apache/kafka/pull/12603#discussion_r968965173 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -799,64 +804,92 @@ protected boolean onJoinPrepare(Timer timer, in

[GitHub] [kafka] hachikuji commented on pull request #12624: KAFKA-14215; Ensure forwarded requests are applied to broker request quota

2022-09-12 Thread GitBox
hachikuji commented on PR #12624: URL: https://github.com/apache/kafka/pull/12624#issuecomment-1244563584 @mumrah Thanks for reviewing. Note that the controller will not actually be applying the throttles in the future since it does not have a direct client connection to throttle. The inten

[GitHub] [kafka] philipnee commented on a diff in pull request #12603: KAFKA-14196: Pausing partition to prevent duplication when autocommit is enabled

2022-09-12 Thread GitBox
philipnee commented on code in PR #12603: URL: https://github.com/apache/kafka/pull/12603#discussion_r968970980 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -799,64 +804,92 @@ protected boolean onJoinPrepare(Timer timer, in

[GitHub] [kafka] hachikuji commented on a diff in pull request #12603: KAFKA-14196: Pausing partition to prevent duplication when autocommit is enabled

2022-09-12 Thread GitBox
hachikuji commented on code in PR #12603: URL: https://github.com/apache/kafka/pull/12603#discussion_r968965173 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -799,64 +804,92 @@ protected boolean onJoinPrepare(Timer timer, in

[GitHub] [kafka] jnh5y commented on a diff in pull request #12555: Optimize self-join

2022-09-12 Thread GitBox
jnh5y commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r968974016 ## streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java: ## @@ -1644,6 +1668,29 @@ private Map getClientCustomProps() { return props; } +p

[GitHub] [kafka] jnh5y commented on a diff in pull request #12555: Optimize self-join

2022-09-12 Thread GitBox
jnh5y commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r968974464 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -270,17 +277,12 @@ private void maybeAddNodeForOptimizationMetadata(

[GitHub] [kafka] akhileshchg opened a new pull request, #12627: KGLOBAL-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads.

2022-09-12 Thread GitBox
akhileshchg opened a new pull request, #12627: URL: https://github.com/apache/kafka/pull/12627 KGLOBAL-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads. The issue with StandardAuthorizer#authorize is, that it looks up aclsByResources (which is of type C

[GitHub] [kafka] jnh5y commented on a diff in pull request #12555: Optimize self-join

2022-09-12 Thread GitBox
jnh5y commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r968982353 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -305,6 +307,36 @@ public void buildAndOptimizeTopology(final boolean

[GitHub] [kafka] jnh5y commented on a diff in pull request #12555: Optimize self-join

2022-09-12 Thread GitBox
jnh5y commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r968986084 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -356,6 +381,95 @@ private void mergeDuplicateSourceNodes() {

[GitHub] [kafka] hachikuji commented on a diff in pull request #12625: KAFKA-14222; KRaft's memory pool should always allocate a buffer

2022-09-12 Thread GitBox
hachikuji commented on code in PR #12625: URL: https://github.com/apache/kafka/pull/12625#discussion_r968988957 ## raft/src/test/java/org/apache/kafka/raft/internals/BatchMemoryPoolTest.java: ## @@ -104,4 +128,15 @@ public void testReleaseBufferNotMatchingBatchSize() {

[GitHub] [kafka] akhileshchg closed pull request #12627: KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads.

2022-09-12 Thread GitBox
akhileshchg closed pull request #12627: KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads. URL: https://github.com/apache/kafka/pull/12627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [kafka] jnh5y commented on a diff in pull request #12555: Optimize self-join

2022-09-12 Thread GitBox
jnh5y commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r968993197 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamSelfJoin.java: ## @@ -0,0 +1,172 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] jsancio commented on a diff in pull request #12625: KAFKA-14222; KRaft's memory pool should always allocate a buffer

2022-09-12 Thread GitBox
jsancio commented on code in PR #12625: URL: https://github.com/apache/kafka/pull/12625#discussion_r968998063 ## raft/src/main/java/org/apache/kafka/raft/internals/BatchMemoryPool.java: ## @@ -72,7 +74,13 @@ public void release(ByteBuffer previouslyAllocated) {

[GitHub] [kafka] jsancio commented on a diff in pull request #12625: KAFKA-14222; KRaft's memory pool should always allocate a buffer

2022-09-12 Thread GitBox
jsancio commented on code in PR #12625: URL: https://github.com/apache/kafka/pull/12625#discussion_r968998214 ## raft/src/test/java/org/apache/kafka/raft/internals/BatchMemoryPoolTest.java: ## @@ -104,4 +128,15 @@ public void testReleaseBufferNotMatchingBatchSize() { as

[GitHub] [kafka] jsancio commented on a diff in pull request #12625: KAFKA-14222; KRaft's memory pool should always allocate a buffer

2022-09-12 Thread GitBox
jsancio commented on code in PR #12625: URL: https://github.com/apache/kafka/pull/12625#discussion_r968998338 ## raft/src/main/java/org/apache/kafka/raft/internals/BatchMemoryPool.java: ## @@ -90,18 +98,12 @@ public long size() { @Override public long availableMemory

[GitHub] [kafka] akhileshchg opened a new pull request, #12628: KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads.

2022-09-12 Thread GitBox
akhileshchg opened a new pull request, #12628: URL: https://github.com/apache/kafka/pull/12628 KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads. The issue with StandardAuthorizer#authorize is, that it looks up aclsByResources (which is of type C

[GitHub] [kafka] mumrah commented on a diff in pull request #12628: KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads.

2022-09-12 Thread GitBox
mumrah commented on code in PR #12628: URL: https://github.com/apache/kafka/pull/12628#discussion_r969006371 ## metadata/src/main/java/org/apache/kafka/metadata/authorizer/StandardAuthorizerData.java: ## @@ -534,49 +535,19 @@ Iterable acls(AclBindingFilter filter) { }

[GitHub] [kafka] akhileshchg commented on a diff in pull request #12628: KAFKA-14214: Introduce read-write lock to StandardAuthorizer for consistent ACL reads.

2022-09-12 Thread GitBox
akhileshchg commented on code in PR #12628: URL: https://github.com/apache/kafka/pull/12628#discussion_r969009047 ## metadata/src/main/java/org/apache/kafka/metadata/authorizer/StandardAuthorizerData.java: ## @@ -534,49 +535,19 @@ Iterable acls(AclBindingFilter filter) { }

[GitHub] [kafka] hachikuji commented on a diff in pull request #12603: KAFKA-14196: Pausing partition to prevent duplication when autocommit is enabled

2022-09-12 Thread GitBox
hachikuji commented on code in PR #12603: URL: https://github.com/apache/kafka/pull/12603#discussion_r969009257 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -756,12 +756,11 @@ protected boolean onJoinPrepare(Timer timer, in

  1   2   >