[jira] [Created] (KAFKA-14156) Built-in partitioner may create suboptimal batches with large linger.ms

Artem Livshits created KAFKA-14156: -- Summary: Built-in partitioner may create suboptimal batches with large linger.ms Key: KAFKA-14156 URL: https://issues.apache.org/jira/browse/KAFKA-14156 Project:

[GitHub] [kafka] ijuma commented on a diff in pull request #12498: KAFKA-13986; Brokers should include node.id in fetches to metadata quorum

ijuma commented on code in PR #12498: URL: https://github.com/apache/kafka/pull/12498#discussion_r942043740 ## core/src/test/scala/unit/kafka/raft/RaftManagerTest.scala: ## @@ -84,21 +84,21 @@ class RaftManagerTest { @Test def testSentinelNodeIdIfBrokerRoleOnly(): Unit = {

[jira] [Created] (KAFKA-14155) Kafka Stream - State transition from RUNNING to ERROR

Harsha Nadig created KAFKA-14155: Summary: Kafka Stream - State transition from RUNNING to ERROR Key: KAFKA-14155 URL: https://issues.apache.org/jira/browse/KAFKA-14155 Project: Kafka Issue T

[GitHub] [kafka] andymg3 commented on a diff in pull request #12499: KAFKA-14154; Ensure AlterPartition not sent to stale controller

andymg3 commented on code in PR #12499: URL: https://github.com/apache/kafka/pull/12499#discussion_r941967874 ## core/src/main/scala/kafka/server/BrokerToControllerChannelManager.scala: ## @@ -364,15 +423,15 @@ class BrokerToControllerRequestThread( } override def doWork

[GitHub] [kafka] andymg3 commented on a diff in pull request #12499: KAFKA-14154; Ensure AlterPartition not sent to stale controller

andymg3 commented on code in PR #12499: URL: https://github.com/apache/kafka/pull/12499#discussion_r941967046 ## core/src/main/scala/kafka/server/BrokerToControllerChannelManager.scala: ## @@ -317,21 +373,24 @@ class BrokerToControllerRequestThread( override def generateReque

[GitHub] [kafka] hachikuji opened a new pull request, #12499: KAFKA-14154; Ensure AlterPartition not sent to stale controller

hachikuji opened a new pull request, #12499: URL: https://github.com/apache/kafka/pull/12499 It is possible currently for a leader to send an `AlterPartition` request to a stale controller which does not have the latest leader epoch discovered through a `LeaderAndIsr` request. In this case,

[jira] [Commented] (KAFKA-14081) Cannot get my MetricsReporter implementation to receive meaningful metrics

[ https://issues.apache.org/jira/browse/KAFKA-14081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17577677#comment-17577677 ] Rens Groothuijsen commented on KAFKA-14081: --- I had a look at the source code,

[jira] [Updated] (KAFKA-14154) Persistent URP after controller soft failure

[ https://issues.apache.org/jira/browse/KAFKA-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-14154: Description: We ran into a scenario where a partition leader was unable to expand the ISR

[GitHub] [kafka] guozhangwang commented on pull request #12497: KAFKA-10199: Expose read only task from state updater

guozhangwang commented on PR #12497: URL: https://github.com/apache/kafka/pull/12497#issuecomment-1210002259 The failures in `DefaultStateUpdaterTest` seems relevant? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Updated] (KAFKA-14154) Persistent URP after controller soft failure

[ https://issues.apache.org/jira/browse/KAFKA-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-14154: Fix Version/s: 3.3.0 > Persistent URP after controller soft failure >

[jira] [Created] (KAFKA-14154) Persistent URP after controller soft failure

Jason Gustafson created KAFKA-14154: --- Summary: Persistent URP after controller soft failure Key: KAFKA-14154 URL: https://issues.apache.org/jira/browse/KAFKA-14154 Project: Kafka Issue Type

[GitHub] [kafka] ahuang98 commented on a diff in pull request #12479: MINOR; Convert some integration tests to run with the KRaft modes

ahuang98 commented on code in PR #12479: URL: https://github.com/apache/kafka/pull/12479#discussion_r941879338 ## core/src/test/scala/unit/kafka/admin/DeleteTopicTest.scala: ## @@ -20,23 +20,37 @@ import java.util import java.util.concurrent.ExecutionException import java.util

[GitHub] [kafka] hachikuji commented on pull request #12498: KAFKA-13986; Brokers should include node.id in fetches to metadata quorum

hachikuji commented on PR #12498: URL: https://github.com/apache/kafka/pull/12498#issuecomment-1209985247 cc @dengziming @niket-goel @jsancio -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] hachikuji opened a new pull request, #12498: KAFKA-13986; Brokers should include node.id in fetches to metadata quorum

hachikuji opened a new pull request, #12498: URL: https://github.com/apache/kafka/pull/12498 Currently we do not set the replicaId in fetches from brokers to the metadata quorum. It is useful to do so since that allows us to debug replication using the `DescribeQuorum` API. ### Commi

[GitHub] [kafka] guozhangwang commented on pull request #12458: MINOR: Adds KRaft versions of most streams system tests

guozhangwang commented on PR #12458: URL: https://github.com/apache/kafka/pull/12458#issuecomment-1209983916 Re-triggered the jenkins build. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [kafka] guozhangwang commented on a diff in pull request #12458: MINOR: Adds KRaft versions of most streams system tests

guozhangwang commented on code in PR #12458: URL: https://github.com/apache/kafka/pull/12458#discussion_r941876941 ## tests/kafkatest/tests/streams/streams_broker_bounce_test.py: ## @@ -205,11 +212,17 @@ def collect_results(self, sleep_time_secs): return data @cl

[jira] [Updated] (KAFKA-14153) UnknownTopicOrPartitionException should include the topic/partition in the returned exception message

[ https://issues.apache.org/jira/browse/KAFKA-14153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alyssa Huang updated KAFKA-14153: - Description: Exception would be more useful if it included the topic or partition that was not

[GitHub] [kafka] jolshan commented on pull request #12487: KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode

jolshan commented on PR #12487: URL: https://github.com/apache/kafka/pull/12487#issuecomment-1209962842 Failed tests passed locally: [Build / JDK 8 and Scala 2.12 / kafka.admin.DeleteOffsetsConsumerGroupCommandIntegrationTest.testDeleteOffsetsNonExistingGroup()](https://ci-builds.apac

[GitHub] [kafka] cmccabe commented on pull request #12448: KAFKA-14114: Adding Metadata Log Processing Error Related Metrics

cmccabe commented on PR #12448: URL: https://github.com/apache/kafka/pull/12448#issuecomment-1209956699 Merged from command line with the changes we discussed. Thanks, @niket-goel -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [kafka] cmccabe closed pull request #12496: KAFKA-14114: Add Metadata Error Related Metrics

cmccabe closed pull request #12496: KAFKA-14114: Add Metadata Error Related Metrics URL: https://github.com/apache/kafka/pull/12496 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[jira] [Created] (KAFKA-14153) UnknownTopicOrPartitionException should include the topic/partition in the returned exception message

Alyssa Huang created KAFKA-14153: Summary: UnknownTopicOrPartitionException should include the topic/partition in the returned exception message Key: KAFKA-14153 URL: https://issues.apache.org/jira/browse/KAFKA-14

[GitHub] [kafka] ahuang98 commented on a diff in pull request #12479: MINOR; Convert some integration tests to run with the KRaft modes

ahuang98 commented on code in PR #12479: URL: https://github.com/apache/kafka/pull/12479#discussion_r941848002 ## core/src/test/scala/integration/kafka/api/RackAwareAutoTopicCreationTest.scala: ## @@ -37,29 +43,57 @@ class RackAwareAutoTopicCreationTest extends KafkaServerTestH

[jira] [Commented] (KAFKA-14069) Allow custom configuration of foreign key join internal topics

[ https://issues.apache.org/jira/browse/KAFKA-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17577634#comment-17577634 ] Guozhang Wang commented on KAFKA-14069: --- That's interesting.. could you check a fe

[jira] [Commented] (KAFKA-14070) Improve documentation for queryMetadataForKey for state stores with Processor API

[ https://issues.apache.org/jira/browse/KAFKA-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17577633#comment-17577633 ] Guozhang Wang commented on KAFKA-14070: --- Hello [~balajirrao] Thanks for the update

[jira] [Created] (KAFKA-14152) Add logic to fence kraft brokers which have fallen behind in replication

Jason Gustafson created KAFKA-14152: --- Summary: Add logic to fence kraft brokers which have fallen behind in replication Key: KAFKA-14152 URL: https://issues.apache.org/jira/browse/KAFKA-14152 Projec

[GitHub] [kafka] ahuang98 commented on a diff in pull request #12479: MINOR; Convert some integration tests to run with the KRaft modes

ahuang98 commented on code in PR #12479: URL: https://github.com/apache/kafka/pull/12479#discussion_r941771144 ## core/src/test/scala/integration/kafka/api/RackAwareAutoTopicCreationTest.scala: ## @@ -37,29 +43,57 @@ class RackAwareAutoTopicCreationTest extends KafkaServerTestH

[GitHub] [kafka] cmccabe merged pull request #12486: MINOR: BrokerMetadataSnapshotter must avoid exceeding batch size

cmccabe merged PR #12486: URL: https://github.com/apache/kafka/pull/12486 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] bbejeck commented on pull request #12494: MINOR: Update site docs for ASF compliance

bbejeck commented on PR #12494: URL: https://github.com/apache/kafka/pull/12494#issuecomment-1209815424 > Should we fix the issues I reported in [apache/kafka-site#433 (comment)](https://github.com/apache/kafka-site/pull/433#issuecomment-1205049698) before merging the images back to Kafka?

[jira] [Commented] (KAFKA-14138) The Exception Throwing Behavior of Transactional Producer is Inconsistent

[ https://issues.apache.org/jira/browse/KAFKA-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17577552#comment-17577552 ] Sagar Rao commented on KAFKA-14138: --- Thanks [~guozhang] . I went through KIP-691 and g

[jira] [Updated] (KAFKA-14151) Add validation to fail fast when base offsets are incorrectly assigned to batches

[ https://issues.apache.org/jira/browse/KAFKA-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruvil Shah updated KAFKA-14151: - Summary: Add validation to fail fast when base offsets are incorrectly assigned to batches (was

[jira] [Updated] (KAFKA-14151) Add additional validation to protect on-disk log segment data from being corrupted

[ https://issues.apache.org/jira/browse/KAFKA-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dhruvil Shah updated KAFKA-14151: - Description: We saw a case where records with incorrect offsets were being written to log segmen

[jira] [Updated] (KAFKA-14151) Add additional validation to protect on-disk log segment data from being corrupted

[ https://issues.apache.org/jira/browse/KAFKA-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent Jiang updated KAFKA-14151: -- Priority: Major (was: Minor) > Add additional validation to protect on-disk log segment data

[jira] [Created] (KAFKA-14151) Add additional validation to protect on-disk log segment data from being corrupted

Vincent Jiang created KAFKA-14151: - Summary: Add additional validation to protect on-disk log segment data from being corrupted Key: KAFKA-14151 URL: https://issues.apache.org/jira/browse/KAFKA-14151

[jira] [Updated] (KAFKA-14151) Add additional validation to protect on-disk log segment data from being corrupted

[ https://issues.apache.org/jira/browse/KAFKA-14151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent Jiang updated KAFKA-14151: -- Priority: Minor (was: Major) > Add additional validation to protect on-disk log segment data

[GitHub] [kafka] hachikuji commented on a diff in pull request #12487: KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode

hachikuji commented on code in PR #12487: URL: https://github.com/apache/kafka/pull/12487#discussion_r941604866 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -881,11 +881,16 @@ class Partition(val topicPartition: TopicPartition, private def isReplicaIsrEligible(

[GitHub] [kafka] jolshan commented on a diff in pull request #12487: KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode

jolshan commented on code in PR #12487: URL: https://github.com/apache/kafka/pull/12487#discussion_r941598423 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -881,11 +881,14 @@ class Partition(val topicPartition: TopicPartition, private def isReplicaIsrEligible(fo

[GitHub] [kafka] hachikuji commented on a diff in pull request #12469: KAFKA-13914: Add command line tool kafka-metadata-quorum.sh

hachikuji commented on code in PR #12469: URL: https://github.com/apache/kafka/pull/12469#discussion_r941548893 ## core/src/main/scala/kafka/admin/MetadataQuorumCommand.scala: ## @@ -0,0 +1,109 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * co

[GitHub] [kafka] jolshan commented on a diff in pull request #12487: KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode

jolshan commented on code in PR #12487: URL: https://github.com/apache/kafka/pull/12487#discussion_r941543760 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -881,11 +881,14 @@ class Partition(val topicPartition: TopicPartition, private def isReplicaIsrEligible(fo

[GitHub] [kafka] jolshan commented on a diff in pull request #12487: KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode

jolshan commented on code in PR #12487: URL: https://github.com/apache/kafka/pull/12487#discussion_r941537699 ## core/src/test/scala/unit/kafka/controller/ControllerIntegrationTest.scala: ## @@ -1045,6 +1049,78 @@ class ControllerIntegrationTest extends QuorumTestHarness {

[jira] [Resolved] (KAFKA-14144) AlterPartition is not idempotent when requests time out

[ https://issues.apache.org/jira/browse/KAFKA-14144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-14144. - Resolution: Fixed > AlterPartition is not idempotent when requests time out > --

[GitHub] [kafka] hachikuji merged pull request #12489: KAFKA-14144: Compare AlterPartition LeaderAndIsr before fencing partition epoch

hachikuji merged PR #12489: URL: https://github.com/apache/kafka/pull/12489 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apach

[jira] [Updated] (KAFKA-14134) Replace EasyMock with Mockito for WorkerConnectorTest

[ https://issues.apache.org/jira/browse/KAFKA-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison updated KAFKA-14134: --- Fix Version/s: 3.4.0 > Replace EasyMock with Mockito for WorkerConnectorTest > -

[GitHub] [kafka] yashmayya commented on a diff in pull request #12450: KAFKA-13809: Propagate full connector configuration to tasks in FileStream connectors

yashmayya commented on code in PR #12450: URL: https://github.com/apache/kafka/pull/12450#discussion_r941408595 ## docs/connect.html: ## @@ -440,19 +443,17 @@ Connector @Override public List> taskConfigs(int maxTasks) { +// This method is where co

[GitHub] [kafka] C0urante commented on a diff in pull request #12490: KAFKA-14147: Prevent deferredTaskUpdates map from growing monotonically in KafkaConfigBackingStore

C0urante commented on code in PR #12490: URL: https://github.com/apache/kafka/pull/12490#discussion_r941405027 ## connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaConfigBackingStore.java: ## @@ -853,6 +853,9 @@ private void processConnectorConfigRecord(String

[GitHub] [kafka] C0urante merged pull request #12472: KAFKA-14134: Replace EasyMock with Mockito for WorkerConnectorTest

C0urante merged PR #12472: URL: https://github.com/apache/kafka/pull/12472 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] C0urante commented on a diff in pull request #12472: KAFKA-14134: Replace EasyMock with Mockito for WorkerConnectorTest

C0urante commented on code in PR #12472: URL: https://github.com/apache/kafka/pull/12472#discussion_r941388856 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerConnectorTest.java: ## @@ -120,43 +105,21 @@ public void testInitializeFailure() { work

[GitHub] [kafka] C0urante commented on a diff in pull request #12450: KAFKA-13809: Propagate full connector configuration to tasks in FileStream connectors

C0urante commented on code in PR #12450: URL: https://github.com/apache/kafka/pull/12450#discussion_r941371749 ## connect/file/src/main/java/org/apache/kafka/connect/file/FileStreamSourceConnector.java: ## @@ -20,36 +20,35 @@ import org.apache.kafka.common.config.ConfigDef; im

[GitHub] [kafka] mimaison commented on pull request #12494: MINOR: Update site docs for ASF compliance

mimaison commented on PR #12494: URL: https://github.com/apache/kafka/pull/12494#issuecomment-1209402011 Thanks @bbejeck ! Should we fix the issues I reported in https://github.com/apache/kafka-site/pull/433#issuecomment-1205049698 before merging the images back to Kafka? -- This is a

[GitHub] [kafka] bbejeck commented on pull request #12494: MINOR: Update site docs for ASF compliance

bbejeck commented on PR #12494: URL: https://github.com/apache/kafka/pull/12494#issuecomment-1209389661 @mimaison added images -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[jira] [Comment Edited] (KAFKA-14014) Flaky test NamedTopologyIntegrationTest.shouldAllowRemovingAndAddingNamedTopologyToRunningApplicationWithMultipleNodesAndResetsOffsets()

[ https://issues.apache.org/jira/browse/KAFKA-14014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573211#comment-17573211 ] Matthew de Detrich edited comment on KAFKA-14014 at 8/9/22 12:58 PM: -

[GitHub] [kafka] bbejeck commented on pull request #12494: MINOR: Update site docs for ASF compliance

bbejeck commented on PR #12494: URL: https://github.com/apache/kafka/pull/12494#issuecomment-1209340844 > The path of the images is `/{{version}}/images/`, so do we need to add them to this repo too? Yes I overlooked that - will do -- This is an automated message from the Apache Gi

[GitHub] [kafka] vamossagar12 commented on pull request #12485: KAFKA-14131: Adding InterruptException when reading to end of Offseto…

vamossagar12 commented on PR #12485: URL: https://github.com/apache/kafka/pull/12485#issuecomment-1209328983 > This should definitely come with a test :) > > I'm also not sure this is the best approach, since the [ExecutorService::shutdownNow Javadocs](https://docs.oracle.com/en/java

[GitHub] [kafka] cadonna opened a new pull request, #12497: KAFKA-10199: Expose read only task from state updater

cadonna opened a new pull request, #12497: URL: https://github.com/apache/kafka/pull/12497 The state updater exposes tasks that are in restoration to the stream thread. To ensure that the stream thread only accesses the tasks to read from the tasks without modifying any internal stat

[jira] [Updated] (KAFKA-14133) Remaining EasyMock to Mockito tests

[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov updated KAFKA-14133: -- Description: {color:#DE350B}There are tests which use both PowerMock and EasyMock. I have put

[jira] [Updated] (KAFKA-14133) Remaining EasyMock to Mockito tests

[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov updated KAFKA-14133: -- Description: {color:#DE350B}There are tests which use both PowerMock and EasyMock. I have put

[jira] [Created] (KAFKA-14150) Allocation of initial partitions is deterministic and produces a leader bias when a broker is offline

David Buckley created KAFKA-14150: - Summary: Allocation of initial partitions is deterministic and produces a leader bias when a broker is offline Key: KAFKA-14150 URL: https://issues.apache.org/jira/browse/KAFKA-

[GitHub] [kafka] yashmayya commented on a diff in pull request #12472: KAFKA-14134: Replace EasyMock with Mockito for WorkerConnectorTest

yashmayya commented on code in PR #12472: URL: https://github.com/apache/kafka/pull/12472#discussion_r941154092 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerConnectorTest.java: ## @@ -672,6 +452,27 @@ protected void assertInitializedMetric(WorkerConnec

[GitHub] [kafka] dengziming commented on a diff in pull request #12414: KAFKA-14073 Logging the reason for Snapshot

dengziming commented on code in PR #12414: URL: https://github.com/apache/kafka/pull/12414#discussion_r941141588 ## core/src/main/scala/kafka/server/metadata/BrokerMetadataSnapshotter.scala: ## @@ -59,11 +60,19 @@ class BrokerMetadataSnapshotter( val writer = writerBuilde

[GitHub] [kafka] mimaison commented on pull request #12494: MINOR: Update site docs for ASF compliance

mimaison commented on PR #12494: URL: https://github.com/apache/kafka/pull/12494#issuecomment-1209131631 The path of the images is `/{{version}}/images/`, so do we need to add them to this repo too? -- This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [kafka] dajac commented on a diff in pull request #12487: KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode

dajac commented on code in PR #12487: URL: https://github.com/apache/kafka/pull/12487#discussion_r941077938 ## core/src/test/scala/unit/kafka/cluster/PartitionTest.scala: ## @@ -1375,8 +1377,11 @@ class PartitionTest extends AbstractPartitionTest { assertEquals(alterPartiti