Re: [PR] KAFKA-15695: Update the local log start offset of a log after rebuilding the auxiliary state [kafka]

2023-12-11 Thread via GitHub
showuon commented on PR #14649: URL: https://github.com/apache/kafka/pull/14649#issuecomment-1851453276 @nikramakrishnan , nice catch! I've set [KAFKA-15529](https://issues.apache.org/jira/browse/KAFKA-15529) as a blocker for v3.7.0. Re-triggering the CI to make sure the tests are flaky.

[jira] [Updated] (KAFKA-15529) Flaky test ReassignReplicaShrinkTest.executeTieredStorageTest

2023-12-11 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-15529: -- Fix Version/s: 3.7.0 > Flaky test ReassignReplicaShrinkTest.executeTieredStorageTest > ---

[jira] [Updated] (KAFKA-15529) Flaky test ReassignReplicaShrinkTest.executeTieredStorageTest

2023-12-11 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-15529: -- Priority: Blocker (was: Major) > Flaky test ReassignReplicaShrinkTest.executeTieredStorageTest >

Re: [PR] KAFKA-15471: Allow independently stop KRaft controllers or brokers [kafka]

2023-12-11 Thread via GitHub
hni61223 commented on PR #14400: URL: https://github.com/apache/kafka/pull/14400#issuecomment-1851446583 Hey Ron, thank you for catching that bug. I added a log message to notify the user that the start process and the stop process should be called from the same directory. WDYT? > T

[PR] MINOR: Fix typos in docker/example/README.md [kafka]

2023-12-11 Thread via GitHub
VedarthConfluent opened a new pull request, #14989: URL: https://github.com/apache/kafka/pull/14989 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary o

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
jolshan commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423490009 ## core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala: ## @@ -1378,6 +1350,28 @@ object GroupMetadataManager { "%X".format(BigInt(1, bytes))

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
jolshan commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423489699 ## core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala: ## @@ -283,36 +284,7 @@ class GroupMetadataManager(brokerId: Int, } else {

[jira] [Created] (KAFKA-15994) Add complex example compose file

2023-12-11 Thread Vedarth Sharma (Jira)
Vedarth Sharma created KAFKA-15994: -- Summary: Add complex example compose file Key: KAFKA-15994 URL: https://issues.apache.org/jira/browse/KAFKA-15994 Project: Kafka Issue Type: Sub-task

[jira] [Updated] (KAFKA-15881) Make changes in Release Process Wiki and Release Process

2023-12-11 Thread Vedarth Sharma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vedarth Sharma updated KAFKA-15881: --- Description: Make changes to Release Process Wiki and docker README.md for detailed release

[jira] [Updated] (KAFKA-15881) Make changes in Release Process Wiki and Release Process

2023-12-11 Thread Vedarth Sharma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vedarth Sharma updated KAFKA-15881: --- Summary: Make changes in Release Process Wiki and Release Process (was: Update release.py s

[jira] [Resolved] (KAFKA-15903) Add github actions workflow for building and pushing RC Docker Image

2023-12-11 Thread Vedarth Sharma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vedarth Sharma resolved KAFKA-15903. Fix Version/s: 3.7.0 Reviewer: Manikumar Resolution: Fixed > Add github ac

[jira] [Resolved] (KAFKA-15880) Add github actions workflow for promoting RC docker image

2023-12-11 Thread Vedarth Sharma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vedarth Sharma resolved KAFKA-15880. Fix Version/s: 3.7.0 Reviewer: Manikumar Resolution: Fixed PR merged > Ad

[jira] [Resolved] (KAFKA-15879) Add documentation and examples for the Docker image

2023-12-11 Thread Vedarth Sharma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vedarth Sharma resolved KAFKA-15879. Fix Version/s: 3.7.0 Reviewer: Manikumar Resolution: Fixed Documentation P

Re: [PR] MINOR: edit enableCleaner description [kafka]

2023-12-11 Thread via GitHub
laststem closed pull request #14703: MINOR: edit enableCleaner description URL: https://github.com/apache/kafka/pull/14703 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] KAFKA-15879: Add documentation and examples for docker image [kafka]

2023-12-11 Thread via GitHub
omkreddy merged PR #14938: URL: https://github.com/apache/kafka/pull/14938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Updated] (KAFKA-10404) Flaky Test kafka.api.SaslSslConsumerTest.testCoordinatorFailover

2023-12-11 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-10404: -- Labels: flaky-test (was: ) > Flaky Test kafka.api.SaslSslConsumerTest.testCoordinatorFailover > -

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
kirktrue commented on PR #14640: URL: https://github.com/apache/kafka/pull/14640#issuecomment-1851280653 A [quick search on Jira](https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20text%20~%20%22testCoordinatorFailover%22) shows that `testCoordinatorFailover` is a know

[jira] [Updated] (KAFKA-7605) Flaky Test `SaslMultiMechanismConsumerTest.testCoordinatorFailover`

2023-12-11 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-7605: - Labels: flaky-test (was: ) > Flaky Test `SaslMultiMechanismConsumerTest.testCoordinatorFailover` > -

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
kirktrue commented on PR #14640: URL: https://github.com/apache/kafka/pull/14640#issuecomment-1851277041 @dajac—I've made the requested changes. Can you take another look? The latest test run had the usual suspects of flaky tests, plus two that stood out: - Build / JDK 17 and S

Re: [PR] KAFKA-15922: Bump MetadataVersion to support JBOD with KRaft [kafka]

2023-12-11 Thread via GitHub
pprovenzano commented on PR #14984: URL: https://github.com/apache/kafka/pull/14984#issuecomment-1851271238 Okay, I think I have addresses the issues. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] MINOR: Allow Block.resetBlockLatch to release blocked operation for end-of-test cleanup [kafka]

2023-12-11 Thread via GitHub
ashwinpankaj commented on code in PR #14987: URL: https://github.com/apache/kafka/pull/14987#discussion_r1423381159 ## connect/runtime/src/test/java/org/apache/kafka/connect/integration/BlockingConnectorTest.java: ## @@ -406,6 +411,7 @@ public Block(String block) {

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
philipnee commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1423372314 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala: ## @@ -948,6 +948,7 @@ class PlaintextConsumerTest extends BaseConsumerTest { // 1 cons

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
hachikuji commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423342213 ## core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala: ## @@ -1378,6 +1350,28 @@ object GroupMetadataManager { "%X".format(BigInt(1, bytes)

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
hachikuji commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423341054 ## core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala: ## @@ -283,36 +284,7 @@ class GroupMetadataManager(brokerId: Int, } else {

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
jolshan commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423323788 ## core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala: ## @@ -909,8 +911,35 @@ private[group] class GroupCoordinator( val group = groupManager.

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
hachikuji commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423319991 ## core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala: ## @@ -909,8 +911,35 @@ private[group] class GroupCoordinator( val group = groupManage

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
jolshan commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423314305 ## core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala: ## @@ -909,8 +911,35 @@ private[group] class GroupCoordinator( val group = groupManager.

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
hachikuji commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423310554 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -915,64 +975,239 @@ class ReplicaManager(val config: KafkaConfig, // probably unblock som

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
jolshan commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423297237 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -915,64 +975,239 @@ class ReplicaManager(val config: KafkaConfig, // probably unblock some

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
hachikuji commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423297221 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -915,64 +975,239 @@ class ReplicaManager(val config: KafkaConfig, // probably unblock som

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
jolshan commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423296867 ## core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala: ## @@ -909,8 +911,35 @@ private[group] class GroupCoordinator( val group = groupManager.

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
hachikuji commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423294351 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -915,64 +975,239 @@ class ReplicaManager(val config: KafkaConfig, // probably unblock som

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
hachikuji commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423293611 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -915,64 +975,239 @@ class ReplicaManager(val config: KafkaConfig, // probably unblock som

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
hachikuji commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423290758 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -915,64 +975,239 @@ class ReplicaManager(val config: KafkaConfig, // probably unblock som

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-11 Thread via GitHub
hachikuji commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1423277341 ## core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala: ## @@ -909,8 +911,35 @@ private[group] class GroupCoordinator( val group = groupManage

Re: [PR] KAFKA-15906: Emit latest MM2 offset syncs every offset.flush.interval.ms [kafka]

2023-12-11 Thread via GitHub
gharris1727 merged PR #14967: URL: https://github.com/apache/kafka/pull/14967 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apa

Re: [PR] KAFKA-15906: Emit latest MM2 offset syncs every offset.flush.interval.ms [kafka]

2023-12-11 Thread via GitHub
gharris1727 commented on PR #14967: URL: https://github.com/apache/kafka/pull/14967#issuecomment-1851119930 Test failures appear unrelated, and tests pass locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[jira] [Updated] (KAFKA-15981) update Group size only when groups size changes

2023-12-11 Thread Jeff Kim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Kim updated KAFKA-15981: - Description: Currently, we increment generic group metrics whenever we create a new Group object when w

[PR] KAFKA-15981: update Group size only when groups size changes [kafka]

2023-12-11 Thread via GitHub
jeffkbkim opened a new pull request, #14988: URL: https://github.com/apache/kafka/pull/14988 Currently, we increment generic group metrics whenever we create a new Group object when we load a partition. This is incorrect as the partition may contain several records for the same group if in

[jira] [Updated] (KAFKA-15981) update Group size only when groups size changes

2023-12-11 Thread Jeff Kim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Kim updated KAFKA-15981: - Summary: update Group size only when groups size changes (was: only record Group state metrics once per

Re: [PR] KAFKA-15373: fix exception thrown in Admin#describeTopics for unknown ID [kafka]

2023-12-11 Thread via GitHub
jolshan commented on PR #14599: URL: https://github.com/apache/kafka/pull/14599#issuecomment-1851104746 At first I was confused since the original PR had https://github.com/apache/kafka/blob/2a5fbf28820ddcde5ead605e070391059d5d2e18/core/src/test/scala/integration/kafka/api/PlaintextAdminInte

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1423245926 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -177,34 +202,37 @@ public void process(final BackgroundEvent event)

Re: [PR] KAFKA-15827: Prevent KafkaBasedLog subclasses from leaking passed-in clients [kafka]

2023-12-11 Thread via GitHub
gharris1727 commented on code in PR #14763: URL: https://github.com/apache/kafka/pull/14763#discussion_r1423238556 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -812,6 +812,7 @@ public void stop() { ThreadUti

[jira] [Updated] (KAFKA-15981) only record Group state metrics once per group during load

2023-12-11 Thread Jeff Kim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Kim updated KAFKA-15981: - Summary: only record Group state metrics once per group during load (was: only record GenericGroup stat

[jira] [Updated] (KAFKA-15981) only record GenericGroup state metrics once per group during load

2023-12-11 Thread Jeff Kim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Kim updated KAFKA-15981: - Description: Currently, we increment generic group metrics whenever we create a new GenericGroup object

Re: [PR] KAFKA-15827: Prevent KafkaBasedLog subclasses from leaking passed-in clients [kafka]

2023-12-11 Thread via GitHub
C0urante commented on code in PR #14763: URL: https://github.com/apache/kafka/pull/14763#discussion_r1423214226 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -812,6 +812,7 @@ public void stop() { ThreadUtils.

[PR] MINOR: Allow Block.resetBlockLatch to release blocked operation for end-of-test cleanup [kafka]

2023-12-11 Thread via GitHub
gharris1727 opened a new pull request, #14987: URL: https://github.com/apache/kafka/pull/14987 The kafka.utils.TestUtils.verifyNoUnexpectedThreads method used before and after core tests finds threads leaked by these tests. If these tests run before core tests, then those tests will all fai

Re: [PR] KAFKA-15988: Reuse embedded clusters across test cases in Connect OffsetsApiIntegrationTest suite [kafka]

2023-12-11 Thread via GitHub
C0urante commented on code in PR #14966: URL: https://github.com/apache/kafka/pull/14966#discussion_r1423211506 ## connect/runtime/src/test/java/org/apache/kafka/connect/integration/OffsetsApiIntegrationTest.java: ## @@ -324,127 +363,151 @@ public void testAlterSinkConnectorOff

Re: [PR] KAFKA-15358: Added QueuedReplicaToDirAssignments metric [kafka]

2023-12-11 Thread via GitHub
mwesterby commented on PR #14942: URL: https://github.com/apache/kafka/pull/14942#issuecomment-1851003645 Should be fixed now! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] KAFKA-15988: Reuse embedded clusters across test cases in Connect OffsetsApiIntegrationTest suite [kafka]

2023-12-11 Thread via GitHub
C0urante commented on code in PR #14966: URL: https://github.com/apache/kafka/pull/14966#discussion_r1423199416 ## connect/runtime/src/test/java/org/apache/kafka/connect/integration/OffsetsApiIntegrationTest.java: ## @@ -324,127 +363,151 @@ public void testAlterSinkConnectorOff

Re: [PR] KAFKA-15988: Reuse embedded clusters across test cases in Connect OffsetsApiIntegrationTest suite [kafka]

2023-12-11 Thread via GitHub
C0urante commented on code in PR #14966: URL: https://github.com/apache/kafka/pull/14966#discussion_r1423199242 ## connect/runtime/src/test/java/org/apache/kafka/connect/integration/OffsetsApiIntegrationTest.java: ## @@ -70,74 +80,115 @@ public class OffsetsApiIntegrationTest {

Re: [PR] KAFKA-15922: Bump MetadataVersion to support JBOD with KRaft [kafka]

2023-12-11 Thread via GitHub
rondagostino commented on PR #14984: URL: https://github.com/apache/kafka/pull/14984#issuecomment-1850991968 @pprovenzano Can you look at the test failures? `60 tests have failed. There are 0 new tests failing, 60 existing failing` https://ci-builds.apache.org/blue/organizations/jenkin

Re: [PR] KAFKA-15358: Added QueuedReplicaToDirAssignments metric [kafka]

2023-12-11 Thread via GitHub
rondagostino commented on PR #14942: URL: https://github.com/apache/kafka/pull/14942#issuecomment-1850990787 @mwesterby Looks like we have a checkstyle issue: ``` Task :server:checkstyleTest [ant:checkstyle] [ERROR] /home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-14942/s

Re: [PR] KAFKA-15950: Serialize broker heartbeat requests [kafka]

2023-12-11 Thread via GitHub
rondagostino commented on PR #14903: URL: https://github.com/apache/kafka/pull/14903#issuecomment-1850988044 `12 tests have failed. There are 12 new tests failing, 0 existing failing` I think this PR can be merged now assuming @cmccabe takes a look at the question above regarding pre

Re: [PR] KAFKA-15826: Close consumer when sink task is cancelled [kafka]

2023-12-11 Thread via GitHub
gharris1727 commented on PR #14762: URL: https://github.com/apache/kafka/pull/14762#issuecomment-1850986158 I found this same leak again when debugging some failures of the `kafka.utils.TestUtils.verifyNoUnexpectedThreads` assertion in core tests. Whenever we leak one of these clients in co

Re: [PR] KAFKA-15827: Prevent KafkaBasedLog subclasses from leaking passed-in clients [kafka]

2023-12-11 Thread via GitHub
gharris1727 commented on PR #14763: URL: https://github.com/apache/kafka/pull/14763#issuecomment-1850969462 Hi @C0urante I found another very similar problem, where InternalTopicsIntegrationTest was causing us to leak clients. There, the "bad topics" cause startServices to throw an ex

[jira] [Resolved] (KAFKA-15563) Provide informative error messages when Connect REST requests time out

2023-12-11 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-15563. --- Fix Version/s: 3.7.0 Resolution: Fixed > Provide informative error messages when Conn

Re: [PR] KAFKA-15563: Provide informative error messages when Connect REST requests time out [kafka]

2023-12-11 Thread via GitHub
C0urante merged PR #14562: URL: https://github.com/apache/kafka/pull/14562 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-15563: Provide informative error messages when Connect REST requests time out [kafka]

2023-12-11 Thread via GitHub
C0urante commented on PR #14562: URL: https://github.com/apache/kafka/pull/14562#issuecomment-1850942821 All CI failures appear unrelated, merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

2023-12-11 Thread via GitHub
lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1423159520 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -619,11 +838,36 @@ private CompletableFuture> chainFuture(fi

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
cadonna commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1423147503 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -201,13 +224,35 @@ public long maximumTimeToWait(long currentTi

Re: [PR] KAFKA-15358: Added QueuedReplicaToDirAssignments metric [kafka]

2023-12-11 Thread via GitHub
soarez commented on code in PR #14942: URL: https://github.com/apache/kafka/pull/14942#discussion_r1423140834 ## server/src/main/java/org/apache/kafka/server/AssignmentsManager.java: ## @@ -93,10 +100,24 @@ public AssignmentsManager(Time time, "broker-" + broker

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1423135362 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -177,34 +202,37 @@ public void process(final BackgroundEvent event)

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1423134165 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -790,6 +791,297 @@ public void testOnSubscriptionUpdatedTra

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1423128358 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerUtils.java: ## @@ -224,4 +224,21 @@ else if (t instanceof KafkaException) thr

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1423127956 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1499,6 +1575,149 @@ private void subscribeInternal(Collection top

Re: [PR] KAFKA-15358: Added QueuedReplicaToDirAssignments metric [kafka]

2023-12-11 Thread via GitHub
rondagostino commented on PR #14942: URL: https://github.com/apache/kafka/pull/14942#issuecomment-1850876018 @mwesterby Can you resolve the conflicts here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-15471: Allow independently stop KRaft controllers or brokers [kafka]

2023-12-11 Thread via GitHub
rondagostino commented on PR #14400: URL: https://github.com/apache/kafka/pull/14400#issuecomment-1850870460 Thanks for describing the testing. I just did a test to see what would happen if the config file was specified with a relative path, and it did not work well. For example: # sta

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
philipnee commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1423103409 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -201,13 +224,35 @@ public long maximumTimeToWait(long current

[jira] [Updated] (KAFKA-15993) Enable integration tests that relies on rebalance listener

2023-12-11 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15993: --- Parent: KAFKA-14246 Issue Type: Sub-task (was: Task) > Enable integration tests that relies

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
philipnee commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1850852690 Per request: https://issues.apache.org/jira/browse/KAFKA-15993 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[PR] MINOR: add entity type to topic names in ConsumerGroupDescribeResponse.json [kafka]

2023-12-11 Thread via GitHub
dongnuo123 opened a new pull request, #14986: URL: https://github.com/apache/kafka/pull/14986 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of test

[jira] [Updated] (KAFKA-15993) Enable integration tests that relies on rebalance listener

2023-12-11 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15993: --- Labels: consumer-threading-refactor kip-848-preview (was: ) > Enable integration tests that relies

[jira] [Updated] (KAFKA-15993) Enable integration tests that relies on rebalance listener

2023-12-11 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15993: --- Component/s: consumer > Enable integration tests that relies on rebalance listener > ---

[jira] [Updated] (KAFKA-15993) Enable integration tests that relies on rebalance listener

2023-12-11 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15993: --- Priority: Blocker (was: Major) > Enable integration tests that relies on rebalance listener > -

[jira] [Created] (KAFKA-15993) Enable integration tests that relies on rebalance listener

2023-12-11 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15993: -- Summary: Enable integration tests that relies on rebalance listener Key: KAFKA-15993 URL: https://issues.apache.org/jira/browse/KAFKA-15993 Project: Kafka Issue

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
philipnee commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1850844332 Hey @lianetm - You are right. It uses rebalance callback in the test so it will need to be enabled after implementing the rebalance callback. However, I did verify the leave group requ

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
lianetm commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1423084344 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -201,13 +224,35 @@ public long maximumTimeToWait(long currentTi

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1423064727 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1499,6 +1575,149 @@ private void subscribeInternal(Collection top

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1423062667 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -790,6 +812,197 @@ public void testOnSubscriptionUpdatedTra

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
lianetm commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1850787095 @philipnee thanks for the updates! One last comment. Important follow-up to this PR is to enable the specific max poll related integration tests (like `testMaxPollIntervalMs`), that are c

Re: [PR] KAFKA-15991: Fix flaky test testGroupIdNotNullAndValid [kafka]

2023-12-11 Thread via GitHub
lucasbru commented on PR #14944: URL: https://github.com/apache/kafka/pull/14944#issuecomment-1850757955 @cadonna If you have more comments on #14930, please feel free to merge this, I will anyways have conflicts with every single CTR/KIP-848 PR. Instead, if #14930 is good to go, let

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1423031514 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/OffsetMetadataManager.java: ## @@ -892,6 +893,43 @@ public void replay( } } +/**

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1423031807 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -881,6 +882,120 @@ public String toString() { }

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1423030686 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -2416,14 +2418,49 @@ class KafkaApis(val requestChannel: RequestChannel, numAppends.decrementAndGet()

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1423030220 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -2361,25 +2361,27 @@ class KafkaApis(val requestChannel: RequestChannel, * request, so there could be mu

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1423029751 ## core/src/main/scala/kafka/coordinator/group/CoordinatorPartitionWriter.scala: ## @@ -190,4 +170,56 @@ class CoordinatorPartitionWriter[T]( throw Errors.NOT_LE

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1423029434 ## core/src/main/scala/kafka/coordinator/group/CoordinatorLoaderImpl.scala: ## @@ -135,7 +136,22 @@ class CoordinatorLoaderImpl[T]( memoryRecords.batches.f

[PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac opened a new pull request, #14985: URL: https://github.com/apache/kafka/pull/14985 This patch wires the handling of makers written by the transaction coordinator via the WriteTxnMarkers API. In the old group coordinator, the markers are written to the logs and the group coordinator is

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
lucasbru commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1850740142 @philipnee thanks for the updates. @cadonna I'll wait for you green light this by approving. -- This is an automated message from the Apache Git Service. To respond to the message, p

[jira] [Resolved] (KAFKA-15574) Update states and transitions for membership manager state machine

2023-12-11 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-15574. - Fix Version/s: 3.7.0 Resolution: Fixed > Update states and transitions for membership man

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
philipnee commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1850712115 Hi @lianetm and @lucasbru - Per your request, I've just ran the tests using ``` Arguments.of("kraft+kip848", "consumer")) ``` All tests passed except `testRemoteAssig

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-11 Thread via GitHub
dajac merged PR #14857: URL: https://github.com/apache/kafka/pull/14857 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-11 Thread via GitHub
dajac commented on PR #14857: URL: https://github.com/apache/kafka/pull/14857#issuecomment-1850707387 I looked at all the failed tests and none of them are related to this PR. The "existing failure" are related to JBDO changes. Merging to trunk. -- This is an automated message from the Ap

Re: [PR] KAFKA-15913: Migrate async consumer tests to mocks [kafka]

2023-12-11 Thread via GitHub
lucasbru commented on code in PR #14930: URL: https://github.com/apache/kafka/pull/14930#discussion_r1423004412 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1535,6 +1591,11 @@ int callbacks() { return invoker.callb

Re: [PR] KAFKA-15143: Adding in MockFixedKeyProcessorContext and Test [kafka]

2023-12-11 Thread via GitHub
lihaosky commented on code in PR #14605: URL: https://github.com/apache/kafka/pull/14605#discussion_r1422983101 ## streams/test-utils/src/test/java/org/apache/kafka/streams/test/MockFixedKeyProcessorContextTest.java: ## @@ -0,0 +1,315 @@ +/* + * Licensed to the Apache Software F

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
philipnee commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1850676664 hi @cadonna - Sorry about unintentionally ignoring your question. I've just responded to it. -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
philipnee commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1422983344 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -201,13 +224,35 @@ public long maximumTimeToWait(long current

Re: [PR] KAFKA-15417 flip joinSpuriousLookBackTimeMs and emit non-joined items [kafka]

2023-12-11 Thread via GitHub
lihaosky commented on PR #14426: URL: https://github.com/apache/kafka/pull/14426#issuecomment-1850672002 @mjsax , @guozhangwang , can we merge this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] Revert "MINOR: Add junit properties to display parameterized test names (#14687)" [kafka]

2023-12-11 Thread via GitHub
cadonna commented on PR #14961: URL: https://github.com/apache/kafka/pull/14961#issuecomment-1850662683 @alok123t I just read your comment. FYI, I worked on fixing this PR today. The [resulting PR](https://github.com/apache/kafka/pull/14983) is ready to merge. I just need an approval. --

  1   2   >