Re: [PR] MINOR: fix zookeeper_migration_test system test in 3.5 [kafka]

2023-12-11 Thread via GitHub
showuon merged PR #14927: URL: https://github.com/apache/kafka/pull/14927 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] KAFKA-15879: Add documentation and examples for docker image [kafka]

2023-12-11 Thread via GitHub
omkreddy commented on code in PR #14938: URL: https://github.com/apache/kafka/pull/14938#discussion_r1422015243 ## docker/examples/jvm/cluster/combined/ssl/docker-compose.yml: ## @@ -0,0 +1,113 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contribut

Re: [PR] MINOR: fix leak in `GroupEndToEndAuthorizationTest` [kafka]

2023-12-11 Thread via GitHub
lucasbru merged PR #14975: URL: https://github.com/apache/kafka/pull/14975 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] MINOR: Bug fixes for AssignmentsManager [kafka]

2023-12-11 Thread via GitHub
dajac commented on PR #14954: URL: https://github.com/apache/kafka/pull/14954#issuecomment-1849511808 @soarez `testAlterReplicaLogDirs(String).quorum=kraft – kafka.api.PlaintextAdminIntegrationTest` still fails in [trunk](https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka

[PR] KAFKA-15817: Avoid reconnecting to the same IP address (#14813) [kafka]

2023-12-11 Thread via GitHub
cadonna opened a new pull request, #14980: URL: https://github.com/apache/kafka/pull/14980 If multiple addresses are available. This change is a follow-up to #9902. When re-resolving DNS after disconnecting, it is possible (likely, even) that we will resolve the same set of addresses in the

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-12-11 Thread via GitHub
mjsax merged PR #14948: URL: https://github.com/apache/kafka/pull/14948 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
cadonna commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1422075316 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -201,13 +224,35 @@ public long maximumTimeToWait(long currentTi

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-11 Thread via GitHub
dajac commented on PR #14857: URL: https://github.com/apache/kafka/pull/14857#issuecomment-1849536819 Merged trunk to address conflicts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
cadonna commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1849541992 > @lianetm / @cadonna any more comments on this? @lucasbru I did not get any answer to the following comment: https://github.com/apache/kafka/pull/14873#discussion_r1418729793

[PR] KAFKA-15237: Enforcing write timeouts when writing to the log [kafka]

2023-12-11 Thread via GitHub
vamossagar12 opened a new pull request, #14981: URL: https://github.com/apache/kafka/pull/14981 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[jira] [Updated] (KAFKA-13922) Unable to generate coverage reports for the whole project

2023-12-11 Thread Viktor Somogyi-Vass (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Somogyi-Vass updated KAFKA-13922: Labels: cloudera (was: ) > Unable to generate coverage reports for the whole proj

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1422131602 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ApplicationEventProcessor.java: ## @@ -113,16 +113,15 @@ public void process(ApplicationEvent

[jira] [Created] (KAFKA-15992) Make MM2 heartbeats topic name configurable

2023-12-11 Thread Viktor Somogyi-Vass (Jira)
Viktor Somogyi-Vass created KAFKA-15992: --- Summary: Make MM2 heartbeats topic name configurable Key: KAFKA-15992 URL: https://issues.apache.org/jira/browse/KAFKA-15992 Project: Kafka Iss

[jira] [Updated] (KAFKA-15992) Make MM2 heartbeats topic name configurable

2023-12-11 Thread Viktor Somogyi-Vass (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Somogyi-Vass updated KAFKA-15992: Component/s: mirrormaker > Make MM2 heartbeats topic name configurable > -

Re: [PR] MINOR: Bug fixes for AssignmentsManager [kafka]

2023-12-11 Thread via GitHub
OmniaGM commented on PR #14954: URL: https://github.com/apache/kafka/pull/14954#issuecomment-1849633538 > @soarez `testAlterReplicaLogDirs(String).quorum=kraft – kafka.api.PlaintextAdminIntegrationTest` still fails in [trunk](https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fk

Re: [PR] MINOR: Bug fixes for AssignmentsManager [kafka]

2023-12-11 Thread via GitHub
dajac commented on PR #14954: URL: https://github.com/apache/kafka/pull/14954#issuecomment-1849637350 Thanks @OmniaGM! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] MINOR: Fix rack-aware assignment tests [kafka]

2023-12-11 Thread via GitHub
mjsax merged PR #14965: URL: https://github.com/apache/kafka/pull/14965 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] Update upgrade-guide.html for KIP-960 and KIP-968 [kafka]

2023-12-11 Thread via GitHub
mjsax merged PR #14958: URL: https://github.com/apache/kafka/pull/14958 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1422136734 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1499,6 +1575,149 @@ private void subscribeInternal(Collection topics

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1422269556 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -177,34 +202,37 @@ public void process(final BackgroundEvent event) {

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
dajac commented on PR #14640: URL: https://github.com/apache/kafka/pull/14640#issuecomment-1849773951 @kirktrue I just replied to the open comments and left more comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] KAFKA-15161: Fix InvalidReplicationFactorException at connect startup [kafka]

2023-12-11 Thread via GitHub
viktorsomogyi commented on PR #13975: URL: https://github.com/apache/kafka/pull/13975#issuecomment-1849789158 @mimaison do you think you can review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] KAFKA-15363: Run completion handlers for duped AssignmentEvent [kafka]

2023-12-11 Thread via GitHub
soarez opened a new pull request, #14982: URL: https://github.com/apache/kafka/pull/14982 AssignmentEvents are deduped in AssignmentsManager, but it is important to consider any involved completion handlers and ensure they are eventually called. This addresses a failing test: testA

Re: [PR] KAFKA-15363: Run completion handlers for duped AssignmentEvent [kafka]

2023-12-11 Thread via GitHub
soarez commented on PR #14982: URL: https://github.com/apache/kafka/pull/14982#issuecomment-1849817907 @dajac @OmniaGM PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] KAFKA-15870: Move new group coordinator metrics from Yammer to Metrics [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14848: URL: https://github.com/apache/kafka/pull/14848#discussion_r1422297921 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/GroupCoordinatorMetrics.java: ## @@ -29,44 +28,75 @@ import java.util.Arrays; import jav

Re: [PR] KAFKA-15363: Run completion handlers for duped AssignmentEvent [kafka]

2023-12-11 Thread via GitHub
OmniaGM commented on PR #14982: URL: https://github.com/apache/kafka/pull/14982#issuecomment-1849861443 LGTM, this is a straightforward change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[jira] [Assigned] (KAFKA-15992) Make MM2 heartbeats topic name configurable

2023-12-11 Thread Bertalan Kondrat (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bertalan Kondrat reassigned KAFKA-15992: Assignee: Bertalan Kondrat (was: Viktor Somogyi-Vass) > Make MM2 heartbeats topi

[jira] [Assigned] (KAFKA-15497) Refactor build.gradle and split each module configuration on the module itself

2023-12-11 Thread Bertalan Kondrat (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bertalan Kondrat reassigned KAFKA-15497: Assignee: Bertalan Kondrat > Refactor build.gradle and split each module configur

Re: [PR] KAFKA-15883: Implement RemoteCopyLagBytes [kafka]

2023-12-11 Thread via GitHub
showuon commented on PR #14832: URL: https://github.com/apache/kafka/pull/14832#issuecomment-1849962133 @clolov , could you address @kamalcph 's comments? I hope this KIP can be included into v3.7.0. -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] KAFKA-15237: Enforcing write timeouts when writing to the log [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14981: URL: https://github.com/apache/kafka/pull/14981#discussion_r1422418778 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -595,22 +609,25 @@ class CoordinatorWriteEvent implements Co

[PR] MINOR: Add junit properties to display parameterized test names [kafka]

2023-12-11 Thread via GitHub
cadonna opened a new pull request, #14983: URL: https://github.com/apache/kafka/pull/14983 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

Re: [PR] KAFKA-15237: Implement write operation timeout [kafka]

2023-12-11 Thread via GitHub
dajac commented on PR #14981: URL: https://github.com/apache/kafka/pull/14981#issuecomment-1850048978 @vamossagar12 I updated the title to match the one of the Jira. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-15913: Migrate async consumer tests to mocks [kafka]

2023-12-11 Thread via GitHub
lucasbru commented on code in PR #14930: URL: https://github.com/apache/kafka/pull/14930#discussion_r1422459407 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -482,6 +497,47 @@ private void process(final GroupMetadataUpdateEve

Re: [PR] KAFKA-15347: implement lazy deserialization for segment [kafka]

2023-12-11 Thread via GitHub
mjsax commented on code in PR #14957: URL: https://github.com/apache/kafka/pull/14957#discussion_r1422442823 ## streams/src/main/java/org/apache/kafka/streams/state/internals/ReadonlyPartiallyDeserializedSegmentValue.java: ## @@ -0,0 +1,206 @@ +/* + * Licensed to the Apache Soft

Re: [PR] KAFKA-15347: implement lazy deserialization for segment [kafka]

2023-12-11 Thread via GitHub
aliehsaeedii commented on code in PR #14957: URL: https://github.com/apache/kafka/pull/14957#discussion_r1422488101 ## streams/src/main/java/org/apache/kafka/streams/state/internals/ReadonlyPartiallyDeserializedSegmentValue.java: ## @@ -0,0 +1,206 @@ +/* + * Licensed to the Apac

Re: [PR] KAFKA-15775: New consumer listTopics and partitionsFor [kafka]

2023-12-11 Thread via GitHub
lucasbru commented on code in PR #14962: URL: https://github.com/apache/kafka/pull/14962#discussion_r1422495787 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataRequestManager.java: ## @@ -55,16 +56,13 @@ * * The manager checks the state of

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-11 Thread via GitHub
OmniaGM commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1422522894 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -867,6 +869,25 @@ class Partition(val topicPartition: TopicPartition, } } + private def createLog

Re: [PR] KAFKA-15883: Implement RemoteCopyLagBytes [kafka]

2023-12-11 Thread via GitHub
clolov commented on code in PR #14832: URL: https://github.com/apache/kafka/pull/14832#discussion_r1422550544 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -386,7 +431,26 @@ class BrokerTopicMetrics(name: Option[String], configOpt: java.util.Optional[Kaf

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-11 Thread via GitHub
rondagostino merged PR #14881: URL: https://github.com/apache/kafka/pull/14881 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.ap

Re: [PR] KAFKA-15358: Added QueuedReplicaToDirAssignments metric [kafka]

2023-12-11 Thread via GitHub
rondagostino commented on code in PR #14942: URL: https://github.com/apache/kafka/pull/14942#discussion_r1422561158 ## server/src/main/java/org/apache/kafka/server/AssignmentsManager.java: ## @@ -240,31 +269,53 @@ public void run() throws Exception { } } +pri

Re: [PR] KAFKA-15208: Upgrade Jackson dependencies to version 2.16.0 [kafka]

2023-12-11 Thread via GitHub
mimaison commented on PR #13662: URL: https://github.com/apache/kafka/pull/13662#issuecomment-1850229092 @divijvaidya Do you have any further comments? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] KAFKA-15950: Serialize broker heartbeat requests [kafka]

2023-12-11 Thread via GitHub
rondagostino commented on PR #14903: URL: https://github.com/apache/kafka/pull/14903#issuecomment-1850245547 `51 tests have failed. There are 49 new tests failing, 2 existing failing` This seems like a lot more failures than we are used to recently. @OmniaGM Can you rebase this onto

Re: [PR] KAFKA-15363: Run completion handlers for duped AssignmentEvent [kafka]

2023-12-11 Thread via GitHub
rondagostino merged PR #14982: URL: https://github.com/apache/kafka/pull/14982 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.ap

Re: [PR] KAFKA-15879: Add documentation and examples for docker image [kafka]

2023-12-11 Thread via GitHub
VedarthConfluent commented on code in PR #14938: URL: https://github.com/apache/kafka/pull/14938#discussion_r1422629371 ## docs/quickstart.html: ## @@ -94,6 +98,12 @@ $ bin/kafka-server-start.sh config/kraft/server.properties +Using docker image + +

Re: [PR] KAFKA-15879: Add documentation and examples for docker image [kafka]

2023-12-11 Thread via GitHub
VedarthConfluent commented on code in PR #14938: URL: https://github.com/apache/kafka/pull/14938#discussion_r1422629627 ## docker/examples/jvm/cluster/combined/ssl/docker-compose.yml: ## @@ -0,0 +1,113 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# c

[jira] [Assigned] (KAFKA-15922) Add MetadataVersion for JBOD

2023-12-11 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano reassigned KAFKA-15922: - Assignee: Proven Provenzano > Add MetadataVersion for JBOD > --

Re: [PR] KAFKA-15879: Add documentation and examples for docker image [kafka]

2023-12-11 Thread via GitHub
VedarthConfluent commented on code in PR #14938: URL: https://github.com/apache/kafka/pull/14938#discussion_r1422629371 ## docs/quickstart.html: ## @@ -94,6 +98,12 @@ $ bin/kafka-server-start.sh config/kraft/server.properties +Using docker image + +

Re: [PR] KAFKA-15879: Add documentation and examples for docker image [kafka]

2023-12-11 Thread via GitHub
VedarthConfluent commented on code in PR #14938: URL: https://github.com/apache/kafka/pull/14938#discussion_r1422647778 ## docs/quickstart.html: ## @@ -94,6 +98,12 @@ $ bin/kafka-server-start.sh config/kraft/server.properties +Using docker image + +

Re: [PR] KAFKA-15695: Update the local log start offset of a log after rebuilding the auxiliary state [kafka]

2023-12-11 Thread via GitHub
nikramakrishnan commented on PR #14649: URL: https://github.com/apache/kafka/pull/14649#issuecomment-1850310706 @satishd apologies for the late response, I was on vacation. I have updated the PR to remove the integration test for now, and it should be ready to merge (given the unit test pas

Re: [PR] KAFKA-15942: Implement ConsumerInterceptor [kafka]

2023-12-11 Thread via GitHub
Joker-5 commented on PR #14963: URL: https://github.com/apache/kafka/pull/14963#issuecomment-1850317127 @vamossagar12 Thanks for reviewing the PR! At the beginning I misunderstood this ticket, now I understood and the code has already updated. When commit request returns successfu

[PR] KAFKA-15922: Bump MetadataVersion to support JBOD with KRaft [kafka]

2023-12-11 Thread via GitHub
pprovenzano opened a new pull request, #14984: URL: https://github.com/apache/kafka/pull/14984 The changes needed to make JBOD work in KRaft mode and to enable migration JBOD systems from ZK to KRaft are in. -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] KAFKA-15883: Implement RemoteCopyLagBytes [kafka]

2023-12-11 Thread via GitHub
clolov commented on code in PR #14832: URL: https://github.com/apache/kafka/pull/14832#discussion_r1422676113 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -404,6 +468,7 @@ object BrokerTopicStats { val ProduceMessageConversionsPerSec = "ProduceMessageC

Re: [PR] KAFKA-15883: Implement RemoteCopyLagBytes [kafka]

2023-12-11 Thread via GitHub
clolov commented on code in PR #14832: URL: https://github.com/apache/kafka/pull/14832#discussion_r1422675847 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -386,7 +431,26 @@ class BrokerTopicMetrics(name: Option[String], configOpt: java.util.Optional[Kaf

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
lianetm commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1422677966 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -790,6 +812,197 @@ public void testOnSubscriptionUpdatedTran

Re: [PR] KAFKA-15471: Allow independently stop KRaft controllers or brokers [kafka]

2023-12-11 Thread via GitHub
rondagostino commented on PR #14400: URL: https://github.com/apache/kafka/pull/14400#issuecomment-1850346431 Thanks for the PR! Given that we have no automated test, please add comments documenting the output of appropriate manual tests of this feature. -- This is an automated message fr

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1422704717 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -790,6 +812,197 @@ public void testOnSubscriptionUpdatedTransi

Re: [PR] KAFKA-15801: improve Kafka broker/NetworkClient logging for connectiv… [kafka]

2023-12-11 Thread via GitHub
Joker-5 commented on PR #14799: URL: https://github.com/apache/kafka/pull/14799#issuecomment-1850377238 Hi @dajac, the code has already updated. Would you have time to take a look at this PR? Thanks so much. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] KAFKA-15950: Serialize broker heartbeat requests [kafka]

2023-12-11 Thread via GitHub
OmniaGM commented on PR #14903: URL: https://github.com/apache/kafka/pull/14903#issuecomment-1850387349 > `51 tests have failed. There are 49 new tests failing, 2 existing failing` > > This seems like a lot more failures than we are used to recently. @OmniaGM Can you rebase this onto

[jira] [Assigned] (KAFKA-15893) Bump MetadataVersion for directory assignments

2023-12-11 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano reassigned KAFKA-15893: - Assignee: Proven Provenzano > Bump MetadataVersion for directory assignments >

Re: [PR] KAFKA-7839 Add quotes to CLASSPATH on Java start [kafka]

2023-12-11 Thread via GitHub
rwunderer closed pull request #6166: KAFKA-7839 Add quotes to CLASSPATH on Java start URL: https://github.com/apache/kafka/pull/6166 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] KAFKA-15913: Migrate async consumer tests to mocks [kafka]

2023-12-11 Thread via GitHub
cadonna commented on code in PR #14930: URL: https://github.com/apache/kafka/pull/14930#discussion_r1422458903 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -16,8 +16,43 @@ */ package org.apache.kafka.clients.consumer.

Re: [PR] KAFKA-15358: Added QueuedReplicaToDirAssignments metric [kafka]

2023-12-11 Thread via GitHub
rondagostino commented on PR #14942: URL: https://github.com/apache/kafka/pull/14942#issuecomment-1850488367 Can we get the conflicts resolved? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-15913: Migrate async consumer tests to mocks [kafka]

2023-12-11 Thread via GitHub
lucasbru commented on code in PR #14930: URL: https://github.com/apache/kafka/pull/14930#discussion_r1422818505 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -482,6 +497,47 @@ private void process(final GroupMetadataUpdateEve

Re: [PR] KAFKA-15883: Implement RemoteCopyLagBytes [kafka]

2023-12-11 Thread via GitHub
clolov commented on PR #14832: URL: https://github.com/apache/kafka/pull/14832#issuecomment-1850504833 Heya, sorry, I will try to provide an updated version tomorrow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-15913: Migrate async consumer tests to mocks [kafka]

2023-12-11 Thread via GitHub
lucasbru commented on code in PR #14930: URL: https://github.com/apache/kafka/pull/14930#discussion_r1422826824 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -16,8 +16,43 @@ */ package org.apache.kafka.clients.consumer

Re: [PR] KAFKA-15913: Migrate async consumer tests to mocks [kafka]

2023-12-11 Thread via GitHub
lucasbru commented on code in PR #14930: URL: https://github.com/apache/kafka/pull/14930#discussion_r1422828083 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -42,192 +77,142 @@ import org.apache.kafka.clients.consumer.i

Re: [PR] KAFKA-15913: Migrate async consumer tests to mocks [kafka]

2023-12-11 Thread via GitHub
lucasbru commented on code in PR #14930: URL: https://github.com/apache/kafka/pull/14930#discussion_r1422830104 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -42,192 +77,142 @@ import org.apache.kafka.clients.consumer.i

[jira] [Updated] (KAFKA-15950) Serialize broker heartbeat requests

2023-12-11 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-15950: Summary: Serialize broker heartbeat requests (was: CommunicationEvent should be scheduled with EarliestDe

[jira] [Updated] (KAFKA-15950) Serialize broker heartbeat requests

2023-12-11 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-15950: Description: {{KafkaEventQueue}} does de-duping and only allows one outstanding  {{CommunicationEvent}} in

[jira] [Updated] (KAFKA-15950) Serialize broker heartbeat requests

2023-12-11 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-15950: Description: {{KafkaEventQueue}} does de-duping and only allows one outstanding  {{CommunicationEvent}} in

[jira] [Updated] (KAFKA-15950) Serialize broker heartbeat requests

2023-12-11 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-15950: Description: This is a follow up issue from the discussion in https://github.com/apache/kafka/pull/14836#

[jira] [Updated] (KAFKA-15950) Serialize broker heartbeat requests

2023-12-11 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-15950: Description: This is a followup issue from the discussion in [https://github.com/apache/kafka/pull/14836#

Re: [PR] KAFKA-15913: Migrate async consumer tests to mocks [kafka]

2023-12-11 Thread via GitHub
cadonna commented on code in PR #14930: URL: https://github.com/apache/kafka/pull/14930#discussion_r1422846003 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -16,8 +16,43 @@ */ package org.apache.kafka.clients.consumer.

Re: [PR] MINOR: Add junit properties to display parameterized test names [kafka]

2023-12-11 Thread via GitHub
cadonna commented on PR #14983: URL: https://github.com/apache/kafka/pull/14983#issuecomment-1850549930 There was not a single streams failure in the builds: https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-14983/1/#showFailuresLink -- This is an automated message from the Apach

Re: [PR] KAFKA-15991: Fix flaky test testGroupIdNotNullAndValid [kafka]

2023-12-11 Thread via GitHub
cadonna commented on PR #14944: URL: https://github.com/apache/kafka/pull/14944#issuecomment-1850574652 @lucasbru This PR is ready to merge. Should I wait for https://github.com/apache/kafka/pull/14930 and close this since this is also solved that? Or should we just merge this as a temporar

Re: [PR] KAFKA-15883: Implement RemoteCopyLagBytes [kafka]

2023-12-11 Thread via GitHub
clolov commented on code in PR #14832: URL: https://github.com/apache/kafka/pull/14832#discussion_r1422903137 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -368,6 +403,16 @@ class BrokerTopicMetrics(name: Option[String], configOpt: java.util.Optional[Kaf

Re: [PR] KAFKA-15883: Implement RemoteCopyLagBytes [kafka]

2023-12-11 Thread via GitHub
clolov commented on code in PR #14832: URL: https://github.com/apache/kafka/pull/14832#discussion_r1422910744 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -386,7 +431,26 @@ class BrokerTopicMetrics(name: Option[String], configOpt: java.util.Optional[Kaf

Re: [PR] KAFKA-15883: Implement RemoteCopyLagBytes [kafka]

2023-12-11 Thread via GitHub
clolov commented on code in PR #14832: URL: https://github.com/apache/kafka/pull/14832#discussion_r1422910744 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -386,7 +431,26 @@ class BrokerTopicMetrics(name: Option[String], configOpt: java.util.Optional[Kaf

Re: [PR] Revert "MINOR: Add junit properties to display parameterized test names (#14687)" [kafka]

2023-12-11 Thread via GitHub
cadonna commented on PR #14961: URL: https://github.com/apache/kafka/pull/14961#issuecomment-1850662683 @alok123t I just read your comment. FYI, I worked on fixing this PR today. The [resulting PR](https://github.com/apache/kafka/pull/14983) is ready to merge. I just need an approval. --

Re: [PR] KAFKA-15417 flip joinSpuriousLookBackTimeMs and emit non-joined items [kafka]

2023-12-11 Thread via GitHub
lihaosky commented on PR #14426: URL: https://github.com/apache/kafka/pull/14426#issuecomment-1850672002 @mjsax , @guozhangwang , can we merge this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
philipnee commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1422983344 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -201,13 +224,35 @@ public long maximumTimeToWait(long current

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
philipnee commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1850676664 hi @cadonna - Sorry about unintentionally ignoring your question. I've just responded to it. -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] KAFKA-15143: Adding in MockFixedKeyProcessorContext and Test [kafka]

2023-12-11 Thread via GitHub
lihaosky commented on code in PR #14605: URL: https://github.com/apache/kafka/pull/14605#discussion_r1422983101 ## streams/test-utils/src/test/java/org/apache/kafka/streams/test/MockFixedKeyProcessorContextTest.java: ## @@ -0,0 +1,315 @@ +/* + * Licensed to the Apache Software F

Re: [PR] KAFKA-15913: Migrate async consumer tests to mocks [kafka]

2023-12-11 Thread via GitHub
lucasbru commented on code in PR #14930: URL: https://github.com/apache/kafka/pull/14930#discussion_r1423004412 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1535,6 +1591,11 @@ int callbacks() { return invoker.callb

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-11 Thread via GitHub
dajac commented on PR #14857: URL: https://github.com/apache/kafka/pull/14857#issuecomment-1850707387 I looked at all the failed tests and none of them are related to this PR. The "existing failure" are related to JBDO changes. Merging to trunk. -- This is an automated message from the Ap

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-11 Thread via GitHub
dajac merged PR #14857: URL: https://github.com/apache/kafka/pull/14857 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
philipnee commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1850712115 Hi @lianetm and @lucasbru - Per your request, I've just ran the tests using ``` Arguments.of("kraft+kip848", "consumer")) ``` All tests passed except `testRemoteAssig

[jira] [Resolved] (KAFKA-15574) Update states and transitions for membership manager state machine

2023-12-11 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-15574. - Fix Version/s: 3.7.0 Resolution: Fixed > Update states and transitions for membership man

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
lucasbru commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1850740142 @philipnee thanks for the updates. @cadonna I'll wait for you green light this by approving. -- This is an automated message from the Apache Git Service. To respond to the message, p

[PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac opened a new pull request, #14985: URL: https://github.com/apache/kafka/pull/14985 This patch wires the handling of makers written by the transaction coordinator via the WriteTxnMarkers API. In the old group coordinator, the markers are written to the logs and the group coordinator is

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1423029434 ## core/src/main/scala/kafka/coordinator/group/CoordinatorLoaderImpl.scala: ## @@ -135,7 +136,22 @@ class CoordinatorLoaderImpl[T]( memoryRecords.batches.f

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1423029751 ## core/src/main/scala/kafka/coordinator/group/CoordinatorPartitionWriter.scala: ## @@ -190,4 +170,56 @@ class CoordinatorPartitionWriter[T]( throw Errors.NOT_LE

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1423030220 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -2361,25 +2361,27 @@ class KafkaApis(val requestChannel: RequestChannel, * request, so there could be mu

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1423030686 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -2416,14 +2418,49 @@ class KafkaApis(val requestChannel: RequestChannel, numAppends.decrementAndGet()

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1423031807 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -881,6 +882,120 @@ public String toString() { }

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-11 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1423031514 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/OffsetMetadataManager.java: ## @@ -892,6 +893,43 @@ public void replay( } } +/**

Re: [PR] KAFKA-15991: Fix flaky test testGroupIdNotNullAndValid [kafka]

2023-12-11 Thread via GitHub
lucasbru commented on PR #14944: URL: https://github.com/apache/kafka/pull/14944#issuecomment-1850757955 @cadonna If you have more comments on #14930, please feel free to merge this, I will anyways have conflicts with every single CTR/KIP-848 PR. Instead, if #14930 is good to go, let

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-11 Thread via GitHub
lianetm commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1850787095 @philipnee thanks for the updates! One last comment. Important follow-up to this PR is to enable the specific max poll related integration tests (like `testMaxPollIntervalMs`), that are c

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1423062667 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -790,6 +812,197 @@ public void testOnSubscriptionUpdatedTra

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-11 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1423064727 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1499,6 +1575,149 @@ private void subscribeInternal(Collection top

  1   2   >