[jira] [Updated] (KAFKA-14401) Connector/Tasks reading offsets can get stuck if underneath WorkThread dies

2023-09-11 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Rao updated KAFKA-14401: -- Description: When a connector or task tries to read the offsets from the offsets topic, it issues `Of

[GitHub] [kafka] vamossagar12 opened a new pull request, #14372: KAFKA-14401: Fail kafka log read end requests if underneath work thread is dead

2023-09-11 Thread via GitHub
vamossagar12 opened a new pull request, #14372: URL: https://github.com/apache/kafka/pull/14372 Please refer to [KAFKA-14401](https://issues.apache.org/jira/browse/KAFKA-14401) for a detailed description of the problem that this PR aims to solve. -- This is an automated message from the

[GitHub] [kafka] satishd commented on pull request #14347: KAFKA-15439: Transactions test with tiered storage

2023-09-11 Thread via GitHub
satishd commented on PR #14347: URL: https://github.com/apache/kafka/pull/14347#issuecomment-1715057569 Thanks @kamalcph for the updated PR, I will merge the changes if there are no related test failures in jenkins jobs once they finish. -- This is an automated message from the Apache Git

[GitHub] [kafka] JimmyWang6 commented on pull request #14327: KAFKA-14048:Implement regex subscriptions

2023-09-11 Thread via GitHub
JimmyWang6 commented on PR #14327: URL: https://github.com/apache/kafka/pull/14327#issuecomment-1715053133 @dajac Almost done. Could you pls help to review the code? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Assigned] (KAFKA-15453) Enable `testFencingOnTransactionExpiration` in TransactionsWithTieredStoreTest

2023-09-11 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen reassigned KAFKA-15453: - Assignee: Luke Chen > Enable `testFencingOnTransactionExpiration` in TransactionsWithTieredStor

[GitHub] [kafka] satishd commented on a diff in pull request #14347: KAFKA-15439: Transactions test with tiered storage

2023-09-11 Thread via GitHub
satishd commented on code in PR #14347: URL: https://github.com/apache/kafka/pull/14347#discussion_r1322433575 ## storage/src/test/java/org/apache/kafka/tiered/storage/integration/TransactionsWithTieredStoreTest.java: ## @@ -0,0 +1,135 @@ +/* + * Licensed to the Apache Software

[jira] [Created] (KAFKA-15453) Enable `testFencingOnTransactionExpiration` in TransactionsWithTieredStoreTest

2023-09-11 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15453: -- Summary: Enable `testFencingOnTransactionExpiration` in TransactionsWithTieredStoreTest Key: KAFKA-15453 URL: https://issues.apache.org/jira/browse/KAFKA-15453 Pr

[GitHub] [kafka] satishd commented on pull request #14347: KAFKA-15439: Transactions test with tiered storage

2023-09-11 Thread via GitHub
satishd commented on PR #14347: URL: https://github.com/apache/kafka/pull/14347#issuecomment-1715012947 Thanks @kamalcph for the updated PR, I will merge the changes if there are no related test failures in jenkins jobs. -- This is an automated message from the Apache Git Service. To resp

[jira] [Assigned] (KAFKA-14855) Harden integration testing logic for asserting that a connector is deleted

2023-09-11 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya reassigned KAFKA-14855: -- Assignee: Yash Mayya > Harden integration testing logic for asserting that a connector is del

[GitHub] [kafka] yashmayya opened a new pull request, #14371: KAFKA-14855: Harden integration testing logic for asserting that a connector is deleted

2023-09-11 Thread via GitHub
yashmayya opened a new pull request, #14371: URL: https://github.com/apache/kafka/pull/14371 From https://issues.apache.org/jira/browse/KAFKA-14855: > In the Connect embedded integration testing framework, the [EmbeddedConnectClusterAssertions::assertConnectorAndTasksAreStopped metho

[GitHub] [kafka] kamalcph commented on pull request #14347: KAFKA-15439: Transactions test with tiered storage

2023-09-11 Thread via GitHub
kamalcph commented on PR #14347: URL: https://github.com/apache/kafka/pull/14347#issuecomment-1715003976 @showuon @satishd Fixed the scala 2.12 compilation failures with the latest commit. -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[jira] [Resolved] (KAFKA-15352) Ensure consistency while deleting the remote log segments

2023-09-11 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana resolved KAFKA-15352. Resolution: Fixed > Ensure consistency while deleting the remote log segments > --

[GitHub] [kafka] satishd merged pull request #14349: KAFKA-15352: Update log-start-offset before initiating deletion of remote segments

2023-09-11 Thread via GitHub
satishd merged PR #14349: URL: https://github.com/apache/kafka/pull/14349 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] satishd commented on pull request #14349: KAFKA-15352: Update log-start-offset before initiating deletion of remote segments

2023-09-11 Thread via GitHub
satishd commented on PR #14349: URL: https://github.com/apache/kafka/pull/14349#issuecomment-1714949944 There are a few unrelated test failures. Merging it to trunk and 3.6 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [kafka] lianetm commented on pull request #14346: KAFKA-15115 - KAFKA-15163; Reset/Validate positions implementation & API integration

2023-09-11 Thread via GitHub
lianetm commented on PR #14346: URL: https://github.com/apache/kafka/pull/14346#issuecomment-1714883627 Unit tests added for the validate positions, it was definitely missing. Also minor fix for a param that ended up wrong in the last merge with the exponential backoff change. Update

[GitHub] [kafka] lianetm commented on a diff in pull request #14346: KAFKA-15115 - KAFKA-15163; Reset/Validate positions implementation & API integration

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14346: URL: https://github.com/apache/kafka/pull/14346#discussion_r1322293224 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/OffsetsRequestManagerTest.java: ## @@ -455,6 +463,89 @@ public void testRequestFails_Authentication

[GitHub] [kafka] JimmyWang6 commented on pull request #14327: KAFKA-14048:Implement regex subscriptions

2023-09-11 Thread via GitHub
JimmyWang6 commented on PR #14327: URL: https://github.com/apache/kafka/pull/14327#issuecomment-1714879176 @dajac Almost done. Could you pls help to review the code? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Updated] (KAFKA-15452) Custom KafkaPrincipalBuilder Cannot Access SslPrincipalMapper

2023-09-11 Thread Raghu Baddam (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Baddam updated KAFKA-15452: - Description: When implementing a custom KafkaPrincipalBuilder, it is not possible to access *Ss

[jira] [Updated] (KAFKA-15452) Custom KafkaPrincipalBuilder Cannot Access SslPrincipalMapper

2023-09-11 Thread Raghu Baddam (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Baddam updated KAFKA-15452: - Description: When implementing a custom KafkaPrincipalBuilder, it is not possible to access *Ss

[jira] [Updated] (KAFKA-15452) Custom KafkaPrincipalBuilder Cannot Access SslPrincipalMapper

2023-09-11 Thread Raghu Baddam (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Baddam updated KAFKA-15452: - Summary: Custom KafkaPrincipalBuilder Cannot Access SslPrincipalMapper (was: Custom Principal B

[GitHub] [kafka] splett2 commented on a diff in pull request #14053: KAFKA-15221; Fix the race between fetch requests from a rebooted follower.

2023-09-11 Thread via GitHub
splett2 commented on code in PR #14053: URL: https://github.com/apache/kafka/pull/14053#discussion_r1322060958 ## core/src/main/scala/kafka/cluster/Replica.scala: ## @@ -98,14 +103,21 @@ class Replica(val brokerId: Int, val topicPartition: TopicPartition) extends Log * fetc

[jira] [Created] (KAFKA-15452) Custom Principal Builders Can't Access SslPrincipalMapper

2023-09-11 Thread Raghu Baddam (Jira)
Raghu Baddam created KAFKA-15452: Summary: Custom Principal Builders Can't Access SslPrincipalMapper Key: KAFKA-15452 URL: https://issues.apache.org/jira/browse/KAFKA-15452 Project: Kafka Iss

[GitHub] [kafka] showuon commented on pull request #14347: KAFKA-15439: Transactions test with tiered storage

2023-09-11 Thread via GitHub
showuon commented on PR #14347: URL: https://github.com/apache/kafka/pull/14347#issuecomment-1714835205 @kamalcph , there's a compile error. Please help fix it. Thanks. https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-14347/9/pipeline -- This is an aut

[GitHub] [kafka] mjsax commented on pull request #14360: KAFKA-15307: update/note deprecated configs

2023-09-11 Thread via GitHub
mjsax commented on PR #14360: URL: https://github.com/apache/kafka/pull/14360#issuecomment-1714822586 > There might be some discussion to be had regarding what to say about why these are deprecated. Not sure if we need to put this into the docs? If we want to say something about it,

[GitHub] [kafka] mjsax commented on a diff in pull request #14360: KAFKA-15307: update/note deprecated configs

2023-09-11 Thread via GitHub
mjsax commented on code in PR #14360: URL: https://github.com/apache/kafka/pull/14360#discussion_r1322251972 ## docs/streams/developer-guide/config-streams.html: ## @@ -259,7 +259,7 @@ num.standby.replicascache.max.bytes.buffering Medium -Maximum number

[GitHub] [kafka] jolshan commented on a diff in pull request #14370: KAFKA-15449: Verify transactional offset commits (KIP-890 part 1)

2023-09-11 Thread via GitHub
jolshan commented on code in PR #14370: URL: https://github.com/apache/kafka/pull/14370#discussion_r135993 ## clients/src/main/java/org/apache/kafka/common/requests/TxnOffsetCommitResponse.java: ## @@ -47,6 +47,9 @@ * - {@link Errors#UNKNOWN_MEMBER_ID} * - {@link Err

[GitHub] [kafka] jolshan commented on a diff in pull request #14370: KAFKA-15449: Verify transactional offset commits (KIP-890 part 1)

2023-09-11 Thread via GitHub
jolshan commented on code in PR #14370: URL: https://github.com/apache/kafka/pull/14370#discussion_r1322167160 ## clients/src/main/java/org/apache/kafka/common/requests/TxnOffsetCommitResponse.java: ## @@ -47,6 +47,9 @@ * - {@link Errors#UNKNOWN_MEMBER_ID} * - {@link Err

[GitHub] [kafka] jolshan commented on a diff in pull request #14370: KAFKA-15449: Verify transactional offset commits (KIP-890 part 1)

2023-09-11 Thread via GitHub
jolshan commented on code in PR #14370: URL: https://github.com/apache/kafka/pull/14370#discussion_r1322167160 ## clients/src/main/java/org/apache/kafka/common/requests/TxnOffsetCommitResponse.java: ## @@ -47,6 +47,9 @@ * - {@link Errors#UNKNOWN_MEMBER_ID} * - {@link Err

[GitHub] [kafka] jolshan commented on a diff in pull request #14370: KAFKA-15449: Verify transactional offset commits (KIP-890 part 1)

2023-09-11 Thread via GitHub
jolshan commented on code in PR #14370: URL: https://github.com/apache/kafka/pull/14370#discussion_r1322167160 ## clients/src/main/java/org/apache/kafka/common/requests/TxnOffsetCommitResponse.java: ## @@ -47,6 +47,9 @@ * - {@link Errors#UNKNOWN_MEMBER_ID} * - {@link Err

[GitHub] [kafka] jolshan commented on a diff in pull request #14370: KAFKA-15449: Verify transactional offset commits (KIP-890 part 1)

2023-09-11 Thread via GitHub
jolshan commented on code in PR #14370: URL: https://github.com/apache/kafka/pull/14370#discussion_r1322167160 ## clients/src/main/java/org/apache/kafka/common/requests/TxnOffsetCommitResponse.java: ## @@ -47,6 +47,9 @@ * - {@link Errors#UNKNOWN_MEMBER_ID} * - {@link Err

[GitHub] [kafka] jolshan commented on a diff in pull request #14370: KAFKA-15449: Verify transactional offset commits (KIP-890 part 1)

2023-09-11 Thread via GitHub
jolshan commented on code in PR #14370: URL: https://github.com/apache/kafka/pull/14370#discussion_r1322167160 ## clients/src/main/java/org/apache/kafka/common/requests/TxnOffsetCommitResponse.java: ## @@ -47,6 +47,9 @@ * - {@link Errors#UNKNOWN_MEMBER_ID} * - {@link Err

[GitHub] [kafka] junrao commented on a diff in pull request #14346: KAFKA-15115 - KAFKA-15163; Reset/Validate positions implementation & API integration

2023-09-11 Thread via GitHub
junrao commented on code in PR #14346: URL: https://github.com/apache/kafka/pull/14346#discussion_r1322174011 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/OffsetsRequestManagerTest.java: ## @@ -455,6 +463,89 @@ public void testRequestFails_AuthenticationE

[GitHub] [kafka] jolshan commented on a diff in pull request #14370: KAFKA-15449: Verify transactional offset commits (KIP-890 part 1)

2023-09-11 Thread via GitHub
jolshan commented on code in PR #14370: URL: https://github.com/apache/kafka/pull/14370#discussion_r1322167160 ## clients/src/main/java/org/apache/kafka/common/requests/TxnOffsetCommitResponse.java: ## @@ -47,6 +47,9 @@ * - {@link Errors#UNKNOWN_MEMBER_ID} * - {@link Err

[GitHub] [kafka] jolshan commented on a diff in pull request #14370: KAFKA-15449: Verify transactional offset commits (KIP-890 part 1)

2023-09-11 Thread via GitHub
jolshan commented on code in PR #14370: URL: https://github.com/apache/kafka/pull/14370#discussion_r1322166759 ## clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java: ## @@ -51,6 +51,11 @@ * {@link Errors#CLUSTER_AUTHORIZATION_FAILED} * {@link Errors#

[GitHub] [kafka] jolshan opened a new pull request, #14370: KAFKA-15449: Verify transactional offset commits (KIP-890 part 1)

2023-09-11 Thread via GitHub
jolshan opened a new pull request, #14370: URL: https://github.com/apache/kafka/pull/14370 Previous commits left out TxnOffsetCommits which go through the group coordinator (not directly from the producer). I've wired up the methods to include the transactional id and state partition

[GitHub] [kafka] junrao commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-11 Thread via GitHub
junrao commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1322084716 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchBuffer.java: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] [kafka] soarez opened a new pull request, #14369: KAFKA-15357: Aggregate and propagate assignments and logdir failures

2023-09-11 Thread via GitHub
soarez opened a new pull request, #14369: URL: https://github.com/apache/kafka/pull/14369 A new DirectoryEventManager accumulates, batches, and sends assignments or failure events to the Controller, prioritizing assignments to ensure the Controller has the correct assignment view before

[GitHub] [kafka] soarez opened a new pull request, #14368: KAFKA-15451: Send offline dirs in broker heartbeat

2023-09-11 Thread via GitHub
soarez opened a new pull request, #14368: URL: https://github.com/apache/kafka/pull/14368 Offline log directories can be propagated via BrokerHeartbeatRequest. The broker halts if it fails to propagate the failure to the controller within the log directory failure timeout window.

[GitHub] [kafka] mumrah commented on pull request #14367: KAFKA-15450 Don't allow ZK migration with JBOD

2023-09-11 Thread via GitHub
mumrah commented on PR #14367: URL: https://github.com/apache/kafka/pull/14367#issuecomment-1714674132 Test failures look unrelated https://github.com/apache/kafka/assets/55116/3f25b731-593f-4809-97ea-47ceca61dc03";> -- This is an automated message from the Apache Git Servic

[GitHub] [kafka] lianetm commented on pull request #14346: KAFKA-15115 - KAFKA-15163; Reset/Validate positions implementation & API integration

2023-09-11 Thread via GitHub
lianetm commented on PR #14346: URL: https://github.com/apache/kafka/pull/14346#issuecomment-1714633959 Thanks for the review @junrao . All comments addressed, only pending is adding more unit tests for the validatePositions. I'm adding them and will update shortly -- This is an automate

[jira] [Updated] (KAFKA-15451) Send offline dirs in broker heartbeat\

2023-09-11 Thread Igor Soarez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Soarez updated KAFKA-15451: Summary: Send offline dirs in broker heartbeat\ (was: Include offline dirs in BrokerHeartbeatRequ

[jira] [Updated] (KAFKA-15451) Send offline dirs in broker heartbeat

2023-09-11 Thread Igor Soarez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Soarez updated KAFKA-15451: Summary: Send offline dirs in broker heartbeat (was: Send offline dirs in broker heartbeat\) > S

[GitHub] [kafka] lianetm commented on a diff in pull request #14346: KAFKA-15115 - KAFKA-15163; Reset/Validate positions implementation & API integration

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14346: URL: https://github.com/apache/kafka/pull/14346#discussion_r1322093959 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/DefaultBackgroundThreadTest.java: ## @@ -181,6 +187,89 @@ public void testListOffsetsEventIsProcess

[jira] [Updated] (KAFKA-15438) Review exception caching logic used for reset/validate positions in async consumer

2023-09-11 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-15438: --- Description: The refactored async consumer reuses part of the core logic required for reset

[jira] [Created] (KAFKA-15451) Include offline dirs in BrokerHeartbeatRequest

2023-09-11 Thread Igor Soarez (Jira)
Igor Soarez created KAFKA-15451: --- Summary: Include offline dirs in BrokerHeartbeatRequest Key: KAFKA-15451 URL: https://issues.apache.org/jira/browse/KAFKA-15451 Project: Kafka Issue Type: Sub-

[GitHub] [kafka] lianetm commented on a diff in pull request #14346: KAFKA-15115 - KAFKA-15163; Reset/Validate positions implementation & API integration

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14346: URL: https://github.com/apache/kafka/pull/14346#discussion_r1322073098 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ValidatePositionsApplicationEvent.java: ## @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache S

[jira] [Updated] (KAFKA-15357) Propagate assignments and logdir failures events to controller

2023-09-11 Thread Igor Soarez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Soarez updated KAFKA-15357: Description: A new DirectoryEventManager accumulates, batches, and sends assignments or failure e

[jira] [Updated] (KAFKA-15357) Aggregate and propagate assignments and logdir failures

2023-09-11 Thread Igor Soarez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Soarez updated KAFKA-15357: Summary: Aggregate and propagate assignments and logdir failures (was: Propagate assignments and

[GitHub] [kafka] lianetm commented on a diff in pull request #14346: KAFKA-15115 - KAFKA-15163; Reset/Validate positions implementation & API integration

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14346: URL: https://github.com/apache/kafka/pull/14346#discussion_r1322072099 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ResetPositionsApplicationEvent.java: ## @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Soft

[GitHub] [kafka] lianetm commented on a diff in pull request #14346: KAFKA-15115 - KAFKA-15163; Reset/Validate positions implementation & API integration

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14346: URL: https://github.com/apache/kafka/pull/14346#discussion_r1322070445 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ResetPositionsApplicationEvent.java: ## @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Soft

[GitHub] [kafka] splett2 commented on a diff in pull request #14053: KAFKA-15221; Fix the race between fetch requests from a rebooted follower.

2023-09-11 Thread via GitHub
splett2 commented on code in PR #14053: URL: https://github.com/apache/kafka/pull/14053#discussion_r1322065839 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -858,7 +859,7 @@ class Partition(val topicPartition: TopicPartition, // No need to calculate low waterm

[GitHub] [kafka] splett2 commented on a diff in pull request #14053: KAFKA-15221; Fix the race between fetch requests from a rebooted follower.

2023-09-11 Thread via GitHub
splett2 commented on code in PR #14053: URL: https://github.com/apache/kafka/pull/14053#discussion_r1322065839 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -858,7 +859,7 @@ class Partition(val topicPartition: TopicPartition, // No need to calculate low waterm

[GitHub] [kafka] splett2 commented on a diff in pull request #14053: KAFKA-15221; Fix the race between fetch requests from a rebooted follower.

2023-09-11 Thread via GitHub
splett2 commented on code in PR #14053: URL: https://github.com/apache/kafka/pull/14053#discussion_r1322060958 ## core/src/main/scala/kafka/cluster/Replica.scala: ## @@ -98,14 +103,21 @@ class Replica(val brokerId: Int, val topicPartition: TopicPartition) extends Log * fetc

[GitHub] [kafka] lianetm commented on a diff in pull request #14346: KAFKA-15115 - KAFKA-15163; Reset/Validate positions implementation & API integration

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14346: URL: https://github.com/apache/kafka/pull/14346#discussion_r1322059077 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/OffsetsForLeaderEpochUtils.java: ## @@ -0,0 +1,142 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [kafka] lianetm commented on a diff in pull request #14346: KAFKA-15115 - KAFKA-15163; Reset/Validate positions implementation & API integration

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14346: URL: https://github.com/apache/kafka/pull/14346#discussion_r1322041495 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/OffsetsRequestManager.java: ## @@ -154,6 +170,52 @@ public CompletableFuture> fetchOffsets(

[GitHub] [kafka] lianetm commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1322027402 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignorSelection.java: ## @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] [kafka] lianetm commented on pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
lianetm commented on PR #14323: URL: https://github.com/apache/kafka/pull/14323#issuecomment-1714511273 Thanks for the comments @dajac, all addressed. There is still a lot to think about regarding how the client will handle the assignments it receives. In order to unblock the integra

[GitHub] [kafka] philipnee commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
philipnee commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1322009365 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] lianetm commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1322000996 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] lianetm commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1322000996 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] lianetm commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1322000996 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] lianetm commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1321733445 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] mumrah opened a new pull request, #14367: KAFKA-15450 Don't allow ZK migration with JBOD

2023-09-11 Thread via GitHub
mumrah opened a new pull request, #14367: URL: https://github.com/apache/kafka/pull/14367 This patch disables broker startup if ZK migrations are enabled while JBOD is also enabled. Once [KIP-858](https://cwiki.apache.org/confluence/display/KAFKA/KIP-858%3A+Handle+JBOD+broker+disk+failure+i

[GitHub] [kafka] philipnee commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
philipnee commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1321933274 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] philipnee commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
philipnee commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1321926651 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software Foundati

[jira] [Updated] (KAFKA-15450) Disable ZK migration when JBOD configured

2023-09-11 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-15450: - Description: Since JBOD is not yet supported in KRaft (see [KIP-858|https://cwiki.apache.org/co

[jira] [Created] (KAFKA-15450) Disable ZK migration when JBOD configured

2023-09-11 Thread David Arthur (Jira)
David Arthur created KAFKA-15450: Summary: Disable ZK migration when JBOD configured Key: KAFKA-15450 URL: https://issues.apache.org/jira/browse/KAFKA-15450 Project: Kafka Issue Type: Bug

[GitHub] [kafka] cmccabe merged pull request #14356: MINOR: rename BrokerToControllerChannelManager to NodeToControllerChannelManager

2023-09-11 Thread via GitHub
cmccabe merged PR #14356: URL: https://github.com/apache/kafka/pull/14356 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Resolved] (KAFKA-14499) Implement OffsetFetch/OffsetCommit APIs

2023-09-11 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-14499. - Fix Version/s: 3.7.0 Resolution: Fixed > Implement OffsetFetch/OffsetCommit APIs > --

[GitHub] [kafka] dajac merged pull request #14353: KAFKA-14499: [7/7] Add integration tests for OffsetCommit API and OffsetFetch API

2023-09-11 Thread via GitHub
dajac merged PR #14353: URL: https://github.com/apache/kafka/pull/14353 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] philipnee commented on a diff in pull request #14362: KAFKA-14960: TopicMetadata request manager

2023-09-11 Thread via GitHub
philipnee commented on code in PR #14362: URL: https://github.com/apache/kafka/pull/14362#discussion_r1321852308 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CompletableRequest.java: ## @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [kafka] philipnee commented on pull request #14305: KAFKA-14274: [1/7] basic refactoring

2023-09-11 Thread via GitHub
philipnee commented on PR #14305: URL: https://github.com/apache/kafka/pull/14305#issuecomment-1714274031 @divijvaidya - thank you for looking into this proactively! 🤝 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [kafka] divijvaidya commented on pull request #14305: KAFKA-14274: [1/7] basic refactoring

2023-09-11 Thread via GitHub
divijvaidya commented on PR #14305: URL: https://github.com/apache/kafka/pull/14305#issuecomment-1714262640 > Does this metric measure the trunk or all draft/pr? Thank you for looking into this @philipnee. You can add "trunk" as a tag in the above link to get data on trunk. Trunk seem

[GitHub] [kafka] philipnee commented on a diff in pull request #14362: KAFKA-14960: TopicMetadata request manager

2023-09-11 Thread via GitHub
philipnee commented on code in PR #14362: URL: https://github.com/apache/kafka/pull/14362#discussion_r1321832143 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -127,9 +127,9 @@ public void maybeAutoCommit(final Map offsets

[jira] [Updated] (KAFKA-15449) Verify transactional offset commits (KIP-890 part 1)

2023-09-11 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan updated KAFKA-15449: --- Summary: Verify transactional offset commits (KIP-890 part 1) (was: Verify Offset Commits t

[jira] [Resolved] (KAFKA-15380) Try complete actions after callback

2023-09-11 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-15380. Resolution: Fixed > Try complete actions after callback >

[jira] [Resolved] (KAFKA-15424) Make verification a dynamic configuration

2023-09-11 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-15424. Resolution: Fixed > Make verification a dynamic configuration > --

[jira] [Created] (KAFKA-15449) Verify Offset Commits too

2023-09-11 Thread Justine Olshan (Jira)
Justine Olshan created KAFKA-15449: -- Summary: Verify Offset Commits too Key: KAFKA-15449 URL: https://issues.apache.org/jira/browse/KAFKA-15449 Project: Kafka Issue Type: Sub-task

[GitHub] [kafka] philipnee commented on pull request #14305: KAFKA-14274: [1/7] basic refactoring

2023-09-11 Thread via GitHub
philipnee commented on PR #14305: URL: https://github.com/apache/kafka/pull/14305#issuecomment-1714240650 Hey @divijvaidya - Looking at the PR: in fact there isn't any implementation level changes, so I'm not sure why these two functions started to become flaky. Does this metric measure th

[GitHub] [kafka] jolshan commented on a diff in pull request #14353: KAFKA-14499: [7/7] Add integration tests for OffsetCommit API and OffsetFetch API

2023-09-11 Thread via GitHub
jolshan commented on code in PR #14353: URL: https://github.com/apache/kafka/pull/14353#discussion_r1321818294 ## core/src/test/scala/unit/kafka/server/OffsetCommitRequestTest.scala: ## @@ -0,0 +1,163 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] [kafka] dajac commented on a diff in pull request #14353: KAFKA-14499: [7/7] Add integration tests for OffsetCommit API and OffsetFetch API

2023-09-11 Thread via GitHub
dajac commented on code in PR #14353: URL: https://github.com/apache/kafka/pull/14353#discussion_r1321808017 ## core/src/test/scala/unit/kafka/server/OffsetCommitRequestTest.scala: ## @@ -0,0 +1,163 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

[GitHub] [kafka] lianetm commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1321783061 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] lianetm commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1321783061 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] lianetm commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1321737185 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] lianetm commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1321737185 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] lianetm commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
lianetm commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1321733445 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] pranavrth commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
pranavrth commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1321724746 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] dajac commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
dajac commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1321719789 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [kafka] dajac commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
dajac commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1321717875 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [kafka] jolshan commented on a diff in pull request #14353: KAFKA-14499: [7/7] Add integration tests for OffsetCommit API and OffsetFetch API

2023-09-11 Thread via GitHub
jolshan commented on code in PR #14353: URL: https://github.com/apache/kafka/pull/14353#discussion_r1321717776 ## core/src/test/scala/unit/kafka/server/OffsetCommitRequestTest.scala: ## @@ -0,0 +1,163 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] [kafka] jolshan commented on a diff in pull request #14353: KAFKA-14499: [7/7] Add integration tests for OffsetCommit API and OffsetFetch API

2023-09-11 Thread via GitHub
jolshan commented on code in PR #14353: URL: https://github.com/apache/kafka/pull/14353#discussion_r1321714957 ## core/src/test/scala/unit/kafka/server/GroupCoordinatorBaseRequestTest.scala: ## @@ -0,0 +1,158 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] dajac commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
dajac commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1321714750 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [kafka] philipnee commented on pull request #14305: KAFKA-14274: [1/7] basic refactoring

2023-09-11 Thread via GitHub
philipnee commented on PR #14305: URL: https://github.com/apache/kafka/pull/14305#issuecomment-1714085023 @divijvaidya - I was trying to cherry pick some additional changes into trunk however, i actually don't find it flaky - I think these two should be failing constantly (causing by comple

[GitHub] [kafka] mimaison commented on pull request #13842: KAFKA-14995: Automate asf.yaml collaborators refresh

2023-09-11 Thread via GitHub
mimaison commented on PR #13842: URL: https://github.com/apache/kafka/pull/13842#issuecomment-1714074460 @stevenbooke Thanks for the updates and sorry for the delay. I took another look at this PR today. When trying to run the script, I get the following error: ``` Traceback (most

[GitHub] [kafka] philipnee commented on pull request #14305: KAFKA-14274: [1/7] basic refactoring

2023-09-11 Thread via GitHub
philipnee commented on PR #14305: URL: https://github.com/apache/kafka/pull/14305#issuecomment-1714066658 @divijvaidya - thanks for reporting that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] philipnee commented on a diff in pull request #14323: KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-11 Thread via GitHub
philipnee commented on code in PR #14323: URL: https://github.com/apache/kafka/pull/14323#discussion_r1321687226 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] divijvaidya commented on pull request #14330: KAFKA-15410: Delete records with tiered storage integration test (4/4)

2023-09-11 Thread via GitHub
divijvaidya commented on PR #14330: URL: https://github.com/apache/kafka/pull/14330#issuecomment-1714015827 Folks the test introduced in this PR has been flaky lately - https://ge.apache.org/scans/tests?search.rootProjectNames=kafka&search.timeZoneId=Europe/Berlin&tests.container=org.apache.

[GitHub] [kafka] kirktrue closed pull request #14095: [DON'T MERGE] Added whitespace to README

2023-09-11 Thread via GitHub
kirktrue closed pull request #14095: [DON'T MERGE] Added whitespace to README URL: https://github.com/apache/kafka/pull/14095 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14363: KAFKA-14993: Improve TransactionIndex instance handling while copying to and fetching from RSM

2023-09-11 Thread via GitHub
divijvaidya commented on code in PR #14363: URL: https://github.com/apache/kafka/pull/14363#discussion_r1321639606 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -523,7 +533,9 @@ public void markForCleanup() throws IOException {

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14363: KAFKA-14993: Improve TransactionIndex instance handling while copying to and fetching from RSM

2023-09-11 Thread via GitHub
divijvaidya commented on code in PR #14363: URL: https://github.com/apache/kafka/pull/14363#discussion_r1321638678 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -382,6 +388,9 @@ private RemoteIndexCache.Entry createCacheEntry(Remot

  1   2   >