[jira] [Updated] (KAFKA-15376) Explore options of removing data earlier to the current leader's leader epoch lineage for topics enabled with tiered storage.

2023-08-17 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-15376: --- Summary: Explore options of removing data earlier to the current leader's leader epoch linea

[jira] [Comment Edited] (KAFKA-15339) Transient I/O error happening in appending records could lead to the halt of whole cluster

2023-08-17 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755812#comment-17755812 ] Sagar Rao edited comment on KAFKA-15339 at 8/18/23 5:52 AM:

[jira] [Commented] (KAFKA-15339) Transient I/O error happening in appending records could lead to the halt of whole cluster

2023-08-17 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755812#comment-17755812 ] Sagar Rao commented on KAFKA-15339: --- Thanks for reporting this bug [~functioner]. As y

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-17 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1298027732 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -696,11 +704,327 @@ public void run() { } } +public void handleLogSt

[jira] [Created] (KAFKA-15376) Revisit removing data earlier to the current leader for topics enabled with tiered storage.

2023-08-17 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15376: -- Summary: Revisit removing data earlier to the current leader for topics enabled with tiered storage. Key: KAFKA-15376 URL: https://issues.apache.org/jira/browse/KAFKA-15376

[jira] [Updated] (KAFKA-15376) Explore options of removing data earlier to the current leader for topics enabled with tiered storage.

2023-08-17 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-15376: --- Summary: Explore options of removing data earlier to the current leader for topics enabled w

[GitHub] [kafka] satishd commented on pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-17 Thread via GitHub
satishd commented on PR #13947: URL: https://github.com/apache/kafka/pull/13947#issuecomment-1683388925 I will merge the PR to unblock other integration test related PRs as it has got 2 +1s(Jun and me). Remaining items if any can be addressed in followup PRs. -- This is an automat

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-17 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1298029817 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -696,11 +704,327 @@ public void run() { } } +public void handleLogSt

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-17 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1298027732 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -696,11 +704,327 @@ public void run() { } } +public void handleLogSt

[jira] [Commented] (KAFKA-15354) Partition leader is not evenly distributed in kraft mode

2023-08-17 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755805#comment-17755805 ] Sagar Rao commented on KAFKA-15354: --- [~dengziming], I took a look at this. I believe t

[GitHub] [kafka] satishd commented on pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-17 Thread via GitHub
satishd commented on PR #13947: URL: https://github.com/apache/kafka/pull/13947#issuecomment-1683345956 Failed tests do not seem to be related. @kamalcph Please check and confirm. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [kafka] satishd commented on pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-17 Thread via GitHub
satishd commented on PR #13947: URL: https://github.com/apache/kafka/pull/13947#issuecomment-1683345343 I will merge the PR to unblock other integration test related PRs. Remaining items if any can be addressed in followup PRs. -- This is an automated message from the Apache Git Service.

[GitHub] [kafka] haodanchen opened a new pull request, #14241: Remove consumer rejoin to the group's synchronized

2023-08-17 Thread via GitHub
haodanchen opened a new pull request, #14241: URL: https://github.com/apache/kafka/pull/14241 - First, the purpose of the heart-beat thread's `maybeLeave` is to help "fail-fast", so we do our bust not to block it. And if the consumer thread release lock, and heart-beat thread modifies `gene

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-17 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1297944016 ## metadata/src/main/java/org/apache/kafka/image/DelegationTokenDelta.java: ## @@ -0,0 +1,101 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] haodanchen closed pull request #14240: Update AbstractCoordinator.java

2023-08-17 Thread via GitHub
haodanchen closed pull request #14240: Update AbstractCoordinator.java URL: https://github.com/apache/kafka/pull/14240 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-17 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1297930601 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-17 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1297918868 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -2999,7 +2999,38 @@ class KafkaApis(val requestChannel: RequestChannel, } def handleCreateTokenRe

[jira] [Resolved] (KAFKA-15345) KRaft leader should notify the listener only when it has read up to the leader's epoch

2023-08-17 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-15345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] José Armando García Sancio resolved KAFKA-15345. Resolution: Fixed > KRaft leader should notify the listener only w

[GitHub] [kafka] haodanchen opened a new pull request, #14240: Update AbstractCoordinator.java

2023-08-17 Thread via GitHub
haodanchen opened a new pull request, #14240: URL: https://github.com/apache/kafka/pull/14240 Remove synchronized to improve performance. 1. Use the **_state_** directory may help fast fail, because it is the newly value. 2. Without **_state_** snapshot, just need **_generation_** snap

[GitHub] [kafka] jsancio merged pull request #14213: KAFKA-15345; KRaft leader notifies when listener reaches epoch start

2023-08-17 Thread via GitHub
jsancio merged PR #14213: URL: https://github.com/apache/kafka/pull/14213 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] jsancio commented on pull request #14213: KAFKA-15345; KRaft leader notifies when listener reaches epoch start

2023-08-17 Thread via GitHub
jsancio commented on PR #14213: URL: https://github.com/apache/kafka/pull/14213#issuecomment-1683208168 Unable to reproduce the test failures locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] satishd commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-17 Thread via GitHub
satishd commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1297876409 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -166,6 +166,10 @@ public Cache internalCache() { return internalCa

[GitHub] [kafka] vincent81jiang opened a new pull request, #14239: KAFKA-15375: do not write clean shutdown marker file for a log dir if startup has not completed recovering it

2023-08-17 Thread via GitHub
vincent81jiang opened a new pull request, #14239: URL: https://github.com/apache/kafka/pull/14239 When running in kraft mode, LogManager.startup is called in a different thread than the main broker startup thread (by BrokerMetadataPublisher when the first metadata update is received.) If

[GitHub] [kafka] kamalcph opened a new pull request, #14238: KAFKA-15290: Handle topic-level dynamic remote storage enable configuration

2023-08-17 Thread via GitHub
kamalcph opened a new pull request, #14238: URL: https://github.com/apache/kafka/pull/14238 To migrate the existing topics to remote log storage, we have to handle the dynamic topic-level config update to enable remote log storage. *More detailed description of your change, if nece

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-17 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1297648037 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-17 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1297647792 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] kamalcph commented on pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-17 Thread via GitHub
kamalcph commented on PR #13947: URL: https://github.com/apache/kafka/pull/13947#issuecomment-1682800792 @junrao Test failures are unrelated to this PR. I've ran them locally, they were succeeded: ``` Build / JDK 11 and Scala 2.13 / [2] tlsProtocol=TLSv1.2, useInlinePem=true –

[jira] [Assigned] (KAFKA-15375) When running in KRaft mode, LogManager may creates CleanShutdown file by mistake

2023-08-17 Thread Vincent Jiang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent Jiang reassigned KAFKA-15375: - Assignee: Vincent Jiang > When running in KRaft mode, LogManager may creates CleanShutd

[jira] [Updated] (KAFKA-15375) When running in KRaft mode, LogManager may creates CleanShutdown file by mistake

2023-08-17 Thread Vincent Jiang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent Jiang updated KAFKA-15375: -- Description: Consider following sequence when running Kafka in KRaft mode: # A partition log

[jira] [Updated] (KAFKA-15375) When running in KRaft mode, LogManager may creates CleanShutdown file by mistake

2023-08-17 Thread Vincent Jiang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vincent Jiang updated KAFKA-15375: -- Description: Consider following sequence when running Kafka in KRaft mode: # A partition log

[jira] [Created] (KAFKA-15375) When running in KRaft mode, LogManager may creates CleanShutdown file by mistake

2023-08-17 Thread Vincent Jiang (Jira)
Vincent Jiang created KAFKA-15375: - Summary: When running in KRaft mode, LogManager may creates CleanShutdown file by mistake Key: KAFKA-15375 URL: https://issues.apache.org/jira/browse/KAFKA-15375 P

[GitHub] [kafka] junrao commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-17 Thread via GitHub
junrao commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1297576757 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -2173,6 +2230,14 @@ object UnifiedLog extends Logging { } } + private[log] def localRetentionMs(confi

[GitHub] [kafka] C0urante merged pull request #14234: MINOR: Allow writing tombstone offsets for arbitrary partitions in the FileStreamSourceConnector

2023-08-17 Thread via GitHub
C0urante merged PR #14234: URL: https://github.com/apache/kafka/pull/14234 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Updated] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-08-17 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-15102: -- Fix Version/s: 3.3.3 (was: 3.3) > Mirror Maker 2 - KIP690 backward comp

[GitHub] [kafka] C0urante merged pull request #14220: KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2

2023-08-17 Thread via GitHub
C0urante merged PR #14220: URL: https://github.com/apache/kafka/pull/14220 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Commented] (KAFKA-15227) Use plugin.discovery=SERVICE_LOAD in all plugin test suites

2023-08-17 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755658#comment-17755658 ] Greg Harris commented on KAFKA-15227: - We can delay this for some time, as the perfo

[jira] [Assigned] (KAFKA-15227) Use plugin.discovery=SERVICE_LOAD in all plugin test suites

2023-08-17 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris reassigned KAFKA-15227: --- Assignee: (was: Greg Harris) > Use plugin.discovery=SERVICE_LOAD in all plugin test sui

[GitHub] [kafka] mjsax commented on pull request #14225: MINOR: Do not reuse admin client across tests

2023-08-17 Thread via GitHub
mjsax commented on PR #14225: URL: https://github.com/apache/kafka/pull/14225#issuecomment-1682721943 Thanks for the fix @lucasbru! -- Merged to `trunk`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [kafka] mjsax merged pull request #14225: MINOR: Do not reuse admin client across tests

2023-08-17 Thread via GitHub
mjsax merged PR #14225: URL: https://github.com/apache/kafka/pull/14225 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[jira] [Comment Edited] (KAFKA-14206) Upgrade zookeeper to 3.7.1 to address security vulnerabilities

2023-08-17 Thread Dave Wichers (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755598#comment-17755598 ] Dave Wichers edited comment on KAFKA-14206 at 8/17/23 5:36 PM: ---

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-17 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1297536422 ## metadata/src/main/java/org/apache/kafka/controller/ActivationRecordsGenerator.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-17 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1297536422 ## metadata/src/main/java/org/apache/kafka/controller/ActivationRecordsGenerator.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-17 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1297533375 ## metadata/src/main/java/org/apache/kafka/controller/ActivationRecordsGenerator.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-17 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1297531045 ## metadata/src/main/java/org/apache/kafka/controller/ActivationRecordsGenerator.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-17 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1297529651 ## metadata/src/main/java/org/apache/kafka/controller/ActivationRecordsGenerator.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-17 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1297529002 ## metadata/src/main/java/org/apache/kafka/controller/ActivationRecordsGenerator.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[jira] [Commented] (KAFKA-15353) Empty ISR returned from controller after AlterPartition request

2023-08-17 Thread Calvin Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755648#comment-17755648 ] Calvin Liu commented on KAFKA-15353: Opened a PR https://github.com/apache/kafka/pul

[GitHub] [kafka] mumrah opened a new pull request, #14237: KAFKA-15374 Handle case of default broker in config migration

2023-08-17 Thread via GitHub
mumrah opened a new pull request, #14237: URL: https://github.com/apache/kafka/pull/14237 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

[jira] [Commented] (KAFKA-15372) MM2 rolling restart can drop configuration changes silently

2023-08-17 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755647#comment-17755647 ] Greg Harris commented on KAFKA-15372: - [~durban] I should clarify, I believe the beh

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-17 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1297526521 ## metadata/src/main/java/org/apache/kafka/controller/ActivationRecordsGenerator.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-17 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1297525659 ## metadata/src/main/java/org/apache/kafka/controller/ActivationRecordsGenerator.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [kafka] CalvinConfluent opened a new pull request, #14236: KAFKA-15353: make sure AlterPartitionRequestBuilder.build() is idempotent

2023-08-17 Thread via GitHub
CalvinConfluent opened a new pull request, #14236: URL: https://github.com/apache/kafka/pull/14236 As described in https://issues.apache.org/jira/browse/KAFKA-15353 When the AlterPartitionRequest version is < 3 and its builder.build is called multiple times, both newIsrWithEpochs and newI

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-17 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1297521425 ## metadata/src/main/java/org/apache/kafka/controller/ActivationRecordsGenerator.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-17 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1297521425 ## metadata/src/main/java/org/apache/kafka/controller/ActivationRecordsGenerator.java: ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[jira] [Commented] (KAFKA-15372) MM2 rolling restart can drop configuration changes silently

2023-08-17 Thread Daniel Urban (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755643#comment-17755643 ] Daniel Urban commented on KAFKA-15372: -- Hi [~gharris1727], I don't have a determini

[jira] [Created] (KAFKA-15374) ZK migration fails on configs for default broker resource

2023-08-17 Thread David Arthur (Jira)
David Arthur created KAFKA-15374: Summary: ZK migration fails on configs for default broker resource Key: KAFKA-15374 URL: https://issues.apache.org/jira/browse/KAFKA-15374 Project: Kafka Iss

[jira] [Commented] (KAFKA-15372) MM2 rolling restart can drop configuration changes silently

2023-08-17 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755629#comment-17755629 ] Greg Harris commented on KAFKA-15372: - Hi [~durban] Thanks for the bug report. Is t

[GitHub] [kafka] hachikuji commented on a diff in pull request #14213: KAFKA-15345; KRaft leader notifies when listener reaches epoch start

2023-08-17 Thread via GitHub
hachikuji commented on code in PR #14213: URL: https://github.com/apache/kafka/pull/14213#discussion_r1297486408 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java: ## @@ -2656,6 +2656,12 @@ private void maybeFireLeaderChange(LeaderAndEpoch leaderAndEpoch, long ep

[GitHub] [kafka] OmniaGM commented on a diff in pull request #13201: KAFKA-14596: Move TopicCommand to tools

2023-08-17 Thread via GitHub
OmniaGM commented on code in PR #13201: URL: https://github.com/apache/kafka/pull/13201#discussion_r1297480256 ## tools/src/main/java/org/apache/kafka/tools/TopicCommand.java: ## @@ -0,0 +1,1019 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * co

[GitHub] [kafka] jsancio commented on a diff in pull request #14213: KAFKA-15345; KRaft leader notifies when listener reaches epoch start

2023-08-17 Thread via GitHub
jsancio commented on code in PR #14213: URL: https://github.com/apache/kafka/pull/14213#discussion_r1297478419 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java: ## @@ -2653,11 +2652,11 @@ private boolean shouldFireLeaderChange(LeaderAndEpoch leaderAndEpoch) {

[GitHub] [kafka] jsancio commented on a diff in pull request #14213: KAFKA-15345; KRaft leader notifies when listener reaches epoch start

2023-08-17 Thread via GitHub
jsancio commented on code in PR #14213: URL: https://github.com/apache/kafka/pull/14213#discussion_r1297478095 ## raft/src/main/java/org/apache/kafka/raft/internals/RecordsBatchReader.java: ## @@ -117,14 +117,8 @@ private void ensureOpen() { } private Optional> nextB

[GitHub] [kafka] OmniaGM commented on a diff in pull request #13201: KAFKA-14596: Move TopicCommand to tools

2023-08-17 Thread via GitHub
OmniaGM commented on code in PR #13201: URL: https://github.com/apache/kafka/pull/13201#discussion_r1297477323 ## tools/src/main/java/org/apache/kafka/tools/TopicCommand.java: ## @@ -0,0 +1,1019 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * co

[GitHub] [kafka] OmniaGM commented on a diff in pull request #13201: KAFKA-14596: Move TopicCommand to tools

2023-08-17 Thread via GitHub
OmniaGM commented on code in PR #13201: URL: https://github.com/apache/kafka/pull/13201#discussion_r1297476982 ## tools/src/main/java/org/apache/kafka/tools/TopicCommand.java: ## @@ -0,0 +1,1019 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * co

[GitHub] [kafka] OmniaGM commented on a diff in pull request #13201: KAFKA-14596: Move TopicCommand to tools

2023-08-17 Thread via GitHub
OmniaGM commented on code in PR #13201: URL: https://github.com/apache/kafka/pull/13201#discussion_r1297476446 ## tools/src/main/java/org/apache/kafka/tools/TopicCommand.java: ## @@ -0,0 +1,1019 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * co

[GitHub] [kafka] OmniaGM commented on a diff in pull request #13201: KAFKA-14596: Move TopicCommand to tools

2023-08-17 Thread via GitHub
OmniaGM commented on code in PR #13201: URL: https://github.com/apache/kafka/pull/13201#discussion_r1297476207 ## tools/src/main/java/org/apache/kafka/tools/TopicCommand.java: ## @@ -0,0 +1,1019 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * co

[GitHub] [kafka] OmniaGM commented on a diff in pull request #13201: KAFKA-14596: Move TopicCommand to tools

2023-08-17 Thread via GitHub
OmniaGM commented on code in PR #13201: URL: https://github.com/apache/kafka/pull/13201#discussion_r1297475863 ## tools/src/main/java/org/apache/kafka/tools/TopicCommand.java: ## @@ -0,0 +1,1019 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * co

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-17 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1297472470 ## metadata/src/test/java/org/apache/kafka/image/DelegationTokenImageTest.java: ## @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-17 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1297471447 ## metadata/src/main/java/org/apache/kafka/metadata/DelegationTokenData.java: ## Review Comment: TokenInformation is Legacy so I'm not updating a test for it.

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-17 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1297470130 ## metadata/src/main/java/org/apache/kafka/metadata/DelegationTokenData.java: ## Review Comment: I have added a DelegationTokenDataTest. -- This is an auto

[GitHub] [kafka] clolov commented on a diff in pull request #14161: KAFKA-15267: Do not allow Tiered Storage to be disabled while topics have remote.storage.enable property

2023-08-17 Thread via GitHub
clolov commented on code in PR #14161: URL: https://github.com/apache/kafka/pull/14161#discussion_r1297412799 ## core/src/main/scala/kafka/server/ConfigHandler.scala: ## @@ -62,6 +62,12 @@ class TopicConfigHandler(private val logManager: LogManager, kafkaConfig: KafkaC top

[GitHub] [kafka] C0urante commented on a diff in pull request #14230: KAFKA-15226: Add connect-plugin-path and plugin.discovery system test

2023-08-17 Thread via GitHub
C0urante commented on code in PR #14230: URL: https://github.com/apache/kafka/pull/14230#discussion_r1297376126 ## tests/kafkatest/directory_layout/kafka_path.py: ## @@ -41,14 +41,20 @@ CORE_DEPENDANT_TEST_LIBS_JAR_NAME = "core-dependant-testlibs" TOOLS_JAR_NAME = "tools" TOO

[jira] [Assigned] (KAFKA-15353) Empty ISR returned from controller after AlterPartition request

2023-08-17 Thread Calvin Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Calvin Liu reassigned KAFKA-15353: -- Assignee: Calvin Liu > Empty ISR returned from controller after AlterPartition request >

[jira] [Commented] (KAFKA-15353) Empty ISR returned from controller after AlterPartition request

2023-08-17 Thread Calvin Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755614#comment-17755614 ] Calvin Liu commented on KAFKA-15353: Hey [~showuon], thanks for pointing it out! Wil

[GitHub] [kafka] clolov commented on a diff in pull request #14161: KAFKA-15267: Do not allow Tiered Storage to be disabled while topics have remote.storage.enable property

2023-08-17 Thread via GitHub
clolov commented on code in PR #14161: URL: https://github.com/apache/kafka/pull/14161#discussion_r1297412799 ## core/src/main/scala/kafka/server/ConfigHandler.scala: ## @@ -62,6 +62,12 @@ class TopicConfigHandler(private val logManager: LogManager, kafkaConfig: KafkaC top

[jira] [Comment Edited] (KAFKA-14206) Upgrade zookeeper to 3.7.1 to address security vulnerabilities

2023-08-17 Thread Dave Wichers (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755598#comment-17755598 ] Dave Wichers edited comment on KAFKA-14206 at 8/17/23 3:46 PM: ---

[GitHub] [kafka] clolov commented on a diff in pull request #14161: KAFKA-15267: Do not allow Tiered Storage to be disabled while topics have remote.storage.enable property

2023-08-17 Thread via GitHub
clolov commented on code in PR #14161: URL: https://github.com/apache/kafka/pull/14161#discussion_r1297412799 ## core/src/main/scala/kafka/server/ConfigHandler.scala: ## @@ -62,6 +62,12 @@ class TopicConfigHandler(private val logManager: LogManager, kafkaConfig: KafkaC top

[jira] [Created] (KAFKA-15373) AdminClient#describeTopics should not throw InvalidTopicException if topic ID is not found

2023-08-17 Thread Michael Edgar (Jira)
Michael Edgar created KAFKA-15373: - Summary: AdminClient#describeTopics should not throw InvalidTopicException if topic ID is not found Key: KAFKA-15373 URL: https://issues.apache.org/jira/browse/KAFKA-15373

[jira] [Created] (KAFKA-15372) MM2 rolling restart can drop configuration changes silently

2023-08-17 Thread Daniel Urban (Jira)
Daniel Urban created KAFKA-15372: Summary: MM2 rolling restart can drop configuration changes silently Key: KAFKA-15372 URL: https://issues.apache.org/jira/browse/KAFKA-15372 Project: Kafka

[GitHub] [kafka] rondagostino commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-17 Thread via GitHub
rondagostino commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1297350659 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -215,9 +215,9 @@ class KafkaApis(val requestChannel: RequestChannel, case ApiKeys.DESCRIBE_LOG_

[GitHub] [kafka] C0urante commented on a diff in pull request #14220: KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2

2023-08-17 Thread via GitHub
C0urante commented on code in PR #14220: URL: https://github.com/apache/kafka/pull/14220#discussion_r1297349746 ## docs/upgrade.html: ## @@ -43,6 +43,10 @@ Notable changes in 3 See https://cwiki.apache.org/confluence/display/KAFKA/KIP-925%3A+Rack+aware+task+assignme

[GitHub] [kafka] C0urante commented on pull request #14220: KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2

2023-08-17 Thread via GitHub
C0urante commented on PR #14220: URL: https://github.com/apache/kafka/pull/14220#issuecomment-1682432780 > I was thinking that in addition to this upgrade note, we would also need to backport the configuration documentation to keep the links valid, and because the documentation would be inc

[GitHub] [kafka] C0urante opened a new pull request, #14235: KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2

2023-08-17 Thread via GitHub
C0urante opened a new pull request, #14235: URL: https://github.com/apache/kafka/pull/14235 Backports the changes from https://github.com/apache/kafka/pull/14220 onto 3.5. Since we do not have autogenerated configuration docs for MM2 on 3.5 and older branches, information on the property (a

[GitHub] [kafka] lucasbru commented on pull request #14225: MINOR: Do not reuse admin client across tests

2023-08-17 Thread via GitHub
lucasbru commented on PR #14225: URL: https://github.com/apache/kafka/pull/14225#issuecomment-1682378421 Test failures are unrelated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [kafka] C0urante closed pull request #14233: MINOR: Permit tombstone offsets with any partition for file source connector

2023-08-17 Thread via GitHub
C0urante closed pull request #14233: MINOR: Permit tombstone offsets with any partition for file source connector URL: https://github.com/apache/kafka/pull/14233 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [kafka] C0urante commented on pull request #14233: MINOR: Permit tombstone offsets with any partition for file source connector

2023-08-17 Thread via GitHub
C0urante commented on PR #14233: URL: https://github.com/apache/kafka/pull/14233#issuecomment-1682364193 Closing in favor of https://github.com/apache/kafka/pull/14234 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [kafka] yashmayya commented on pull request #14234: MINOR: Allow writing tombstone offsets for arbitrary partitions in the FileStreamSourceConnector

2023-08-17 Thread via GitHub
yashmayya commented on PR #14234: URL: https://github.com/apache/kafka/pull/14234#issuecomment-1682363708 Ah whoops, I missed that, my bad! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [kafka] C0urante commented on pull request #14234: MINOR: Allow writing tombstone offsets for arbitrary partitions in the FileStreamSourceConnector

2023-08-17 Thread via GitHub
C0urante commented on PR #14234: URL: https://github.com/apache/kafka/pull/14234#issuecomment-1682361707 @yashmayya Beat you to it :) https://github.com/apache/kafka/pull/14233 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [kafka] yashmayya commented on pull request #14234: MINOR: Allow writing tombstone offsets for arbitrary partitions in the FileStreamSourceConnector

2023-08-17 Thread via GitHub
yashmayya commented on PR #14234: URL: https://github.com/apache/kafka/pull/14234#issuecomment-1682360405 @C0urante now that https://github.com/apache/kafka/pull/14005 has been merged, could you please take a look at this minor change to `FileStreamSourceConnector::alterOffsets` (based on t

[GitHub] [kafka] yashmayya opened a new pull request, #14234: MINOR: Allow writing tombstone offsets for arbitrary partitions in the FileStreamSourceConnector

2023-08-17 Thread via GitHub
yashmayya opened a new pull request, #14234: URL: https://github.com/apache/kafka/pull/14234 - Background - https://cwiki.apache.org/confluence/display/KAFKA/KIP-875%3A+First-class+offsets+support+in+Kafka+Connect, https://github.com/apache/kafka/pull/13945#discussion_r1266867639, https://

[GitHub] [kafka] C0urante opened a new pull request, #14233: MINOR: Permit tombstone offsets with any partition for file source connector

2023-08-17 Thread via GitHub
C0urante opened a new pull request, #14233: URL: https://github.com/apache/kafka/pull/14233 This aligns the behavior of the file and MM2 source connectors with regards to tombstone offsets submitted via the REST API. Partition validation is intentionally disabled for tombstone offsets to al

[GitHub] [kafka] C0urante merged pull request #14005: KAFKA-15177: Implement KIP-875 SourceConnector::alterOffset API in MirrorMaker 2 connectors

2023-08-17 Thread via GitHub
C0urante merged PR #14005: URL: https://github.com/apache/kafka/pull/14005 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] OmniaGM commented on pull request #13204: KAFKA-14593: Move LeaderElectionCommand to tools

2023-08-17 Thread via GitHub
OmniaGM commented on PR #13204: URL: https://github.com/apache/kafka/pull/13204#issuecomment-1682276851 The test is failing because of an unrelated package `:storage:test` with JDK 11 and Scala 2.13. -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (KAFKA-8128) Dynamic JAAS change for clients

2023-08-17 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755543#comment-17755543 ] Gabor Somogyi commented on KAFKA-8128: -- Sounds great. > Dynamic JAAS change for cli

[GitHub] [kafka] riedelmax commented on pull request #14124: Kafka-14509; [1/2] Define ConsumerGroupDescribe API request and response schemas and classes.

2023-08-17 Thread via GitHub
riedelmax commented on PR #14124: URL: https://github.com/apache/kafka/pull/14124#issuecomment-1682236535 Right now the method does almost nothing.. I could strip it down to this version, which is enough to make tests pass: ``` def handleConsumerGroupDescribe(request: RequestChannel.R

[GitHub] [kafka] clolov commented on pull request #14116: KAFKA-15167: Tiered Storage Test Harness Framework

2023-08-17 Thread via GitHub
clolov commented on PR #14116: URL: https://github.com/apache/kafka/pull/14116#issuecomment-1682210532 Huh, yeah, no idea. I have been trying to replicate it locally and it appears to be doing exactly what you would expect it to. -- This is an automated message from the Apache Git Service

[GitHub] [kafka] clolov commented on pull request #14116: KAFKA-15167: Tiered Storage Test Harness Framework

2023-08-17 Thread via GitHub
clolov commented on PR #14116: URL: https://github.com/apache/kafka/pull/14116#issuecomment-1682078236 Heya @kamalcph! I have seen this error pop up when server.shutdown hasn't been called in integration tests, I would suggest looking into whether for whatever reason there is something whic

[GitHub] [kafka] kamalcph commented on pull request #14116: KAFKA-15167: Tiered Storage Test Harness Framework

2023-08-17 Thread via GitHub
kamalcph commented on PR #14116: URL: https://github.com/apache/kafka/pull/14116#issuecomment-1682062995 How to fix the below error in the test? The test is currently marked as DISABLED but still fails with below error, there are some left-over threads: Build / JDK 11 and Scala 2.13 /

[jira] [Commented] (KAFKA-14589) Move ConsumerGroupCommand to tools

2023-08-17 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755426#comment-17755426 ] Omnia Ibrahim commented on KAFKA-14589: --- [~nizhikov] sure > Move ConsumerGroupCom

[jira] [Assigned] (KAFKA-14588) Move ConfigCommand to tools

2023-08-17 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Omnia Ibrahim reassigned KAFKA-14588: - Assignee: Nikolay Izhikov (was: Omnia Ibrahim) > Move ConfigCommand to tools > ---

[jira] [Commented] (KAFKA-14588) Move ConfigCommand to tools

2023-08-17 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755425#comment-17755425 ] Omnia Ibrahim commented on KAFKA-14588: --- [~nizhikov] sure go ahead as am focusing

  1   2   >