Re: [PR] Replace lambda expressions with method references for ReplicationControlManager [kafka]

2024-07-11 Thread via GitHub
gongxuanzhang commented on PR #16547: URL: https://github.com/apache/kafka/pull/16547#issuecomment-182852 LGTM, @chia7712 Can we batch modify this refacible code? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] KAFKA-16853: Split RemoteLogManagerScheduledThreadPool [kafka]

2024-07-11 Thread via GitHub
abhijeetk88 commented on code in PR #16502: URL: https://github.com/apache/kafka/pull/16502#discussion_r1673509846 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -686,60 +702,98 @@ List getLeaderEpochEntries(UnifiedLog log, long startOffset, long en }

Re: [PR] KAFKA-16853: Split RemoteLogManagerScheduledThreadPool [kafka]

2024-07-11 Thread via GitHub
abhijeetk88 commented on PR #16502: URL: https://github.com/apache/kafka/pull/16502#issuecomment-189362 @showuon I have responded to your comments. Please take another look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] KAFKA-16853: Split RemoteLogManagerScheduledThreadPool [kafka]

2024-07-11 Thread via GitHub
abhijeetk88 commented on code in PR #16502: URL: https://github.com/apache/kafka/pull/16502#discussion_r1673511532 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -241,7 +245,12 @@ public RemoteLogManager(RemoteLogManagerConfig rlmConfig, indexCach

Re: [PR] KAFKA-16853: Split RemoteLogManagerScheduledThreadPool [kafka]

2024-07-11 Thread via GitHub
showuon commented on code in PR #16502: URL: https://github.com/apache/kafka/pull/16502#discussion_r1673519734 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -686,60 +702,98 @@ List getLeaderEpochEntries(UnifiedLog log, long startOffset, long en }

Re: [PR] KAFKA-16853: Split RemoteLogManagerScheduledThreadPool [kafka]

2024-07-11 Thread via GitHub
showuon commented on code in PR #16502: URL: https://github.com/apache/kafka/pull/16502#discussion_r1673520175 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -241,7 +245,12 @@ public RemoteLogManager(RemoteLogManagerConfig rlmConfig, indexCache =

[jira] [Commented] (KAFKA-9738) Add Generics Type Parameters to forwarded() in MockProcessorContext

2024-07-11 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17864948#comment-17864948 ] Bruno Cadonna commented on KAFKA-9738: -- The old `MockProcessorContext` is still not

Re: [PR] KAFKA-17085: Handle tasks in state updater before tasks in task registry [kafka]

2024-07-11 Thread via GitHub
cadonna merged PR #16561: URL: https://github.com/apache/kafka/pull/16561 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Updated] (KAFKA-17111) ServiceConfigurationError in JsonSerializer/Deserializer during Plugin Discovery

2024-07-11 Thread Josep Prat (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josep Prat updated KAFKA-17111: --- Fix Version/s: 3.8.0 > ServiceConfigurationError in JsonSerializer/Deserializer during Plugin > Dis

Re: [PR] [ KAFKA-17049 ] fix Incremental rebalances assign too many tasks for the same connector together [kafka]

2024-07-11 Thread via GitHub
yazgoo commented on code in PR #16486: URL: https://github.com/apache/kafka/pull/16486#discussion_r1673685382 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java: ## @@ -793,6 +796,46 @@ protected void assignConnector

Re: [PR] [ KAFKA-17049 ] fix Incremental rebalances assign too many tasks for the same connector together [kafka]

2024-07-11 Thread via GitHub
yazgoo commented on code in PR #16486: URL: https://github.com/apache/kafka/pull/16486#discussion_r1673685382 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java: ## @@ -793,6 +796,46 @@ protected void assignConnector

[jira] [Commented] (KAFKA-17094) Make it possible to list registered KRaft nodes in order to know which nodes should be unregistered

2024-07-11 Thread Igor Soarez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17864980#comment-17864980 ] Igor Soarez commented on KAFKA-17094: - I agree it makes sense to make this informati

[jira] [Updated] (KAFKA-17098) Error Opening RocksDBStore

2024-07-11 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-17098: -- Priority: Critical (was: Minor) > Error Opening RocksDBStore > -- > >

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2024-07-11 Thread via GitHub
soarez commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1673749083 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -60,7 +63,11 @@ object StorageTool extends Logging { if (!metadataVersion.isKRaftSupported) {

Re: [PR] MINOR: Add logs when metadata update is not successful [kafka]

2024-07-11 Thread via GitHub
subhashiyer9 commented on code in PR #16496: URL: https://github.com/apache/kafka/pull/16496#discussion_r1673749240 ## clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java: ## @@ -1201,13 +1201,26 @@ private ClusterAndWaitTime waitOnMetadata(String topic,

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2024-07-11 Thread via GitHub
soarez commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1673749083 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -60,7 +63,11 @@ object StorageTool extends Logging { if (!metadataVersion.isKRaftSupported) {

[PR] KAFKA-17098: Re-add task to state updater if transit to RUNNING fails [kafka]

2024-07-11 Thread via GitHub
cadonna opened a new pull request, #16570: URL: https://github.com/apache/kafka/pull/16570 When Streams tries to transit a restored active task to RUNNING, the first thing it does is getting the committed offsets for this task. If getting the offsets expires a timeout, Streams does not re-t

Re: [PR] KAFKA-16853: Split RemoteLogManagerScheduledThreadPool [kafka]

2024-07-11 Thread via GitHub
kamalcph commented on code in PR #16502: URL: https://github.com/apache/kafka/pull/16502#discussion_r1673686505 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -241,12 +245,17 @@ public RemoteLogManager(RemoteLogManagerConfig rlmConfig, indexCache

[jira] [Assigned] (KAFKA-17098) Error Opening RocksDBStore

2024-07-11 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna reassigned KAFKA-17098: - Assignee: Bruno Cadonna > Error Opening RocksDBStore > -- > >

[jira] [Commented] (KAFKA-17098) Error Opening RocksDBStore

2024-07-11 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17864987#comment-17864987 ] Bruno Cadonna commented on KAFKA-17098: --- [~eduwerc] I think, I found the issue. Se

[jira] [Created] (KAFKA-17120) Race condition in KafkaStreams.close can result in StreamsException "Failed to shut down while in state"

2024-07-11 Thread Andy Coates (Jira)
Andy Coates created KAFKA-17120: --- Summary: Race condition in KafkaStreams.close can result in StreamsException "Failed to shut down while in state" Key: KAFKA-17120 URL: https://issues.apache.org/jira/browse/KAFKA-1

Re: [PR] KAFKA-17111: Explicitly Register Afterburner Module in JsonSerializer and JsonDeserializer [kafka]

2024-07-11 Thread via GitHub
jlprat commented on PR #16565: URL: https://github.com/apache/kafka/pull/16565#issuecomment-582924 Test failures are unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] KAFKA-17111: Explicitly Register Afterburner Module in JsonSerializer and JsonDeserializer [kafka]

2024-07-11 Thread via GitHub
jlprat commented on PR #16565: URL: https://github.com/apache/kafka/pull/16565#issuecomment-585080 @mimaison or @divijvaidya As you reviewed https://github.com/apache/kafka/pull/14992 do you want to take a look at this one as well? -- This is an automated message from the Apache Git

Re: [PR] KAFKA-16791: Add thread detection to ClusterTestExtensions [kafka]

2024-07-11 Thread via GitHub
FrankYang0529 commented on PR #16499: URL: https://github.com/apache/kafka/pull/16499#issuecomment-597872 > @FrankYang0529 please take a look at build error Fixed it. The latest CI doesn't have build error. Could you take a look? Thank you. -- This is an automated message from

[jira] [Created] (KAFKA-17121) junit-platform.properties files in published artifacts pollute the test classpath of consumers

2024-07-11 Thread Andy Wilkinson (Jira)
Andy Wilkinson created KAFKA-17121: -- Summary: junit-platform.properties files in published artifacts pollute the test classpath of consumers Key: KAFKA-17121 URL: https://issues.apache.org/jira/browse/KAFKA-17121

[jira] [Updated] (KAFKA-17079) scoverage plugin not found in maven repo, version 1.9.3

2024-07-11 Thread kaushik srinivas (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kaushik srinivas updated KAFKA-17079: - Priority: Major (was: Blocker) > scoverage plugin not found in maven repo, version 1.9.

Re: [PR] MINOR:Topic command integration test migrate to new test infra [kafka]

2024-07-11 Thread via GitHub
TaiJuWu commented on code in PR #16127: URL: https://github.com/apache/kafka/pull/16127#discussion_r1673847880 ## clients/src/test/java/org/apache/kafka/test/TestUtils.java: ## @@ -706,4 +706,9 @@ public static ApiVersionsResponse createApiVersionsResponse( ApiV

Re: [PR] MINOR:Topic command integration test migrate to new test infra [kafka]

2024-07-11 Thread via GitHub
TaiJuWu commented on code in PR #16127: URL: https://github.com/apache/kafka/pull/16127#discussion_r1673848590 ## core/src/test/java/kafka/testkit/KafkaClusterTestKit.java: ## @@ -191,7 +191,7 @@ private KafkaConfig createNodeConfig(TestKitNode node) { contr

Re: [PR] MINOR:Topic command integration test migrate to new test infra [kafka]

2024-07-11 Thread via GitHub
TaiJuWu commented on code in PR #16127: URL: https://github.com/apache/kafka/pull/16127#discussion_r1673847880 ## clients/src/test/java/org/apache/kafka/test/TestUtils.java: ## @@ -706,4 +706,9 @@ public static ApiVersionsResponse createApiVersionsResponse( ApiV

Re: [PR] MINOR:Topic command integration test migrate to new test infra [kafka]

2024-07-11 Thread via GitHub
TaiJuWu commented on code in PR #16127: URL: https://github.com/apache/kafka/pull/16127#discussion_r1673847880 ## clients/src/test/java/org/apache/kafka/test/TestUtils.java: ## @@ -706,4 +706,9 @@ public static ApiVersionsResponse createApiVersionsResponse( ApiV

[jira] [Commented] (KAFKA-6939) Change the default of log.message.timestamp.difference.max.ms to 500 years

2024-07-11 Thread Ksolves India Limited (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17865033#comment-17865033 ] Ksolves India Limited commented on KAFKA-6939: -- The configuration [log.mess

Re: [PR] KAFKA-16730: Initial version of share group consumer client code [kafka]

2024-07-11 Thread via GitHub
omkreddy commented on code in PR #16461: URL: https://github.com/apache/kafka/pull/16461#discussion_r1673850589 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Acknowledgements.java: ## @@ -0,0 +1,323 @@ +/* + * Licensed to the Apache Software Foundation (AS

Re: [PR] KAFKA-16057 : Change the configuration for adminclient connections.max.idle.ms to 9 minutes [kafka]

2024-07-11 Thread via GitHub
Nancy-ksolves commented on PR #16548: URL: https://github.com/apache/kafka/pull/16548#issuecomment-760243 Hi @mimaison, to proceed with changing the default value of a public configuration, we require access to the Kafka Improvement Proposal (KIP). Could you please provide details on ho

Re: [PR] KAFKA-16730: Initial version of share group consumer client code [kafka]

2024-07-11 Thread via GitHub
AndrewJSchofield commented on code in PR #16461: URL: https://github.com/apache/kafka/pull/16461#discussion_r1673903246 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ShareConsumeRequestManager.java: ## @@ -0,0 +1,888 @@ +/* + * Licensed to the Apache Softw

Re: [PR] KAFKA-17111: Explicitly Register Afterburner Module in JsonSerializer and JsonDeserializer [kafka]

2024-07-11 Thread via GitHub
jlprat merged PR #16565: URL: https://github.com/apache/kafka/pull/16565 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.o

Re: [PR] KAFKA-17098: Re-add task to state updater if transit to RUNNING fails [kafka]

2024-07-11 Thread via GitHub
cadonna commented on PR #16570: URL: https://github.com/apache/kafka/pull/16570#issuecomment-789726 Streams system tests have zero failures on this PR: http://confluent-open-source-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/trunk/2024-07-11--001.f881b68f-ae76-4fa

Re: [PR] KAFKA-16057 : Change the configuration for adminclient connections.max.idle.ms to 9 minutes [kafka]

2024-07-11 Thread via GitHub
mimaison commented on PR #16548: URL: https://github.com/apache/kafka/pull/16548#issuecomment-834086 If you give me you Confluence userid, I can grant you permissions to create a KIP. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [PR] KAFKA-17111: Explicitly Register Afterburner Module in JsonSerializer and JsonDeserializer [kafka]

2024-07-11 Thread via GitHub
jlprat commented on PR #16565: URL: https://github.com/apache/kafka/pull/16565#issuecomment-844388 Cherry picked as well to `3.8`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Commented] (KAFKA-17112) StreamThread shutdown calls completeShutdown only in CREATED state

2024-07-11 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17865048#comment-17865048 ] Bruno Cadonna commented on KAFKA-17112: --- [~aoli-al] I think you are right. We are

[jira] [Commented] (KAFKA-17094) Make it possible to list registered KRaft nodes in order to know which nodes should be unregistered

2024-07-11 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17865049#comment-17865049 ] Mickael Maison commented on KAFKA-17094: [~soarez] It seems this is a topic that

Re: [PR] MINOR: Add logs when metadata update is not successful [kafka]

2024-07-11 Thread via GitHub
subhashiyer9 commented on code in PR #16496: URL: https://github.com/apache/kafka/pull/16496#discussion_r1673749240 ## clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java: ## @@ -1201,13 +1201,26 @@ private ClusterAndWaitTime waitOnMetadata(String topic,

Re: [PR] KAFKA-16959: ConfigCommand should allow to define both `entity-default` and `entity-name` [kafka]

2024-07-11 Thread via GitHub
m1a2st commented on PR #16381: URL: https://github.com/apache/kafka/pull/16381#issuecomment-873610 @chia7712, add new test for `InvalidConfigurationException`, PTAL, Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] KAFKA-17102: FetchRequest#forgottenTopics would return incorrect data [kafka]

2024-07-11 Thread via GitHub
m1a2st commented on PR #16557: URL: https://github.com/apache/kafka/pull/16557#issuecomment-96 @chia7712, Thanks for your comments, PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[jira] [Updated] (KAFKA-17121) junit-platform.properties files in published artifacts pollute the test classpath of consumers

2024-07-11 Thread Andy Wilkinson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Wilkinson updated KAFKA-17121: --- Description: https://github.com/apache/kafka/commit/6e998cffdd33e343945877ccee1fec8337c7d57d

[jira] [Updated] (KAFKA-17121) junit-platform.properties files in published artifacts pollute the test classpath of consumers

2024-07-11 Thread Andy Wilkinson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Wilkinson updated KAFKA-17121: --- Description: https://github.com/apache/kafka/commit/6e998cffdd33e343945877ccee1fec8337c7d57d

[PR] Add replace.null.with.default configuration to ValueToKey and ReplaceField (KIP-1040) [kafka]

2024-07-11 Thread via GitHub
FrankYang0529 opened a new pull request, #16571: URL: https://github.com/apache/kafka/pull/16571 [KIP-1040](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=303794677) mentioned adding new configuration to InsertField, ExtractField, HeaderFrom, Cast, SetSchemaMetadata, Time

[jira] [Commented] (KAFKA-17112) StreamThread shutdown calls completeShutdown only in CREATED state

2024-07-11 Thread Ao Li (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17865080#comment-17865080 ] Ao Li commented on KAFKA-17112: --- Yes, I'm happy to submit a patch. I'm thinking of two pot

Re: [PR] KAFKA-17017: AsyncKafkaConsumer#unsubscribe does not clean the assigned partitions [kafka]

2024-07-11 Thread via GitHub
lianetm commented on code in PR #16449: URL: https://github.com/apache/kafka/pull/16449#discussion_r1674077031 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1501,11 +1501,11 @@ public void unsubscribe() { private void

[jira] [Updated] (KAFKA-17115) Closing newly-created legacy consumers during rebalance can cause rebalances to hang

2024-07-11 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-17115: -- Component/s: clients > Closing newly-created legacy consumers during rebalance can cause rebalances >

Re: [PR] KAFKA-17017: AsyncKafkaConsumer#unsubscribe does not clean the assigned partitions [kafka]

2024-07-11 Thread via GitHub
lianetm commented on PR #16449: URL: https://github.com/apache/kafka/pull/16449#issuecomment-2223050194 Hey @FrankYang0529 , thanks for the updates. Only one nit left. Also I notice the build did not complete, please merge trunk latest changes too and let's see what we get on the next run.

[jira] [Updated] (KAFKA-17116) New consumer may not send effective leave group if member ID received after close

2024-07-11 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-17116: -- Component/s: clients > New consumer may not send effective leave group if member ID received after >

Re: [PR] KAFKA-16584 Make log processing summary configurable or debug [kafka]

2024-07-11 Thread via GitHub
dujian0068 commented on code in PR #16509: URL: https://github.com/apache/kafka/pull/16509#discussion_r1674098676 ## streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java: ## @@ -834,6 +834,10 @@ public class StreamsConfig extends AbstractConfig { private static

Re: [PR] KAFKA-16584 Make log processing summary configurable or debug [kafka]

2024-07-11 Thread via GitHub
dujian0068 commented on PR #16509: URL: https://github.com/apache/kafka/pull/16509#issuecomment-2223084635 Thank you @mjsax I have updated config-stream doc. Do I need to update upgrade-guide? Can you tell me in which cases I need to update upgrade-guide? -- This is an automated messa

Re: [PR] KAFKA-17098: Re-add task to state updater if transit to RUNNING fails [kafka]

2024-07-11 Thread via GitHub
jlprat commented on PR #16570: URL: https://github.com/apache/kafka/pull/16570#issuecomment-2223114080 Test failures seem unrelated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] KAFKA-15999 Migrate HeartbeatRequestManagerTest away from ConsumerTestBuilder [kafka]

2024-07-11 Thread via GitHub
lianetm commented on code in PR #16200: URL: https://github.com/apache/kafka/pull/16200#discussion_r1674138238 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -201,8 +202,42 @@ public void testSuccessfulHeartbeatTiming

Re: [PR] KAFKA-15999 Migrate HeartbeatRequestManagerTest away from ConsumerTestBuilder [kafka]

2024-07-11 Thread via GitHub
lianetm commented on code in PR #16200: URL: https://github.com/apache/kafka/pull/16200#discussion_r1674142299 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -201,8 +202,42 @@ public void testSuccessfulHeartbeatTiming

Re: [PR] KAFKA-17017: AsyncKafkaConsumer#unsubscribe does not clean the assigned partitions [kafka]

2024-07-11 Thread via GitHub
FrankYang0529 commented on code in PR #16449: URL: https://github.com/apache/kafka/pull/16449#discussion_r1674145282 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1501,11 +1501,11 @@ public void unsubscribe() { privat

[jira] [Updated] (KAFKA-17061) KafkaController takes long time to connect to newly added broker after registration on large cluster

2024-07-11 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haruki Okada updated KAFKA-17061: - Description: h2. Environment * Kafka version: 3.3.2 * Cluster: 200~ brokers * Total num parti

[jira] [Updated] (KAFKA-17061) KafkaController takes long time to connect to newly added broker after registration on large cluster

2024-07-11 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haruki Okada updated KAFKA-17061: - Attachment: flame.html > KafkaController takes long time to connect to newly added broker after

[jira] [Updated] (KAFKA-17061) KafkaController takes long time to connect to newly added broker after registration on large cluster

2024-07-11 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haruki Okada updated KAFKA-17061: - Attachment: flame-patched.html > KafkaController takes long time to connect to newly added broke

Re: [PR] KAFKA-15999 Migrate HeartbeatRequestManagerTest away from ConsumerTestBuilder [kafka]

2024-07-11 Thread via GitHub
lianetm commented on code in PR #16200: URL: https://github.com/apache/kafka/pull/16200#discussion_r1674150218 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -251,23 +304,25 @@ public void testSkippingHeartbeat(final

[jira] [Updated] (KAFKA-17061) KafkaController takes long time to connect to newly added broker after registration on large cluster

2024-07-11 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haruki Okada updated KAFKA-17061: - Attachment: screenshot-flame.png > KafkaController takes long time to connect to newly added bro

[jira] [Updated] (KAFKA-17061) KafkaController takes long time to connect to newly added broker after registration on large cluster

2024-07-11 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haruki Okada updated KAFKA-17061: - Attachment: screenshot-flame-patched.png > KafkaController takes long time to connect to newly a

[jira] [Updated] (KAFKA-17061) KafkaController takes long time to connect to newly added broker after registration on large cluster

2024-07-11 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haruki Okada updated KAFKA-17061: - Description: h2. Environment * Kafka version: 3.3.2 * Cluster: 200~ brokers * Total num parti

[jira] [Commented] (KAFKA-17061) KafkaController takes long time to connect to newly added broker after registration on large cluster

2024-07-11 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17865110#comment-17865110 ] Haruki Okada commented on KAFKA-17061: -- Did a micro benchmark to check the performa

Re: [PR] KAFKA-17061 Improve the performance of isReplicaOnline [kafka]

2024-07-11 Thread via GitHub
ocadaruma commented on PR #16529: URL: https://github.com/apache/kafka/pull/16529#issuecomment-2223158388 @chia7712 Added a flamegraph. PTAL https://issues.apache.org/jira/browse/KAFKA-17061?focusedCommentId=17865110&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#

Re: [PR] KAFKA-15999 Migrate HeartbeatRequestManagerTest away from ConsumerTestBuilder [kafka]

2024-07-11 Thread via GitHub
lianetm commented on code in PR #16200: URL: https://github.com/apache/kafka/pull/16200#discussion_r1674165119 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -756,18 +801,19 @@ public void testFencedMemberStopHeartbe

Re: [PR] KAFKA-15999 Migrate HeartbeatRequestManagerTest away from ConsumerTestBuilder [kafka]

2024-07-11 Thread via GitHub
lianetm commented on code in PR #16200: URL: https://github.com/apache/kafka/pull/16200#discussion_r1674174436 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -659,95 +749,50 @@ public void testPollTimerExpirationShou

Re: [PR] KAFKA-15999 Migrate HeartbeatRequestManagerTest away from ConsumerTestBuilder [kafka]

2024-07-11 Thread via GitHub
lianetm commented on code in PR #16200: URL: https://github.com/apache/kafka/pull/16200#discussion_r1674207651 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -201,8 +202,42 @@ public void testSuccessfulHeartbeatTiming

Re: [PR] KAFKA-15999 Migrate HeartbeatRequestManagerTest away from ConsumerTestBuilder [kafka]

2024-07-11 Thread via GitHub
lianetm commented on code in PR #16200: URL: https://github.com/apache/kafka/pull/16200#discussion_r1674210447 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -201,8 +202,42 @@ public void testSuccessfulHeartbeatTiming

Re: [PR] KAFKA-15999 Migrate HeartbeatRequestManagerTest away from ConsumerTestBuilder [kafka]

2024-07-11 Thread via GitHub
lianetm commented on code in PR #16200: URL: https://github.com/apache/kafka/pull/16200#discussion_r1674210447 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -201,8 +202,42 @@ public void testSuccessfulHeartbeatTiming

[jira] [Commented] (KAFKA-17119) After enabled kafka-ranger-plugin and baned the user for using describe in policy, but that user still can use describe.

2024-07-11 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17865131#comment-17865131 ] Greg Harris commented on KAFKA-17119: - Hi [~StarBoy1005] The Apache Kafka project do

[jira] [Resolved] (KAFKA-17119) After enabled kafka-ranger-plugin and baned the user for using describe in policy, but that user still can use describe.

2024-07-11 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-17119. - Resolution: Cannot Reproduce > After enabled kafka-ranger-plugin and baned the user for using de

[jira] [Resolved] (KAFKA-17111) ServiceConfigurationError in JsonSerializer/Deserializer during Plugin Discovery

2024-07-11 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-17111. - Resolution: Fixed > ServiceConfigurationError in JsonSerializer/Deserializer during Plugin > Di

Re: [PR] KAFKA-17098: Re-add task to state updater if transit to RUNNING fails [kafka]

2024-07-11 Thread via GitHub
jlprat merged PR #16570: URL: https://github.com/apache/kafka/pull/16570 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.o

[jira] [Updated] (KAFKA-17098) Error Opening RocksDBStore

2024-07-11 Thread Josep Prat (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josep Prat updated KAFKA-17098: --- Fix Version/s: 3.8.0 > Error Opening RocksDBStore > -- > > K

[jira] [Commented] (KAFKA-17117) Avoid instantiating classpath plugins when service loading plugins

2024-07-11 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17865134#comment-17865134 ] Greg Harris commented on KAFKA-17117: - Considering that the only plugins typically o

Re: [PR] KAFKA-17098: Re-add task to state updater if transit to RUNNING fails [kafka]

2024-07-11 Thread via GitHub
mjsax commented on PR #16570: URL: https://github.com/apache/kafka/pull/16570#issuecomment-2223275885 Who is cherry-picking this to `trunk`? @jlprat are you doing this, or should we (@cadonna) do the cherry-pick? -- This is an automated message from the Apache Git Service. To respond to t

[jira] [Comment Edited] (KAFKA-17101) Mirror maker internal topics cleanup policy changes to 'delete' from 'compact'

2024-07-11 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17864935#comment-17864935 ] Greg Harris edited comment on KAFKA-17101 at 7/11/24 3:40 PM:

Re: [PR] KAFKA-17098: Re-add task to state updater if transit to RUNNING fails [kafka]

2024-07-11 Thread via GitHub
jlprat commented on PR #16570: URL: https://github.com/apache/kafka/pull/16570#issuecomment-2223277883 I'll cherry pick this one to `trunk` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[jira] [Commented] (KAFKA-17101) Mirror maker internal topics cleanup policy changes to 'delete' from 'compact'

2024-07-11 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17865137#comment-17865137 ] Greg Harris commented on KAFKA-17101: - [~kaushik srinivas] Unfortunately I don't see

[jira] [Commented] (KAFKA-17101) Mirror maker internal topics cleanup policy changes to 'delete' from 'compact'

2024-07-11 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17865138#comment-17865138 ] Greg Harris commented on KAFKA-17101: - Also you mentioned MM1, is that a typo or are

Re: [PR] KAFKA-17098: Re-add task to state updater if transit to RUNNING fails [kafka]

2024-07-11 Thread via GitHub
jlprat commented on PR #16570: URL: https://github.com/apache/kafka/pull/16570#issuecomment-2223290582 Cherry picked to `trunk` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[PR] KAFKA-17073: Deprecate ReplicaVerificationTool in 3.9 [kafka]

2024-07-11 Thread via GitHub
dongjinleekr opened a new pull request, #16572: URL: https://github.com/apache/kafka/pull/16572 ### Summary If the user runs `ReplicaVerificationTool`, it shows a deprecation warning. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementatio

[PR] KAFKA-16741: Add ShareGroupHeartbeat API support - 2/N (KIP-932) [kafka]

2024-07-11 Thread via GitHub
apoorvmittal10 opened a new pull request, #16573: URL: https://github.com/apache/kafka/pull/16573 ShareGroupHeartbeat API support as defined in KIP-932. The heartbeat persists Group and Member information on `__consumer_offsets` topic. The PR also moves some of the ShareGroupConfigs t

Re: [PR] KAFKA-16192: Introduce transaction.version and usage of flexible records to coordinators [kafka]

2024-07-11 Thread via GitHub
junrao commented on code in PR #16183: URL: https://github.com/apache/kafka/pull/16183#discussion_r1674294082 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -217,8 +217,11 @@ public enum MetadataVersion { // Add ELR related support

[jira] [Resolved] (KAFKA-10814) improving ability of handling exception in kafka

2024-07-11 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-10814. - Resolution: Incomplete Unclear what the problem is. Closing. > improving ability of han

[jira] [Commented] (KAFKA-9738) Add Generics Type Parameters to forwarded() in MockProcessorContext

2024-07-11 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17865145#comment-17865145 ] Matthias J. Sax commented on KAFKA-9738: Given that `Processor` is deprecated alr

[jira] [Assigned] (KAFKA-9738) Add Generics Type Parameters to forwarded() in MockProcessorContext

2024-07-11 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-9738: -- Assignee: Matthias J. Sax > Add Generics Type Parameters to forwarded() in MockProcess

Re: [PR] KAFKA-15999 Migrate HeartbeatRequestManagerTest away from ConsumerTestBuilder [kafka]

2024-07-11 Thread via GitHub
lianetm commented on code in PR #16200: URL: https://github.com/apache/kafka/pull/16200#discussion_r1674309494 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -425,6 +482,7 @@ topicId, mkSortedSet(0) assertEqu

Re: [PR] KAFKA-16192: Introduce transaction.version and usage of flexible records to coordinators [kafka]

2024-07-11 Thread via GitHub
jolshan commented on code in PR #16183: URL: https://github.com/apache/kafka/pull/16183#discussion_r1674310171 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -217,8 +217,11 @@ public enum MetadataVersion { // Add ELR related suppor

Re: [PR] MINOR: Add logs when metadata update is not successful [kafka]

2024-07-11 Thread via GitHub
jolshan commented on code in PR #16496: URL: https://github.com/apache/kafka/pull/16496#discussion_r1674314203 ## clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java: ## @@ -1201,13 +1201,26 @@ private ClusterAndWaitTime waitOnMetadata(String topic, Integ

[jira] [Updated] (KAFKA-9738) Add Generics Type Parameters to forwarded() in MockProcessorContext

2024-07-11 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-9738: --- Labels: kip (was: ) > Add Generics Type Parameters to forwarded() in MockProcessorContext >

[jira] [Updated] (KAFKA-9738) Add Generics Type Parameters to forwarded() in MockProcessorContext

2024-07-11 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-9738: --- Description: The method {{forwarded()}} to capture the forwarded records in {{MockProcessorC

Re: [PR] KAFKA-15999 Migrate HeartbeatRequestManagerTest away from ConsumerTestBuilder [kafka]

2024-07-11 Thread via GitHub
lianetm commented on code in PR #16200: URL: https://github.com/apache/kafka/pull/16200#discussion_r1674318250 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -447,7 +505,8 @@ private ConsumerGroupHeartbeatRequest get

[jira] [Comment Edited] (KAFKA-16937) Consider inlineing Time#waitObject to ProducerMetadata#awaitUpdate

2024-07-11 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17865164#comment-17865164 ] Greg Harris edited comment on KAFKA-16937 at 7/11/24 5:15 PM:

[jira] [Commented] (KAFKA-16937) Consider inlineing Time#waitObject to ProducerMetadata#awaitUpdate

2024-07-11 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17865164#comment-17865164 ] Greg Harris commented on KAFKA-16937: - I thought about this some more. Right now the

Re: [PR] KAFKA-17055: Change KafkaRaftClientTest and KafkaRaftClientSnapshotTest nodes to use random positive number id [kafka]

2024-07-11 Thread via GitHub
jsancio commented on code in PR #16563: URL: https://github.com/apache/kafka/pull/16563#discussion_r1674382183 ## raft/src/test/java/org/apache/kafka/raft/KafkaRaftClientTest.java: ## @@ -281,15 +281,20 @@ public void testGrantVotesWhenShuttingDown(boolean withKip853Rpc) throws

Re: [PR] MINOR: Fix transactions_upgrade_test to run transactions during upgrade [kafka]

2024-07-11 Thread via GitHub
jolshan commented on PR #16462: URL: https://github.com/apache/kafka/pull/16462#issuecomment-2223500451 Finally got to confirming the test results 👍 All passed. will merge now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

Re: [PR] KAFKA-16772: Introduce kraft.version to support KIP-853 [kafka]

2024-07-11 Thread via GitHub
jsancio commented on code in PR #16230: URL: https://github.com/apache/kafka/pull/16230#discussion_r1674397738 ## clients/src/main/java/org/apache/kafka/common/requests/ApiVersionsResponse.java: ## @@ -313,10 +313,12 @@ private static FinalizedFeatureKeyCollection createFinaliz

  1   2   >