[GitHub] [kafka] yashmayya commented on a diff in pull request #13424: KAFKA-14783 (KIP-875): New STOPPED state for connectors

2023-04-05 Thread via GitHub
yashmayya commented on code in PR #13424: URL: https://github.com/apache/kafka/pull/13424#discussion_r1158101868 ## connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedConnectCluster.java: ## @@ -553,6 +570,54 @@ public ActiveTopicsInfo connectorTopics(S

[GitHub] [kafka] showuon commented on a diff in pull request #13456: KAFKA-14850: introduce InMemoryLeaderEpochCheckpoint

2023-04-05 Thread via GitHub
showuon commented on code in PR #13456: URL: https://github.com/apache/kafka/pull/13456#discussion_r1158149051 ## storage/src/main/java/org/apache/kafka/storage/internals/epoch/LeaderEpochFileCache.java: ## @@ -147,6 +152,16 @@ private List removeWhileMatching(Iterator

[GitHub] [kafka] showuon commented on a diff in pull request #13456: KAFKA-14850: introduce InMemoryLeaderEpochCheckpoint

2023-04-05 Thread via GitHub
showuon commented on code in PR #13456: URL: https://github.com/apache/kafka/pull/13456#discussion_r1158148779 ## storage/src/main/java/org/apache/kafka/storage/internals/epoch/LeaderEpochFileCache.java: ## @@ -147,6 +152,16 @@ private List removeWhileMatching(Iterator

[GitHub] [kafka] showuon commented on pull request #13459: KAFKA-14592: Move FeatureCommand to tools

2023-04-05 Thread via GitHub
showuon commented on PR #13459: URL: https://github.com/apache/kafka/pull/13459#issuecomment-1497091640 > @showuon in this case I think it is not necessary to wait for that KIP because we have a wrapper script and users are supposed to use that. Thanks @fvaleri , but I don't think we

[jira] [Created] (KAFKA-14880) TransactionMetadata with producer epoch -1 should be expirable

2023-04-05 Thread David Jacot (Jira)
David Jacot created KAFKA-14880: --- Summary: TransactionMetadata with producer epoch -1 should be expirable Key: KAFKA-14880 URL: https://issues.apache.org/jira/browse/KAFKA-14880 Project: Kafka

[GitHub] [kafka] dajac opened a new pull request, #13499: KAFKA-14880; TransactionMetadata with producer epoch -1 should be expirable

2023-04-05 Thread via GitHub
dajac opened a new pull request, #13499: URL: https://github.com/apache/kafka/pull/13499 We have seen the following error in logs: ``` "Mar 22, 2019 @ 21:57:56.655",Error,"kafka-0-0","transaction-log-manager-0","Uncaught exception in scheduled task 'transactionalId-expiration'","

[GitHub] [kafka] satishd commented on a diff in pull request #13487: KAFKA-9550 Copying log segments to tiered storage in RemoteLogManager

2023-04-05 Thread via GitHub
satishd commented on code in PR #13487: URL: https://github.com/apache/kafka/pull/13487#discussion_r1158197887 ## core/src/main/scala/kafka/log/remote/RemoteLogManager.scala: ## @@ -272,15 +364,270 @@ class RemoteLogManager(rlmConfig: RemoteLogManagerConfig, None } +

[GitHub] [kafka] satishd commented on a diff in pull request #13487: KAFKA-9550 Copying log segments to tiered storage in RemoteLogManager

2023-04-05 Thread via GitHub
satishd commented on code in PR #13487: URL: https://github.com/apache/kafka/pull/13487#discussion_r1158198885 ## core/src/main/scala/kafka/log/remote/RemoteLogManager.scala: ## @@ -143,24 +216,44 @@ class RemoteLogManager(rlmConfig: RemoteLogManagerConfig,

[GitHub] [kafka] satishd commented on a diff in pull request #13487: KAFKA-9550 Copying log segments to tiered storage in RemoteLogManager

2023-04-05 Thread via GitHub
satishd commented on code in PR #13487: URL: https://github.com/apache/kafka/pull/13487#discussion_r1158203473 ## core/src/main/scala/kafka/log/remote/RemoteLogManager.scala: ## @@ -272,15 +364,270 @@ class RemoteLogManager(rlmConfig: RemoteLogManagerConfig, None } +

[GitHub] [kafka] satishd commented on a diff in pull request #13487: KAFKA-9550 Copying log segments to tiered storage in RemoteLogManager

2023-04-05 Thread via GitHub
satishd commented on code in PR #13487: URL: https://github.com/apache/kafka/pull/13487#discussion_r1158205451 ## core/src/main/scala/kafka/log/remote/RemoteLogManager.scala: ## @@ -272,15 +364,270 @@ class RemoteLogManager(rlmConfig: RemoteLogManagerConfig, None } +

[GitHub] [kafka] showuon commented on a diff in pull request #13487: KAFKA-9550 Copying log segments to tiered storage in RemoteLogManager

2023-04-05 Thread via GitHub
showuon commented on code in PR #13487: URL: https://github.com/apache/kafka/pull/13487#discussion_r1158212079 ## core/src/main/scala/kafka/log/remote/RemoteLogManager.scala: ## @@ -272,15 +364,270 @@ class RemoteLogManager(rlmConfig: RemoteLogManagerConfig, None } +

[GitHub] [kafka] mimaison commented on a diff in pull request #12992: KAFKA-14376-KIP887: Add ConfigProvider to make use of environment variables

2023-04-05 Thread via GitHub
mimaison commented on code in PR #12992: URL: https://github.com/apache/kafka/pull/12992#discussion_r1158216658 ## clients/src/main/java/org/apache/kafka/common/config/provider/EnvVarConfigProvider.java: ## @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] [kafka] vamossagar12 commented on a diff in pull request #11433: KAFKA-13295: Avoiding Transation timeouts arising due to long restora…

2023-04-05 Thread via GitHub
vamossagar12 commented on code in PR #11433: URL: https://github.com/apache/kafka/pull/11433#discussion_r1158249466 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -342,7 +342,18 @@ public void handleAssignment(final Map> activeTasks

[GitHub] [kafka] vamossagar12 commented on pull request #11433: KAFKA-13295: Avoiding Transation timeouts arising due to long restora…

2023-04-05 Thread via GitHub
vamossagar12 commented on PR #11433: URL: https://github.com/apache/kafka/pull/11433#issuecomment-1497172421 > > Regarding system tests, is there a way to trigger via Jenkins ? (Sorry I am not aware of this). > > https://jenkins.confluent.io/job/system-test-kafka-branch-builder/build

[GitHub] [kafka] Schm1tz1 commented on a diff in pull request #12992: KAFKA-14376-KIP887: Add ConfigProvider to make use of environment variables

2023-04-05 Thread via GitHub
Schm1tz1 commented on code in PR #12992: URL: https://github.com/apache/kafka/pull/12992#discussion_r1158272582 ## clients/src/main/java/org/apache/kafka/common/config/provider/EnvVarConfigProvider.java: ## @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] [kafka] Schm1tz1 commented on pull request #12992: KAFKA-14376-KIP887: Add ConfigProvider to make use of environment variables

2023-04-05 Thread via GitHub
Schm1tz1 commented on PR #12992: URL: https://github.com/apache/kafka/pull/12992#issuecomment-1497209688 Done - just passed local build, test and checkstyle. Will push the changes now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Resolved] (KAFKA-14741) Add description field to connector configs

2023-04-05 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-14741. Resolution: Won't Do > Add description field to connector configs > --

[GitHub] [kafka] showuon commented on a diff in pull request #13492: KAFKA-14752: Kafka examples improvements

2023-04-05 Thread via GitHub
showuon commented on code in PR #13492: URL: https://github.com/apache/kafka/pull/13492#discussion_r1158297917 ## checkstyle/import-control-core.xml: ## @@ -71,7 +71,6 @@ - Review Comment: Nice catch! -- This is an automated message from the Apache Git Se

[GitHub] [kafka] fvaleri commented on pull request #13459: KAFKA-14592: Move FeatureCommand to tools

2023-04-05 Thread via GitHub
fvaleri commented on PR #13459: URL: https://github.com/apache/kafka/pull/13459#issuecomment-1497243050 @showuon, KIP-906 says that we should use the reflection way only in case of missing wrapper scripts to avoid a breaking change (the run-class way uses the package name). -- This is an

[GitHub] [kafka] mimaison commented on a diff in pull request #12992: KAFKA-14376-KIP887: Add ConfigProvider to make use of environment variables

2023-04-05 Thread via GitHub
mimaison commented on code in PR #12992: URL: https://github.com/apache/kafka/pull/12992#discussion_r1158337696 ## clients/src/main/java/org/apache/kafka/common/config/provider/EnvVarConfigProvider.java: ## @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] [kafka] Schm1tz1 commented on a diff in pull request #12992: KAFKA-14376-KIP887: Add ConfigProvider to make use of environment variables

2023-04-05 Thread via GitHub
Schm1tz1 commented on code in PR #12992: URL: https://github.com/apache/kafka/pull/12992#discussion_r1158353376 ## clients/src/main/java/org/apache/kafka/common/config/provider/EnvVarConfigProvider.java: ## @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] [kafka] showuon commented on pull request #13456: KAFKA-14850: introduce InMemoryLeaderEpochCheckpoint

2023-04-05 Thread via GitHub
showuon commented on PR #13456: URL: https://github.com/apache/kafka/pull/13456#issuecomment-1497382648 Failed tests are unrelated: ``` Build / JDK 17 and Scala 2.13 / kafka.security.authorizer.AuthorizerTest.testAuthorizeWithPrefixedResource(String).quorum=kraft Build / JDK

[GitHub] [kafka] showuon merged pull request #13456: KAFKA-14850: introduce InMemoryLeaderEpochCheckpoint

2023-04-05 Thread via GitHub
showuon merged PR #13456: URL: https://github.com/apache/kafka/pull/13456 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] tinaselenge opened a new pull request, #13500: Install missing iputils-ping for system tests

2023-04-05 Thread via GitHub
tinaselenge opened a new pull request, #13500: URL: https://github.com/apache/kafka/pull/13500 Some system tests from `kafkatest.tests.core.network_degrade_test` are failing due to missing utility iputils-ping. ``` [DEBUG - 2023-02-04 01:34:56,322 - network_degrade_test - test_latenc

[GitHub] [kafka] songnon commented on a diff in pull request #13451: KAFKA-14808: fix leaderless partition issue when controller removes u…

2023-04-05 Thread via GitHub
songnon commented on code in PR #13451: URL: https://github.com/apache/kafka/pull/13451#discussion_r1158550359 ## core/src/main/scala/kafka/controller/KafkaController.scala: ## @@ -802,8 +802,11 @@ class KafkaController(val config: KafkaConfig, // If there is a reassignme

[GitHub] [kafka] splett2 opened a new pull request, #13501: MINOR: Use `exists` instead of return in DelayedFetch

2023-04-05 Thread via GitHub
splett2 opened a new pull request, #13501: URL: https://github.com/apache/kafka/pull/13501 Using a return statement in a scala lambda is somewhat non-idiomatic. We can use an `exists` instead of a `foreach` and still short-circuit the loop on a `true`. ### Committer Checklist (exclud

[GitHub] [kafka] ijuma commented on a diff in pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

2023-04-05 Thread via GitHub
ijuma commented on code in PR #13135: URL: https://github.com/apache/kafka/pull/13135#discussion_r1158577698 ## clients/src/main/java/org/apache/kafka/common/utils/BytesStreamBufferSource.java: ## @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

[GitHub] [kafka] mimaison commented on pull request #12992: KAFKA-14376-KIP887: Add ConfigProvider to make use of environment variables

2023-04-05 Thread via GitHub
mimaison commented on PR #12992: URL: https://github.com/apache/kafka/pull/12992#issuecomment-1497642780 @OneCricketeer @viktorsomogyi @C0urante do you have any other questions or suggestions? -- This is an automated message from the Apache Git Service. To respond to the message, please l

[jira] [Commented] (KAFKA-14084) Support SCRAM when using KRaft mode

2023-04-05 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17708949#comment-17708949 ] Mickael Maison commented on KAFKA-14084: Is https://issues.apache.org/jira/brows

[GitHub] [kafka] divijvaidya commented on pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

2023-04-05 Thread via GitHub
divijvaidya commented on PR #13135: URL: https://github.com/apache/kafka/pull/13135#issuecomment-1497651824 Thank you @ijuma for your review. I will address the comments in next one week since I am on vacation for next few days. > Was this done? Yes, consumer performance impact

[GitHub] [kafka] OneCricketeer commented on a diff in pull request #12992: KAFKA-14376-KIP887: Add ConfigProvider to make use of environment variables

2023-04-05 Thread via GitHub
OneCricketeer commented on code in PR #12992: URL: https://github.com/apache/kafka/pull/12992#discussion_r1158667599 ## clients/src/test/java/org/apache/kafka/common/config/provider/EnvVarConfigProviderTest.java: ## @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [kafka] jolshan opened a new pull request, #13502: KAFKA-14790: Add more AddPartitionsToTxn tests in KafkaApis and Authorizer tests

2023-04-05 Thread via GitHub
jolshan opened a new pull request, #13502: URL: https://github.com/apache/kafka/pull/13502 Added tests to KafkaApis. We can decide if this is sufficient or we want Authorizer specific tests too. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implemen

[GitHub] [kafka] jolshan commented on a diff in pull request #13499: KAFKA-14880; TransactionMetadata with producer epoch -1 should be expirable

2023-04-05 Thread via GitHub
jolshan commented on code in PR #13499: URL: https://github.com/apache/kafka/pull/13499#discussion_r1158721755 ## core/src/main/scala/kafka/coordinator/transaction/TransactionMetadata.scala: ## @@ -382,7 +382,10 @@ private[transaction] class TransactionMetadata(val transactiona

[GitHub] [kafka] viktorsomogyi commented on a diff in pull request #12992: KAFKA-14376-KIP887: Add ConfigProvider to make use of environment variables

2023-04-05 Thread via GitHub
viktorsomogyi commented on code in PR #12992: URL: https://github.com/apache/kafka/pull/12992#discussion_r1158707122 ## clients/src/test/java/org/apache/kafka/common/config/provider/EnvVarConfigProviderTest.java: ## @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [kafka] dajac commented on a diff in pull request #13499: KAFKA-14880; TransactionMetadata with producer epoch -1 should be expirable

2023-04-05 Thread via GitHub
dajac commented on code in PR #13499: URL: https://github.com/apache/kafka/pull/13499#discussion_r1158725694 ## core/src/main/scala/kafka/coordinator/transaction/TransactionMetadata.scala: ## @@ -382,7 +382,10 @@ private[transaction] class TransactionMetadata(val transactionalI

[GitHub] [kafka] jolshan commented on a diff in pull request #13499: KAFKA-14880; TransactionMetadata with producer epoch -1 should be expirable

2023-04-05 Thread via GitHub
jolshan commented on code in PR #13499: URL: https://github.com/apache/kafka/pull/13499#discussion_r1158727282 ## core/src/main/scala/kafka/coordinator/transaction/TransactionMetadata.scala: ## @@ -382,7 +382,10 @@ private[transaction] class TransactionMetadata(val transactiona

[GitHub] [kafka] jolshan commented on pull request #13463: KAFKA-14854: Refactor inter broker send thread to handle all interbroker requests on one thread

2023-04-05 Thread via GitHub
jolshan commented on PR #13463: URL: https://github.com/apache/kafka/pull/13463#issuecomment-1497771584 After some offline discussion on the threading model and how we mute connections on destination, this PR will be put on hold for now. -- This is an automated message from the Apache Git

[GitHub] [kafka] mjsax merged pull request #13442: KAFKA-14491: [20/N] Add public-facing methods for versioned stores

2023-04-05 Thread via GitHub
mjsax merged PR #13442: URL: https://github.com/apache/kafka/pull/13442 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] rittikaadhikari opened a new pull request, #13503: MINOR: Refactor TierStateMachine related tests into a separate test file

2023-04-05 Thread via GitHub
rittikaadhikari opened a new pull request, #13503: URL: https://github.com/apache/kafka/pull/13503 This PR builds off of KAFKA-14685 and refactors any tests explicitly related to `ReplicaFetcherTierStateMachine` into a separate testing file `ReplicaFetcherTierStateMachineTest`. -- This i

[jira] [Resolved] (KAFKA-14491) Introduce Versioned Key-Value Stores to Kafka Streams

2023-04-05 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-14491. - Fix Version/s: 3.5.0 Resolution: Fixed > Introduce Versioned Key-Value Stores to

[GitHub] [kafka] junrao commented on a diff in pull request #13454: MINOR: move RecordReader from org.apache.kafka.tools to org.apache.co…

2023-04-05 Thread via GitHub
junrao commented on code in PR #13454: URL: https://github.com/apache/kafka/pull/13454#discussion_r1158757537 ## build.gradle: ## @@ -858,6 +858,7 @@ project(':core') { implementation project(':group-coordinator') implementation project(':metadata') implementation

[jira] [Commented] (KAFKA-14084) Support SCRAM when using KRaft mode

2023-04-05 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17709009#comment-17709009 ] Proven Provenzano commented on KAFKA-14084: --- Migration support is needed and t

[GitHub] [kafka] vamossagar12 opened a new pull request, #13504: Kafka 14750

2023-04-05 Thread via GitHub
vamossagar12 opened a new pull request, #13504: URL: https://github.com/apache/kafka/pull/13504 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of te

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-05 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1158766750 ## core/src/main/scala/kafka/network/RequestChannel.scala: ## @@ -483,14 +493,10 @@ class RequestChannel(val queueSize: Int, def sendShutdownRequest(): Unit = requ

[jira] [Commented] (KAFKA-14084) Support SCRAM when using KRaft mode

2023-04-05 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17709011#comment-17709011 ] Mickael Maison commented on KAFKA-14084: Thanks for the quick reply. Is this par

[GitHub] [kafka] mimaison commented on pull request #11478: KAFKA-13299: Accept duplicate listener on port for IPv4/IPv6

2023-04-05 Thread via GitHub
mimaison commented on PR #11478: URL: https://github.com/apache/kafka/pull/11478#issuecomment-1497814558 @mdedetrich Are you still interested in completing this KIP? IIRC the changes were fine but I was wondering if an integration test would be necessary. @jlprat @C0urante WDYT? -- Th

[GitHub] [kafka] mdedetrich commented on pull request #11478: KAFKA-13299: Accept duplicate listener on port for IPv4/IPv6

2023-04-05 Thread via GitHub
mdedetrich commented on PR #11478: URL: https://github.com/apache/kafka/pull/11478#issuecomment-1497821724 @mimaison Thanks for the reminder! Yes I am still interested, unfortunately due to the non trivial nature of the type of tests I needed to write the this ended up becoming de-pri

[GitHub] [kafka] dajac opened a new pull request, #13505: KAFKA-14462; [5/N] Add ConcurrentEventQueue

2023-04-05 Thread via GitHub
dajac opened a new pull request, #13505: URL: https://github.com/apache/kafka/pull/13505 This patch adds the `ConcurrentEventQueue` which will be used in the runtime of the new group coordinator. The aim of this queue is to basically have a queue per __consumer_group partitions and to ensur

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-05 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1158782105 ## core/src/main/scala/kafka/server/AddPartitionsToTxnManager.scala: ## @@ -0,0 +1,170 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

[jira] [Commented] (KAFKA-14084) Support SCRAM when using KRaft mode

2023-04-05 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17709015#comment-17709015 ] Proven Provenzano commented on KAFKA-14084: --- The migration work I described is

[GitHub] [kafka] jolshan commented on a diff in pull request #13505: KAFKA-14462; [5/N] Add ConcurrentEventQueue

2023-04-05 Thread via GitHub
jolshan commented on code in PR #13505: URL: https://github.com/apache/kafka/pull/13505#discussion_r1158787361 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/ConcurrentEventQueue.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] mjsax commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-05 Thread via GitHub
mjsax commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1158792312 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableValueGetter.java: ## @@ -25,5 +25,21 @@ ValueAndTimestamp get(K key); +/** + *

[GitHub] [kafka] chia7712 commented on pull request #13454: MINOR: move RecordReader from org.apache.kafka.tools to org.apache.co…

2023-04-05 Thread via GitHub
chia7712 commented on PR #13454: URL: https://github.com/apache/kafka/pull/13454#issuecomment-1497842365 > Why does core depend on tools-api? [ConsoleProducer](https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/tools/ConsoleProducer.scala) is still in core module. #

[GitHub] [kafka] chia7712 commented on a diff in pull request #13454: MINOR: move RecordReader from org.apache.kafka.tools to org.apache.co…

2023-04-05 Thread via GitHub
chia7712 commented on code in PR #13454: URL: https://github.com/apache/kafka/pull/13454#discussion_r1158799476 ## build.gradle: ## @@ -1766,13 +1769,69 @@ project(':storage') { } } +project(':tools:tools-api') { Review Comment: I have added the tools.api package to ja

[GitHub] [kafka] chia7712 commented on a diff in pull request #13454: MINOR: move RecordReader from org.apache.kafka.tools to org.apache.co…

2023-04-05 Thread via GitHub
chia7712 commented on code in PR #13454: URL: https://github.com/apache/kafka/pull/13454#discussion_r1158800055 ## build.gradle: ## @@ -1766,13 +1769,69 @@ project(':storage') { } } +project(':tools:tools-api') { Review Comment: The doc-related test is shown in previou

[GitHub] [kafka] jolshan commented on a diff in pull request #13501: MINOR: Use `exists` instead of return in DelayedFetch

2023-04-05 Thread via GitHub
jolshan commented on code in PR #13501: URL: https://github.com/apache/kafka/pull/13501#discussion_r1158802468 ## core/src/main/scala/kafka/server/DelayedFetch.scala: ## @@ -93,14 +93,14 @@ class DelayedFetch( if (endOffset.onOlderSegment(fetchOffset)) {

[GitHub] [kafka] chia7712 opened a new pull request, #13506: MINOR: fix invalid usage in java docs

2023-04-05 Thread via GitHub
chia7712 opened a new pull request, #13506: URL: https://github.com/apache/kafka/pull/13506 noted this issue when testing javadoc command for #13454 the result is shown below. https://user-images.githubusercontent.com/6234750/230157066-e0b27487-4c28-43f8-996b-84ab54ea17cc.png";>

[GitHub] [kafka] jolshan commented on a diff in pull request #13493: KAFKA-14852: Propagate Topic Ids to the Group Coordinator for Offset Fetch

2023-04-05 Thread via GitHub
jolshan commented on code in PR #13493: URL: https://github.com/apache/kafka/pull/13493#discussion_r1158813853 ## core/src/main/scala/kafka/coordinator/group/GroupCoordinatorAdapter.scala: ## @@ -291,24 +291,24 @@ private[group] class GroupCoordinatorAdapter( topics: util.L

[GitHub] [kafka] mjsax commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-05 Thread via GitHub
mjsax commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1158814912 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableKTableInnerJoin.java: ## @@ -153,11 +154,37 @@ public void init(final ProcessorContext context)

[GitHub] [kafka] jolshan commented on a diff in pull request #13493: KAFKA-14852: Propagate Topic Ids to the Group Coordinator for Offset Fetch

2023-04-05 Thread via GitHub
jolshan commented on code in PR #13493: URL: https://github.com/apache/kafka/pull/13493#discussion_r1158816855 ## core/src/main/scala/kafka/coordinator/group/GroupMetadata.scala: ## @@ -824,14 +823,14 @@ private[group] class GroupMetadata(val groupId: String, initialState: Grou

[GitHub] [kafka] jolshan commented on a diff in pull request #13493: KAFKA-14852: Propagate Topic Ids to the Group Coordinator for Offset Fetch

2023-04-05 Thread via GitHub
jolshan commented on code in PR #13493: URL: https://github.com/apache/kafka/pull/13493#discussion_r1158817715 ## core/src/main/scala/kafka/coordinator/group/GroupCoordinatorAdapter.scala: ## @@ -291,24 +291,24 @@ private[group] class GroupCoordinatorAdapter( topics: util.L

[GitHub] [kafka] mjsax commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-05 Thread via GitHub
mjsax commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1158814912 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableKTableInnerJoin.java: ## @@ -153,11 +154,37 @@ public void init(final ProcessorContext context)

[GitHub] [kafka] cadonna commented on a diff in pull request #13269: KAFKA-12634 enforce checkpoint after restoration

2023-04-05 Thread via GitHub
cadonna commented on code in PR #13269: URL: https://github.com/apache/kafka/pull/13269#discussion_r1158824245 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamTaskTest.java: ## @@ -274,6 +287,7 @@ public void cleanup() throws IOException {

[GitHub] [kafka] Schm1tz1 commented on a diff in pull request #12992: KAFKA-14376-KIP887: Add ConfigProvider to make use of environment variables

2023-04-05 Thread via GitHub
Schm1tz1 commented on code in PR #12992: URL: https://github.com/apache/kafka/pull/12992#discussion_r1158851998 ## clients/src/test/java/org/apache/kafka/common/config/provider/EnvVarConfigProviderTest.java: ## @@ -0,0 +1,115 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] Schm1tz1 commented on pull request #12992: KAFKA-14376-KIP887: Add ConfigProvider to make use of environment variables

2023-04-05 Thread via GitHub
Schm1tz1 commented on PR #12992: URL: https://github.com/apache/kafka/pull/12992#issuecomment-1497915645 > Please fix the missing new line issues, otherwise looks good. Thanks! Newlines are fixed. -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [kafka] splett2 commented on a diff in pull request #13501: MINOR: Use `exists` instead of return in DelayedFetch

2023-04-05 Thread via GitHub
splett2 commented on code in PR #13501: URL: https://github.com/apache/kafka/pull/13501#discussion_r1158871931 ## core/src/main/scala/kafka/server/DelayedFetch.scala: ## @@ -93,14 +93,14 @@ class DelayedFetch( if (endOffset.onOlderSegment(fetchOffset)) {

[GitHub] [kafka] guozhangwang commented on a diff in pull request #13369: KAFKA-14172: Should clear cache when active recycled from standby

2023-04-05 Thread via GitHub
guozhangwang commented on code in PR #13369: URL: https://github.com/apache/kafka/pull/13369#discussion_r115604 ## streams/src/main/java/org/apache/kafka/streams/state/internals/NamedCache.java: ## @@ -361,6 +361,13 @@ synchronized void close() { streamsMetrics.rem

[GitHub] [kafka] mjsax merged pull request #13300: KAFKA-10199: Add task updater metrics, part 2

2023-04-05 Thread via GitHub
mjsax merged PR #13300: URL: https://github.com/apache/kafka/pull/13300 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] mjsax commented on a diff in pull request #13369: KAFKA-14172: Should clear cache when active recycled from standby

2023-04-05 Thread via GitHub
mjsax commented on code in PR #13369: URL: https://github.com/apache/kafka/pull/13369#discussion_r1158897055 ## streams/src/main/java/org/apache/kafka/streams/state/internals/NamedCache.java: ## @@ -361,6 +361,13 @@ synchronized void close() { streamsMetrics.removeAllC

[GitHub] [kafka] Hangleton commented on a diff in pull request #13505: KAFKA-14462; [5/N] Add ConcurrentEventQueue

2023-04-05 Thread via GitHub
Hangleton commented on code in PR #13505: URL: https://github.com/apache/kafka/pull/13505#discussion_r1158880294 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/ConcurrentEventQueue.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] Hangleton commented on a diff in pull request #13493: KAFKA-14852: Propagate Topic Ids to the Group Coordinator for Offset Fetch

2023-04-05 Thread via GitHub
Hangleton commented on code in PR #13493: URL: https://github.com/apache/kafka/pull/13493#discussion_r1158907492 ## core/src/main/scala/kafka/coordinator/group/GroupCoordinatorAdapter.scala: ## @@ -291,24 +291,24 @@ private[group] class GroupCoordinatorAdapter( topics: util

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-05 Thread via GitHub
vcrfxia commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1158941945 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableKTableInnerJoin.java: ## @@ -153,11 +154,37 @@ public void init(final ProcessorContext context

[GitHub] [kafka] pprovenzano opened a new pull request, #13507: Scram kraft update user scram credential record [DO NOT MERGE YET]

2023-04-05 Thread via GitHub
pprovenzano opened a new pull request, #13507: URL: https://github.com/apache/kafka/pull/13507 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of tes

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-05 Thread via GitHub
vcrfxia commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1158943426 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableValueGetter.java: ## @@ -25,5 +25,21 @@ ValueAndTimestamp get(K key); +/** +

[GitHub] [kafka] jolshan commented on a diff in pull request #13493: KAFKA-14852: Propagate Topic Ids to the Group Coordinator for Offset Fetch

2023-04-05 Thread via GitHub
jolshan commented on code in PR #13493: URL: https://github.com/apache/kafka/pull/13493#discussion_r1158947529 ## core/src/main/scala/kafka/coordinator/group/GroupMetadata.scala: ## @@ -824,14 +823,14 @@ private[group] class GroupMetadata(val groupId: String, initialState: Grou

[GitHub] [kafka] jolshan commented on a diff in pull request #13505: KAFKA-14462; [5/N] Add ConcurrentEventQueue

2023-04-05 Thread via GitHub
jolshan commented on code in PR #13505: URL: https://github.com/apache/kafka/pull/13505#discussion_r1158951727 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/ConcurrentEventQueue.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] junrao commented on a diff in pull request #13487: KAFKA-9550 Copying log segments to tiered storage in RemoteLogManager

2023-04-05 Thread via GitHub
junrao commented on code in PR #13487: URL: https://github.com/apache/kafka/pull/13487#discussion_r1158975829 ## core/src/main/scala/kafka/log/remote/RemoteLogManager.scala: ## @@ -17,39 +17,97 @@ package kafka.log.remote import kafka.cluster.Partition +import kafka.log.Unif

[GitHub] [kafka] philipnee commented on a diff in pull request #13490: KAFKA-14875: Implement wakeup

2023-04-05 Thread via GitHub
philipnee commented on code in PR #13490: URL: https://github.com/apache/kafka/pull/13490#discussion_r1158996199 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -319,27 +330,50 @@ public Map committed(final Set + * If

[GitHub] [kafka] junrao commented on a diff in pull request #13454: MINOR: move RecordReader from org.apache.kafka.tools to org.apache.co…

2023-04-05 Thread via GitHub
junrao commented on code in PR #13454: URL: https://github.com/apache/kafka/pull/13454#discussion_r1158997043 ## tools/tools-api/src/test/java/org/apache/kafka/tools/api/RecordReaderTest.java: ## @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] jsancio commented on a diff in pull request #11096: Adding reviewers.py to help tag reviewers in commit message

2023-04-05 Thread via GitHub
jsancio commented on code in PR #11096: URL: https://github.com/apache/kafka/pull/11096#discussion_r1159029690 ## reviewers.py: ## @@ -0,0 +1,76 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- + +from collections import defaultdict +import operator +import os +import re + +

[GitHub] [kafka] philipnee commented on a diff in pull request #13490: KAFKA-14875: Implement wakeup

2023-04-05 Thread via GitHub
philipnee commented on code in PR #13490: URL: https://github.com/apache/kafka/pull/13490#discussion_r1159047408 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -130,20 +136,20 @@ public PrototypeAsyncConsumer(final Consume

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13429: KAFKA-14666: Add MM2 in-memory offset translation index for offsets behind replication

2023-04-05 Thread via GitHub
gharris1727 commented on code in PR #13429: URL: https://github.com/apache/kafka/pull/13429#discussion_r1159083568 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/OffsetSyncStore.java: ## @@ -25,17 +25,39 @@ import org.apache.kafka.common.utils.Utils; import or

[GitHub] [kafka] jsancio commented on a diff in pull request #13345: KAFKA-13020; Implement reading Snapshot log append timestamp

2023-04-05 Thread via GitHub
jsancio commented on code in PR #13345: URL: https://github.com/apache/kafka/pull/13345#discussion_r1159095991 ## raft/src/main/java/org/apache/kafka/snapshot/RecordsSnapshotReader.java: ## @@ -121,9 +122,22 @@ private Optional> nextBatch() { Batch batch = iterator.

[GitHub] [kafka] stejani-cflt opened a new pull request, #13508: MINOR: Add information about Min-ISR to log message

2023-04-05 Thread via GitHub
stejani-cflt opened a new pull request, #13508: URL: https://github.com/apache/kafka/pull/13508 Whenever there are changes to the ISR, add an extra string to the existing log message in case the partition is under min ISR. This makes it easier to search the log when partitions go under min

[GitHub] [kafka] guozhangwang commented on pull request #13369: KAFKA-14172: Should clear cache when active recycled from standby

2023-04-05 Thread via GitHub
guozhangwang commented on PR #13369: URL: https://github.com/apache/kafka/pull/13369#issuecomment-1498256168 > LGTM. Feel free to merge (if you don't want to do the rewrite of the test). Managed to do that rewrite :) though it requires some unchecked conditional get still.. -- This

[GitHub] [kafka] vcrfxia opened a new pull request, #13509: KAFKA-14834: [3/N] Timestamped lookups for stream-table joins

2023-04-05 Thread via GitHub
vcrfxia opened a new pull request, #13509: URL: https://github.com/apache/kafka/pull/13509 (This PR is stacked on https://github.com/apache/kafka/pull/13496 and https://github.com/apache/kafka/pull/13497. The first two commits do not need to be reviewed separately.) This PR updates t

[GitHub] [kafka] vcrfxia opened a new pull request, #13510: KAFKA-14834: [4/N] Drop out-of-order records from table-table join with versioned tables

2023-04-05 Thread via GitHub
vcrfxia opened a new pull request, #13510: URL: https://github.com/apache/kafka/pull/13510 (This PR is stacked on https://github.com/apache/kafka/pull/13496 and https://github.com/apache/kafka/pull/13497. The first two commits do not need to be reviewed separately.) This PR updates p

[GitHub] [kafka] guozhangwang merged pull request #13369: KAFKA-14172: Should clear cache when active recycled from standby

2023-04-05 Thread via GitHub
guozhangwang merged PR #13369: URL: https://github.com/apache/kafka/pull/13369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.ap

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13509: KAFKA-14834: [3/N] Timestamped lookups for stream-table joins

2023-04-05 Thread via GitHub
vcrfxia commented on code in PR #13509: URL: https://github.com/apache/kafka/pull/13509#discussion_r1159109536 ## streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamGlobalKTableJoinTest.java: ## @@ -246,4 +262,30 @@ public void shouldJoinOnNullKeyWithNonNul

[GitHub] [kafka] jeffkbkim opened a new pull request, #13511: KAFKA-14869: ignore unknown record types for coordinators

2023-04-05 Thread via GitHub
jeffkbkim opened a new pull request, #13511: URL: https://github.com/apache/kafka/pull/13511 KIP-915: Part 1 of 2 Coordinators should Ignore unknown record types ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify tes

[GitHub] [kafka] guozhangwang commented on pull request #13369: KAFKA-14172: Should clear cache when active recycled from standby

2023-04-05 Thread via GitHub
guozhangwang commented on PR #13369: URL: https://github.com/apache/kafka/pull/13369#issuecomment-1498309901 @mjsax cherry-picked to 3.4 so far; 3.3 gets a lot of conflicts so I'm still working on it. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13429: KAFKA-14666: Add MM2 in-memory offset translation index for offsets behind replication

2023-04-05 Thread via GitHub
gharris1727 commented on code in PR #13429: URL: https://github.com/apache/kafka/pull/13429#discussion_r1159169763 ## connect/mirror/src/test/java/org/apache/kafka/connect/mirror/OffsetSyncStoreTest.java: ## @@ -57,52 +57,96 @@ public void testOffsetTranslation() {

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13429: KAFKA-14666: Add MM2 in-memory offset translation index for offsets behind replication

2023-04-05 Thread via GitHub
gharris1727 commented on code in PR #13429: URL: https://github.com/apache/kafka/pull/13429#discussion_r1159169763 ## connect/mirror/src/test/java/org/apache/kafka/connect/mirror/OffsetSyncStoreTest.java: ## @@ -57,52 +57,96 @@ public void testOffsetTranslation() {

[GitHub] [kafka] junrao commented on a diff in pull request #13489: KAFKA-14617: Fill broker epochs to the AlterPartitionRequest

2023-04-05 Thread via GitHub
junrao commented on code in PR #13489: URL: https://github.com/apache/kafka/pull/13489#discussion_r1159033215 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -977,11 +982,17 @@ class Partition(val topicPartition: TopicPartition, private def isReplicaIsrEligible(

[GitHub] [kafka] showuon commented on a diff in pull request #11096: Adding reviewers.py to help tag reviewers in commit message

2023-04-05 Thread via GitHub
showuon commented on code in PR #11096: URL: https://github.com/apache/kafka/pull/11096#discussion_r1159217911 ## reviewers.py: ## @@ -0,0 +1,76 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- + +from collections import defaultdict +import operator +import os +import re + +

[GitHub] [kafka] showuon commented on a diff in pull request #13459: KAFKA-14592: Move FeatureCommand to tools

2023-04-05 Thread via GitHub
showuon commented on code in PR #13459: URL: https://github.com/apache/kafka/pull/13459#discussion_r1159227186 ## tools/src/main/java/org/apache/kafka/tools/FeatureCommand.java: ## @@ -0,0 +1,331 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * c

[GitHub] [kafka] showuon commented on a diff in pull request #13459: KAFKA-14592: Move FeatureCommand to tools

2023-04-05 Thread via GitHub
showuon commented on code in PR #13459: URL: https://github.com/apache/kafka/pull/13459#discussion_r1159239494 ## tools/src/test/java/org/apache/kafka/tools/FeatureCommandTest.java: ## @@ -0,0 +1,292 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

[GitHub] [kafka] chia7712 commented on a diff in pull request #13454: MINOR: move RecordReader from org.apache.kafka.tools to org.apache.co…

2023-04-05 Thread via GitHub
chia7712 commented on code in PR #13454: URL: https://github.com/apache/kafka/pull/13454#discussion_r1159261837 ## tools/tools-api/src/test/java/org/apache/kafka/tools/api/RecordReaderTest.java: ## @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under o

[GitHub] [kafka] CalvinConfluent commented on a diff in pull request #13489: KAFKA-14617: Fill broker epochs to the AlterPartitionRequest

2023-04-05 Thread via GitHub
CalvinConfluent commented on code in PR #13489: URL: https://github.com/apache/kafka/pull/13489#discussion_r1159263750 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -992,6 +1003,11 @@ class Partition(val topicPartition: TopicPartition, } } + private def

[GitHub] [kafka] yashmayya commented on a diff in pull request #13424: KAFKA-14783 (KIP-875): New STOPPED state for connectors

2023-04-05 Thread via GitHub
yashmayya commented on code in PR #13424: URL: https://github.com/apache/kafka/pull/13424#discussion_r1159264066 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -1090,6 +1090,39 @@ public void putConnectorConfig(final S

  1   2   >