[jira] [Commented] (KAFKA-14696) CVE-2023-25194: Apache Kafka: Possible RCE/Denial of service attack via SASL JAAS JndiLoginModule configuration using Kafka Connect

2023-02-10 Thread MillieZhang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17686925#comment-17686925 ] MillieZhang commented on KAFKA-14696: - As you said, I'm going to fix this by turning

[GitHub] [kafka] showuon commented on a diff in pull request #13206: [KAFKA-14685] Refactor logic to handle OFFSET_MOVED_TO_TIERED_STORAGE error

2023-02-10 Thread via GitHub
showuon commented on code in PR #13206: URL: https://github.com/apache/kafka/pull/13206#discussion_r1102353791 ## core/src/main/java/kafka/server/TierStateMachine.java: ## @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor

[GitHub] [kafka] dajac opened a new pull request, #13230: KAFKA-14704; Follower should truncate before incrementing high watermark

2023-02-10 Thread via GitHub
dajac opened a new pull request, #13230: URL: https://github.com/apache/kafka/pull/13230 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing s

[GitHub] [kafka] vamossagar12 commented on pull request #13127: KAFKA-14586: Moving StreamResetter to tools

2023-02-10 Thread via GitHub
vamossagar12 commented on PR #13127: URL: https://github.com/apache/kafka/pull/13127#issuecomment-1425469233 @cadonna , @fvaleri I ran the system test for StreamsResetter from my local (and fixed the checkstyle). The tests are running, would keep an eye on it => ``` ./gradlew clean

[GitHub] [kafka] divijvaidya commented on pull request #13220: Kafka 14253

2023-02-10 Thread via GitHub
divijvaidya commented on PR #13220: URL: https://github.com/apache/kafka/pull/13220#issuecomment-1425505624 Hey @ouyangnengda I see that this is your first contribution to Kafka. Thank you for your interest in the project! Since you are new to the community, let me fill you in

[GitHub] [kafka] showuon commented on a diff in pull request #13040: KAFKA-14480 Move/Rewrite ProducerStateManager to storage module.

2023-02-10 Thread via GitHub
showuon commented on code in PR #13040: URL: https://github.com/apache/kafka/pull/13040#discussion_r1102512047 ## storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateManager.java: ## @@ -0,0 +1,679 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] divijvaidya commented on pull request #13078: KAFKA-13999: Add ProducerCount metrics (KIP-847)

2023-02-10 Thread via GitHub
divijvaidya commented on PR #13078: URL: https://github.com/apache/kafka/pull/13078#issuecomment-1425521316 > One concern is that the ConcurrentHashMap could take up more space. This metric was introduced to try to get this memory usage under control. The hash map in question here sto

[jira] [Commented] (KAFKA-14704) Follower should truncate before incrementing high watermark

2023-02-10 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17686986#comment-17686986 ] Luke Chen commented on KAFKA-14704: --- [~dajac] , could you add some bug description? Th

[GitHub] [kafka] divijvaidya commented on pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2023-02-10 Thread via GitHub
divijvaidya commented on PR #12590: URL: https://github.com/apache/kafka/pull/12590#issuecomment-1425527332 Thank you @dajac for your patience through this PR. It took a long time but it would definitely improve consumers! Cheers! -- This is an automated message from the Apache Git Servic

[GitHub] [kafka] dajac commented on pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2023-02-10 Thread via GitHub
dajac commented on PR #12590: URL: https://github.com/apache/kafka/pull/12590#issuecomment-1425530666 @divijvaidya Sorry again for the long delay. By the way, I was wondering if we should also do this in the AbstractFetcherThread in order to close sessions used by replication when a broker

[jira] [Updated] (KAFKA-14704) Follower should truncate before incrementing high watermark

2023-02-10 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot updated KAFKA-14704: Description: When a leader becomes a follower, it is likely that it has uncommitted records in it

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13219: MINOR: Simplify JUnit assertions; remove accidental unnecessary code

2023-02-10 Thread via GitHub
divijvaidya commented on code in PR #13219: URL: https://github.com/apache/kafka/pull/13219#discussion_r1102521315 ## clients/src/test/java/org/apache/kafka/common/utils/ImplicitLinkedHashCollectionTest.java: ## @@ -142,8 +143,8 @@ static void expectTraversal(Iterator iterator,

[jira] [Commented] (KAFKA-14704) Follower should truncate before incrementing high watermark

2023-02-10 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17686993#comment-17686993 ] David Jacot commented on KAFKA-14704: - [~showuon] Done. Sorry for that. > Follower

[GitHub] [kafka] divijvaidya commented on pull request #12590: KAFKA-7109: Close fetch sessions on close of consumer

2023-02-10 Thread via GitHub
divijvaidya commented on PR #12590: URL: https://github.com/apache/kafka/pull/12590#issuecomment-1425536975 > @divijvaidya Sorry again for the long delay. By the way, I was wondering if we should also do this in the AbstractFetcherThread in order to close sessions used by replication when a

[jira] [Commented] (KAFKA-14704) Follower should truncate before incrementing high watermark

2023-02-10 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17686996#comment-17686996 ] Luke Chen commented on KAFKA-14704: --- No worries! Nice catch!   > Follower should tru

[GitHub] [kafka] fvaleri commented on a diff in pull request #13214: KAFKA-14577: Move the scala ConsoleProducer from core to tools module

2023-02-10 Thread via GitHub
fvaleri commented on code in PR #13214: URL: https://github.com/apache/kafka/pull/13214#discussion_r1102548459 ## tools/src/main/java/org/apache/kafka/tools/MessageReader.java: ## @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * con

[GitHub] [kafka] fvaleri commented on a diff in pull request #13214: KAFKA-14577: Move the scala ConsoleProducer from core to tools module

2023-02-10 Thread via GitHub
fvaleri commented on code in PR #13214: URL: https://github.com/apache/kafka/pull/13214#discussion_r1102548459 ## tools/src/main/java/org/apache/kafka/tools/MessageReader.java: ## @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * con

[GitHub] [kafka] satishd commented on pull request #13040: KAFKA-14480 Move/Rewrite ProducerStateManager to storage module.

2023-02-10 Thread via GitHub
satishd commented on PR #13040: URL: https://github.com/apache/kafka/pull/13040#issuecomment-1425587517 Thanks @showuon for your review. Addressed them with the latest commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [kafka] fvaleri commented on a diff in pull request #13204: KAFKA-14593: Move LeaderElectionCommand to tools

2023-02-10 Thread via GitHub
fvaleri commented on code in PR #13204: URL: https://github.com/apache/kafka/pull/13204#discussion_r1102366936 ## build.gradle: ## @@ -1756,6 +1756,7 @@ project(':tools') { dependencies { implementation project(':clients') +implementation project(':core') Review C

[GitHub] [kafka] Hangleton commented on a diff in pull request #13214: KAFKA-14577: Move the scala ConsoleProducer from core to tools module

2023-02-10 Thread via GitHub
Hangleton commented on code in PR #13214: URL: https://github.com/apache/kafka/pull/13214#discussion_r1102625366 ## tools/src/main/java/org/apache/kafka/tools/MessageReader.java: ## @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * c

[GitHub] [kafka] Hangleton commented on a diff in pull request #13215: KAFKA-14578: Move ConsumerPerformance to tools

2023-02-10 Thread via GitHub
Hangleton commented on code in PR #13215: URL: https://github.com/apache/kafka/pull/13215#discussion_r1102626937 ## tools/src/main/java/org/apache/kafka/tools/ConsumerPerformance.java: ## @@ -0,0 +1,394 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mor

[GitHub] [kafka] Hangleton commented on a diff in pull request #13215: KAFKA-14578: Move ConsumerPerformance to tools

2023-02-10 Thread via GitHub
Hangleton commented on code in PR #13215: URL: https://github.com/apache/kafka/pull/13215#discussion_r1102629200 ## tools/src/main/java/org/apache/kafka/tools/ConsumerPerformance.java: ## @@ -0,0 +1,394 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mor

[GitHub] [kafka] vamossagar12 commented on pull request #13127: KAFKA-14586: Moving StreamResetter to tools

2023-02-10 Thread via GitHub
vamossagar12 commented on PR #13127: URL: https://github.com/apache/kafka/pull/13127#issuecomment-1425650698 Tests seemed to have passed for one of the builds. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [kafka] mimaison commented on a diff in pull request #12366: KAFKA-14021: Implement new KIP-618 APIs in MirrorSourceConnector

2023-02-10 Thread via GitHub
mimaison commented on code in PR #12366: URL: https://github.com/apache/kafka/pull/12366#discussion_r1102632023 ## connect/mirror/README.md: ## @@ -194,6 +194,10 @@ it is important to keep configuration consistent across flows to the same target cluster. In most cases, your en

[jira] [Created] (KAFKA-14705) Delete kafka.tools.JmxTool redirection object

2023-02-10 Thread Federico Valeri (Jira)
Federico Valeri created KAFKA-14705: --- Summary: Delete kafka.tools.JmxTool redirection object Key: KAFKA-14705 URL: https://issues.apache.org/jira/browse/KAFKA-14705 Project: Kafka Issue Typ

[jira] [Commented] (KAFKA-14705) Delete kafka.tools.JmxTool redirection object

2023-02-10 Thread Federico Valeri (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17687038#comment-17687038 ] Federico Valeri commented on KAFKA-14705: - https://github.com/apache/kafka/pull/

[GitHub] [kafka] Hangleton commented on a diff in pull request #13206: [KAFKA-14685] Refactor logic to handle OFFSET_MOVED_TO_TIERED_STORAGE error

2023-02-10 Thread via GitHub
Hangleton commented on code in PR #13206: URL: https://github.com/apache/kafka/pull/13206#discussion_r1102707234 ## core/src/main/scala/kafka/server/AbstractFetcherThread.scala: ## @@ -683,33 +655,24 @@ abstract class AbstractFetcherThread(name: String, * produced to the

[GitHub] [kafka] ijuma commented on pull request #13195: Minor: Add JmxTool note to 3.5.0 notable changes

2023-02-10 Thread via GitHub
ijuma commented on PR #13195: URL: https://github.com/apache/kafka/pull/13195#issuecomment-1425806964 Was a KIP submitted for this? Generally, if something ends up in the release notes due to user impact, it requires a KIP. -- This is an automated message from the Apache Git Service. To r

[GitHub] [kafka] ijuma commented on pull request #13171: KAFKA-14584: Move StateChangeLogMerger tool

2023-02-10 Thread via GitHub
ijuma commented on PR #13171: URL: https://github.com/apache/kafka/pull/13171#issuecomment-1425808110 Yeah, I think we should consider simply removing the tool if it doesn't work. But if it was exposed in ` bin`, we'd need a KIP for it. -- This is an automated message from the Apache Git

[GitHub] [kafka] fvaleri commented on a diff in pull request #13215: KAFKA-14578: Move ConsumerPerformance to tools

2023-02-10 Thread via GitHub
fvaleri commented on code in PR #13215: URL: https://github.com/apache/kafka/pull/13215#discussion_r1102766672 ## tools/src/main/java/org/apache/kafka/tools/ConsumerPerformance.java: ## @@ -0,0 +1,394 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] [kafka] fvaleri commented on a diff in pull request #13215: KAFKA-14578: Move ConsumerPerformance to tools

2023-02-10 Thread via GitHub
fvaleri commented on code in PR #13215: URL: https://github.com/apache/kafka/pull/13215#discussion_r1102766672 ## tools/src/main/java/org/apache/kafka/tools/ConsumerPerformance.java: ## @@ -0,0 +1,394 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] [kafka] fvaleri commented on a diff in pull request #13215: KAFKA-14578: Move ConsumerPerformance to tools

2023-02-10 Thread via GitHub
fvaleri commented on code in PR #13215: URL: https://github.com/apache/kafka/pull/13215#discussion_r1102766672 ## tools/src/main/java/org/apache/kafka/tools/ConsumerPerformance.java: ## @@ -0,0 +1,394 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

[jira] [Updated] (KAFKA-14705) Delete kafka.tools.JmxTool redirection

2023-02-10 Thread Federico Valeri (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Federico Valeri updated KAFKA-14705: Summary: Delete kafka.tools.JmxTool redirection (was: Delete kafka.tools.JmxTool redirect

[jira] [Commented] (KAFKA-14696) CVE-2023-25194: Apache Kafka: Possible RCE/Denial of service attack via SASL JAAS JndiLoginModule configuration using Kafka Connect

2023-02-10 Thread Manikumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17687087#comment-17687087 ] Manikumar commented on KAFKA-14696: --- Yes, you ca fix cherry picking the commit ae22ec

[jira] [Comment Edited] (KAFKA-14696) CVE-2023-25194: Apache Kafka: Possible RCE/Denial of service attack via SASL JAAS JndiLoginModule configuration using Kafka Connect

2023-02-10 Thread Manikumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17687087#comment-17687087 ] Manikumar edited comment on KAFKA-14696 at 2/10/23 1:58 PM:

[GitHub] [kafka] dmariassy commented on a diff in pull request #13224: KAFKA-14700: Produce request interceptors

2023-02-10 Thread via GitHub
dmariassy commented on code in PR #13224: URL: https://github.com/apache/kafka/pull/13224#discussion_r1102799491 ## core/src/test/scala/integration/kafka/api/ProducerSendTestHelpers.scala: ## @@ -0,0 +1,238 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] pprovenzano commented on a diff in pull request #13114: KAFKA-14084: SCRAM support in KRaft.

2023-02-10 Thread via GitHub
pprovenzano commented on code in PR #13114: URL: https://github.com/apache/kafka/pull/13114#discussion_r1102813837 ## core/src/main/scala/kafka/server/ControllerApis.scala: ## @@ -816,6 +817,17 @@ class ControllerApis(val requestChannel: RequestChannel, } } + def ha

[GitHub] [kafka] pprovenzano commented on a diff in pull request #13114: KAFKA-14084: SCRAM support in KRaft.

2023-02-10 Thread via GitHub
pprovenzano commented on code in PR #13114: URL: https://github.com/apache/kafka/pull/13114#discussion_r1102815060 ## metadata/src/main/java/org/apache/kafka/controller/ScramControlManager.java: ## @@ -0,0 +1,307 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] pprovenzano commented on a diff in pull request #13114: KAFKA-14084: SCRAM support in KRaft.

2023-02-10 Thread via GitHub
pprovenzano commented on code in PR #13114: URL: https://github.com/apache/kafka/pull/13114#discussion_r1102815637 ## metadata/src/main/java/org/apache/kafka/image/ScramCredentialData.java: ## @@ -0,0 +1,119 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] pprovenzano commented on a diff in pull request #13114: KAFKA-14084: SCRAM support in KRaft.

2023-02-10 Thread via GitHub
pprovenzano commented on code in PR #13114: URL: https://github.com/apache/kafka/pull/13114#discussion_r1102815866 ## metadata/src/main/java/org/apache/kafka/image/ScramImage.java: ## @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

[GitHub] [kafka] ijuma commented on a diff in pull request #13040: KAFKA-14480 Move/Rewrite ProducerStateManager to storage module.

2023-02-10 Thread via GitHub
ijuma commented on code in PR #13040: URL: https://github.com/apache/kafka/pull/13040#discussion_r1102842008 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -680,19 +680,23 @@ class UnifiedLog(@volatile var logStartOffset: Long, } private[log] def activeProducer

[GitHub] [kafka] satishd commented on a diff in pull request #13040: KAFKA-14480 Move/Rewrite ProducerStateManager to storage module.

2023-02-10 Thread via GitHub
satishd commented on code in PR #13040: URL: https://github.com/apache/kafka/pull/13040#discussion_r1102874502 ## core/src/test/scala/integration/kafka/api/TransactionsTest.scala: ## @@ -685,7 +685,7 @@ class TransactionsTest extends IntegrationTestHarness { // get here w

[GitHub] [kafka] ijuma commented on a diff in pull request #13040: KAFKA-14480 Move/Rewrite ProducerStateManager to storage module.

2023-02-10 Thread via GitHub
ijuma commented on code in PR #13040: URL: https://github.com/apache/kafka/pull/13040#discussion_r1102882346 ## core/src/test/scala/integration/kafka/api/TransactionsTest.scala: ## @@ -685,7 +685,7 @@ class TransactionsTest extends IntegrationTestHarness { // get here wit

[GitHub] [kafka] ijuma commented on a diff in pull request #13040: KAFKA-14480 Move/Rewrite ProducerStateManager to storage module.

2023-02-10 Thread via GitHub
ijuma commented on code in PR #13040: URL: https://github.com/apache/kafka/pull/13040#discussion_r1102882346 ## core/src/test/scala/integration/kafka/api/TransactionsTest.scala: ## @@ -685,7 +685,7 @@ class TransactionsTest extends IntegrationTestHarness { // get here wit

[GitHub] [kafka] satishd commented on a diff in pull request #13040: KAFKA-14480 Move/Rewrite ProducerStateManager to storage module.

2023-02-10 Thread via GitHub
satishd commented on code in PR #13040: URL: https://github.com/apache/kafka/pull/13040#discussion_r1102914037 ## storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateManager.java: ## @@ -0,0 +1,676 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] satishd commented on a diff in pull request #13040: KAFKA-14480 Move/Rewrite ProducerStateManager to storage module.

2023-02-10 Thread via GitHub
satishd commented on code in PR #13040: URL: https://github.com/apache/kafka/pull/13040#discussion_r1102933380 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -680,19 +680,23 @@ class UnifiedLog(@volatile var logStartOffset: Long, } private[log] def activeProduc

[GitHub] [kafka] C0urante commented on a diff in pull request #12366: KAFKA-14021: Implement new KIP-618 APIs in MirrorSourceConnector

2023-02-10 Thread via GitHub
C0urante commented on code in PR #12366: URL: https://github.com/apache/kafka/pull/12366#discussion_r1102971088 ## connect/mirror/README.md: ## @@ -194,6 +194,10 @@ it is important to keep configuration consistent across flows to the same target cluster. In most cases, your en

[GitHub] [kafka] C0urante commented on pull request #12366: KAFKA-14021: Implement new KIP-618 APIs in MirrorSourceConnector

2023-02-10 Thread via GitHub
C0urante commented on PR #12366: URL: https://github.com/apache/kafka/pull/12366#issuecomment-1426050446 @mimaison I've updated the README and fixed a small bug in the integration test recently introduced in https://github.com/apache/kafka/pull/13137. Mind giving one more quick pass? --

[GitHub] [kafka] ijuma commented on a diff in pull request #13040: KAFKA-14480 Move/Rewrite ProducerStateManager to storage module.

2023-02-10 Thread via GitHub
ijuma commented on code in PR #13040: URL: https://github.com/apache/kafka/pull/13040#discussion_r1102975376 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -680,19 +680,23 @@ class UnifiedLog(@volatile var logStartOffset: Long, } private[log] def activeProducer

[GitHub] [kafka] ijuma commented on a diff in pull request #13040: KAFKA-14480 Move/Rewrite ProducerStateManager to storage module.

2023-02-10 Thread via GitHub
ijuma commented on code in PR #13040: URL: https://github.com/apache/kafka/pull/13040#discussion_r1102976553 ## storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateManager.java: ## @@ -0,0 +1,676 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [kafka] satishd commented on a diff in pull request #13040: KAFKA-14480 Move/Rewrite ProducerStateManager to storage module.

2023-02-10 Thread via GitHub
satishd commented on code in PR #13040: URL: https://github.com/apache/kafka/pull/13040#discussion_r1102997944 ## storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateManager.java: ## @@ -0,0 +1,676 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] cmccabe commented on a diff in pull request #13227: KAFKA-14693; Kafka node should halt instead of exit

2023-02-10 Thread via GitHub
cmccabe commented on code in PR #13227: URL: https://github.com/apache/kafka/pull/13227#discussion_r1103072726 ## server-common/src/main/java/org/apache/kafka/server/fault/ProcessTerminatingFaultHandler.java: ## @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] jolshan commented on a diff in pull request #13230: KAFKA-14704; Follower should truncate before incrementing high watermark

2023-02-10 Thread via GitHub
jolshan commented on code in PR #13230: URL: https://github.com/apache/kafka/pull/13230#discussion_r1103074467 ## core/src/main/scala/kafka/server/AbstractFetcherThread.scala: ## @@ -348,33 +348,39 @@ abstract class AbstractFetcherThread(name: String, Errors.forCo

[jira] [Resolved] (KAFKA-14688) Move org.apache.kafka.server.log.internals to org.apache.kafka.storage.internals.log

2023-02-10 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-14688. - Resolution: Fixed > Move org.apache.kafka.server.log.internals to > org.apache.kafka.storage.in

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13188: KAFKA-14491: [5/N] Basic operations for RocksDB versioned store

2023-02-10 Thread via GitHub
vcrfxia commented on code in PR #13188: URL: https://github.com/apache/kafka/pull/13188#discussion_r1103076551 ## streams/src/main/java/org/apache/kafka/streams/state/VersionedKeyValueStore.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] philipnee commented on a diff in pull request #13190: KAFKA-12639: exit upon expired timer to prevent tight looping

2023-02-10 Thread via GitHub
philipnee commented on code in PR #13190: URL: https://github.com/apache/kafka/pull/13190#discussion_r1103108812 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java: ## @@ -501,13 +501,16 @@ boolean joinGroupIfNeeded(final Timer timer) {

[GitHub] [kafka] jolshan commented on a diff in pull request #13230: KAFKA-14704; Follower should truncate before incrementing high watermark

2023-02-10 Thread via GitHub
jolshan commented on code in PR #13230: URL: https://github.com/apache/kafka/pull/13230#discussion_r1103127795 ## core/src/test/scala/unit/kafka/server/AbstractFetcherThreadTest.scala: ## @@ -1089,6 +1089,85 @@ class AbstractFetcherThreadTest { fetcher.verifyLastFetchedEpoc

[GitHub] [kafka] jolshan commented on a diff in pull request #13230: KAFKA-14704; Follower should truncate before incrementing high watermark

2023-02-10 Thread via GitHub
jolshan commented on code in PR #13230: URL: https://github.com/apache/kafka/pull/13230#discussion_r1103129876 ## core/src/test/scala/unit/kafka/server/AbstractFetcherThreadTest.scala: ## @@ -1387,13 +1466,8 @@ class AbstractFetcherThreadTest { val state = replicaPartitio

[GitHub] [kafka] jsancio commented on a diff in pull request #13227: KAFKA-14693; Kafka node should halt instead of exit

2023-02-10 Thread via GitHub
jsancio commented on code in PR #13227: URL: https://github.com/apache/kafka/pull/13227#discussion_r1103137743 ## server-common/src/main/java/org/apache/kafka/server/fault/ProcessTerminatingFaultHandler.java: ## @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13188: KAFKA-14491: [5/N] Basic operations for RocksDB versioned store

2023-02-10 Thread via GitHub
vcrfxia commented on code in PR #13188: URL: https://github.com/apache/kafka/pull/13188#discussion_r1103079190 ## streams/src/test/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreTest.java: ## @@ -0,0 +1,435 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13188: KAFKA-14491: [5/N] Basic operations for RocksDB versioned store

2023-02-10 Thread via GitHub
vcrfxia commented on code in PR #13188: URL: https://github.com/apache/kafka/pull/13188#discussion_r1103141124 ## streams/src/main/java/org/apache/kafka/streams/state/VersionedKeyValueStore.java: ## @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] jolshan opened a new pull request, #13231: KAFKA-14402: Update AddPartitionsToTxn protocol to batch and handle verifyOnly requests

2023-02-10 Thread via GitHub
jolshan opened a new pull request, #13231: URL: https://github.com/apache/kafka/pull/13231 I've updated the API spec and related classes. Clients should only be able to send up to version 3 requests and that is enforced by using the old Builder and forcing the older builder to build

[GitHub] [kafka] jolshan commented on pull request #13229: 3.2

2023-02-10 Thread via GitHub
jolshan commented on PR #13229: URL: https://github.com/apache/kafka/pull/13229#issuecomment-1426235352 Hey -- what is the goal of this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13189: KAFKA-14491: [6/N] Support restoring RocksDB versioned store from changelog

2023-02-10 Thread via GitHub
vcrfxia commented on code in PR #13189: URL: https://github.com/apache/kafka/pull/13189#discussion_r1103148828 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStore.java: ## @@ -0,0 +1,877 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13189: KAFKA-14491: [6/N] Support restoring RocksDB versioned store from changelog

2023-02-10 Thread via GitHub
vcrfxia commented on code in PR #13189: URL: https://github.com/apache/kafka/pull/13189#discussion_r1103149123 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreRestoreWriteBuffer.java: ## @@ -0,0 +1,269 @@ +/* + * Licensed to the Apache Soft

[GitHub] [kafka] philipnee commented on pull request #13225: KAFKA-6793: Unused configuration logging appears to be noisy and unnecessary

2023-02-10 Thread via GitHub
philipnee commented on PR #13225: URL: https://github.com/apache/kafka/pull/13225#issuecomment-1426247418 @hachikuji - info level logging to prevent misleading warn. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] artemlivshits commented on pull request #13078: KAFKA-13999: Add ProducerCount metrics (KIP-847)

2023-02-10 Thread via GitHub
artemlivshits commented on PR #13078: URL: https://github.com/apache/kafka/pull/13078#issuecomment-1426250015 > Which I assume is in "hundreds" in a busy cluster We had cases of OOM due to number of producer ids, so we definitely don't want to increase memory consumption (and should l

[GitHub] [kafka] jolshan commented on a diff in pull request #13196: KAFKA-14673; Add high watermark listener to Partition/Log layers

2023-02-10 Thread via GitHub
jolshan commented on code in PR #13196: URL: https://github.com/apache/kafka/pull/13196#discussion_r1103164836 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -49,6 +49,35 @@ import org.apache.kafka.server.log.internals.{AppendOrigin, FetchDataInfo, Fetch import sc

[GitHub] [kafka] jolshan commented on a diff in pull request #13196: KAFKA-14673; Add high watermark listener to Partition/Log layers

2023-02-10 Thread via GitHub
jolshan commented on code in PR #13196: URL: https://github.com/apache/kafka/pull/13196#discussion_r1103166441 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -49,6 +49,35 @@ import org.apache.kafka.server.log.internals.{AppendOrigin, FetchDataInfo, Fetch import sc

[GitHub] [kafka] jolshan commented on a diff in pull request #13196: KAFKA-14673; Add high watermark listener to Partition/Log layers

2023-02-10 Thread via GitHub
jolshan commented on code in PR #13196: URL: https://github.com/apache/kafka/pull/13196#discussion_r1103166441 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -49,6 +49,35 @@ import org.apache.kafka.server.log.internals.{AppendOrigin, FetchDataInfo, Fetch import sc

[GitHub] [kafka] philipnee commented on pull request #13021: KAFKA-14468: Implement CommitRequestManager to manage the commit and autocommit requests

2023-02-10 Thread via GitHub
philipnee commented on PR #13021: URL: https://github.com/apache/kafka/pull/13021#issuecomment-1426265591 There's still a couple things to be hashed out in the future, in particular 1. how do we plan to structure the subscription state and 2. what are the API calls for the subscription

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13230: KAFKA-14704; Follower should truncate before incrementing high watermark

2023-02-10 Thread via GitHub
rreddy-22 commented on code in PR #13230: URL: https://github.com/apache/kafka/pull/13230#discussion_r1103221020 ## core/src/main/scala/kafka/server/AbstractFetcherThread.scala: ## @@ -348,33 +348,39 @@ abstract class AbstractFetcherThread(name: String, Errors.for

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13230: KAFKA-14704; Follower should truncate before incrementing high watermark

2023-02-10 Thread via GitHub
rreddy-22 commented on code in PR #13230: URL: https://github.com/apache/kafka/pull/13230#discussion_r1103225207 ## core/src/main/scala/kafka/server/AbstractFetcherThread.scala: ## @@ -348,33 +348,39 @@ abstract class AbstractFetcherThread(name: String, Errors.for

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13184: KAFKA-14671: Refactor PredicatedTransformation to not implement Transformation

2023-02-10 Thread via GitHub
gharris1727 commented on code in PR #13184: URL: https://github.com/apache/kafka/pull/13184#discussion_r1103257875 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/ErrorHandlingTaskTest.java: ## @@ -494,15 +500,25 @@ private void expectTopicCreation(String topic

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13184: KAFKA-14671: Refactor PredicatedTransformation to not implement Transformation

2023-02-10 Thread via GitHub
gharris1727 commented on code in PR #13184: URL: https://github.com/apache/kafka/pull/13184#discussion_r1103260370 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/ErrorHandlingTaskTest.java: ## @@ -494,15 +500,25 @@ private void expectTopicCreation(String topic

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13184: KAFKA-14671: Refactor PredicatedTransformation to not implement Transformation

2023-02-10 Thread via GitHub
gharris1727 commented on code in PR #13184: URL: https://github.com/apache/kafka/pull/13184#discussion_r1103287942 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/ConnectorConfigTest.java: ## @@ -147,10 +151,10 @@ public void singleTransform() { props.

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13230: KAFKA-14704; Follower should truncate before incrementing high watermark

2023-02-10 Thread via GitHub
rreddy-22 commented on code in PR #13230: URL: https://github.com/apache/kafka/pull/13230#discussion_r1103225207 ## core/src/main/scala/kafka/server/AbstractFetcherThread.scala: ## @@ -348,33 +348,39 @@ abstract class AbstractFetcherThread(name: String, Errors.for

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13230: KAFKA-14704; Follower should truncate before incrementing high watermark

2023-02-10 Thread via GitHub
rreddy-22 commented on code in PR #13230: URL: https://github.com/apache/kafka/pull/13230#discussion_r1103221020 ## core/src/main/scala/kafka/server/AbstractFetcherThread.scala: ## @@ -348,33 +348,39 @@ abstract class AbstractFetcherThread(name: String, Errors.for

[GitHub] [kafka] mjsax commented on a diff in pull request #13188: KAFKA-14491: [5/N] Basic operations for RocksDB versioned store

2023-02-10 Thread via GitHub
mjsax commented on code in PR #13188: URL: https://github.com/apache/kafka/pull/13188#discussion_r1102242415 ## streams/src/test/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreTest.java: ## @@ -0,0 +1,435 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13184: KAFKA-14671: Refactor PredicatedTransformation to not implement Transformation

2023-02-10 Thread via GitHub
gharris1727 commented on code in PR #13184: URL: https://github.com/apache/kafka/pull/13184#discussion_r1103295046 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/ConnectorConfig.java: ## @@ -270,28 +270,28 @@ public boolean includeRecordDetailsInErrorLog() {

[GitHub] [kafka] mjsax commented on a diff in pull request #13189: KAFKA-14491: [6/N] Support restoring RocksDB versioned store from changelog

2023-02-10 Thread via GitHub
mjsax commented on code in PR #13189: URL: https://github.com/apache/kafka/pull/13189#discussion_r1103303535 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStore.java: ## @@ -0,0 +1,877 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] cmccabe merged pull request #13226: MINOR: Remove references to HIGHEST_SUPPORTED_VERSION from ZkMigrationClient

2023-02-10 Thread via GitHub
cmccabe merged PR #13226: URL: https://github.com/apache/kafka/pull/13226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] guozhangwang commented on a diff in pull request #13189: KAFKA-14491: [6/N] Support restoring RocksDB versioned store from changelog

2023-02-10 Thread via GitHub
guozhangwang commented on code in PR #13189: URL: https://github.com/apache/kafka/pull/13189#discussion_r1103405393 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStore.java: ## @@ -0,0 +1,877 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] hachikuji commented on a diff in pull request #13230: KAFKA-14704; Follower should truncate before incrementing high watermark

2023-02-10 Thread via GitHub
hachikuji commented on code in PR #13230: URL: https://github.com/apache/kafka/pull/13230#discussion_r1103426128 ## core/src/test/scala/unit/kafka/server/AbstractFetcherThreadTest.scala: ## @@ -1089,6 +1089,85 @@ class AbstractFetcherThreadTest { fetcher.verifyLastFetchedEp

[GitHub] [kafka] hachikuji commented on a diff in pull request #13230: KAFKA-14704; Follower should truncate before incrementing high watermark

2023-02-10 Thread via GitHub
hachikuji commented on code in PR #13230: URL: https://github.com/apache/kafka/pull/13230#discussion_r1103436599 ## core/src/main/scala/kafka/server/AbstractFetcherThread.scala: ## @@ -348,33 +348,39 @@ abstract class AbstractFetcherThread(name: String, Errors.for

[GitHub] [kafka] hachikuji commented on a diff in pull request #13227: KAFKA-14693; Kafka node should halt instead of exit

2023-02-10 Thread via GitHub
hachikuji commented on code in PR #13227: URL: https://github.com/apache/kafka/pull/13227#discussion_r1103438546 ## core/src/main/scala/kafka/server/SharedServer.scala: ## @@ -60,7 +60,10 @@ class StandardFaultHandlerFactory extends FaultHandlerFactory { action: Runnable

[GitHub] [kafka] hachikuji commented on a diff in pull request #13227: KAFKA-14693; Kafka node should halt instead of exit

2023-02-10 Thread via GitHub
hachikuji commented on code in PR #13227: URL: https://github.com/apache/kafka/pull/13227#discussion_r1103438546 ## core/src/main/scala/kafka/server/SharedServer.scala: ## @@ -60,7 +60,10 @@ class StandardFaultHandlerFactory extends FaultHandlerFactory { action: Runnable

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13208: KAFKA-5756: Wait for concurrent offset flush to complete before starting next flush

2023-02-10 Thread via GitHub
gharris1727 commented on code in PR #13208: URL: https://github.com/apache/kafka/pull/13208#discussion_r1103447933 ## connect/runtime/src/main/java/org/apache/kafka/connect/storage/OffsetStorageWriter.java: ## @@ -98,6 +104,24 @@ private boolean flushing() { return toFl

[GitHub] [kafka] mjsax merged pull request #13188: KAFKA-14491: [5/N] Basic operations for RocksDB versioned store

2023-02-10 Thread via GitHub
mjsax merged PR #13188: URL: https://github.com/apache/kafka/pull/13188 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] gharris1727 commented on pull request #13178: KAFKA-12468, KAFKA-13659, KAFKA-12566: Fix MM2 causing negative downstream lag

2023-02-10 Thread via GitHub
gharris1727 commented on PR #13178: URL: https://github.com/apache/kafka/pull/13178#issuecomment-1426559007 I think my commit to synchronize the OffsetSyncStore introduced a dramatic performance regression that is causing the current test failures. Where previously the test would comfortabl

[GitHub] [kafka] dajac commented on pull request #13195: Minor: Add JmxTool note to 3.5.0 notable changes

2023-02-10 Thread via GitHub
dajac commented on PR #13195: URL: https://github.com/apache/kafka/pull/13195#issuecomment-1426647094 I am not aware of any KIP regarding the migration of the tools from core module to tools module. Having a small one for those problematic tools is not a bad idea. For the context, pa

[GitHub] [kafka] ijuma commented on pull request #13195: Minor: Add JmxTool note to 3.5.0 notable changes

2023-02-10 Thread via GitHub
ijuma commented on PR #13195: URL: https://github.com/apache/kafka/pull/13195#issuecomment-1426647425 @dajac well, that depends on whether it's considered public api. My position is that the only API is the script. If there is no script, it's not public API. How would users even know about

[GitHub] [kafka] dajac commented on a diff in pull request #13214: KAFKA-14577: Move the scala ConsoleProducer from core to tools module

2023-02-10 Thread via GitHub
dajac commented on code in PR #13214: URL: https://github.com/apache/kafka/pull/13214#discussion_r1103557466 ## tools/src/main/java/org/apache/kafka/tools/MessageReader.java: ## @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contr

[GitHub] [kafka] dajac commented on a diff in pull request #13214: KAFKA-14577: Move the scala ConsoleProducer from core to tools module

2023-02-10 Thread via GitHub
dajac commented on code in PR #13214: URL: https://github.com/apache/kafka/pull/13214#discussion_r1103557897 ## tools/src/main/java/org/apache/kafka/tools/MessageReader.java: ## @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contr

[GitHub] [kafka] dajac commented on pull request #13195: Minor: Add JmxTool note to 3.5.0 notable changes

2023-02-10 Thread via GitHub
dajac commented on PR #13195: URL: https://github.com/apache/kafka/pull/13195#issuecomment-1426652327 Yeah, it is definitely a gray area. I don’t think that those tools are documented in the Kafka documentation. Somehow, JmxTool is known and you can find references to it on Stackoverflow an

[GitHub] [kafka] dajac commented on pull request #13195: Minor: Add JmxTool note to 3.5.0 notable changes

2023-02-10 Thread via GitHub
dajac commented on PR #13195: URL: https://github.com/apache/kafka/pull/13195#issuecomment-1426652393 By the way, there is another case here: https://github.com/apache/kafka/pull/13214. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t