[jira] [Resolved] (KAFKA-14254) Format timestamps in assignor logs as dates instead of integers

2023-02-14 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-14254. --- Fix Version/s: 3.4.0 Resolution: Fixed > Format timestamps in assignor logs as dates

[GitHub] [kafka] showuon commented on a diff in pull request #13100: MINOR: add size check for tagged fields

2023-02-14 Thread via GitHub
showuon commented on code in PR #13100: URL: https://github.com/apache/kafka/pull/13100#discussion_r1105435108 ## clients/src/main/java/org/apache/kafka/common/protocol/types/TaggedFields.java: ## @@ -100,6 +100,13 @@ public NavigableMap read(ByteBuffer buffer) { }

[jira] [Commented] (KAFKA-14597) [Streams] record-e2e-latency-max is not reporting correct metrics

2023-02-14 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17688362#comment-17688362 ] Bruno Cadonna commented on KAFKA-14597: --- [~talestonini] Thank you for the ticket!

[GitHub] [kafka] showuon opened a new pull request, #13244: KAFKA-14495: assert the cache size for each operation

2023-02-14 Thread via GitHub
showuon opened a new pull request, #13244: URL: https://github.com/apache/kafka/pull/13244 Improve the test verification to make sure the cache size is expected before and after each call. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation

[GitHub] [kafka] showuon commented on pull request #13244: KAFKA-14495: assert the cache size for each operation

2023-02-14 Thread via GitHub
showuon commented on PR #13244: URL: https://github.com/apache/kafka/pull/13244#issuecomment-1429346498 @satishd , call for review. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [kafka] dajac merged pull request #13230: KAFKA-14704; Follower should truncate before incrementing high watermark

2023-02-14 Thread via GitHub
dajac merged PR #13230: URL: https://github.com/apache/kafka/pull/13230 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] dajac commented on pull request #13230: KAFKA-14704; Follower should truncate before incrementing high watermark

2023-02-14 Thread via GitHub
dajac commented on PR #13230: URL: https://github.com/apache/kafka/pull/13230#issuecomment-1429361939 Merged to trunk, 3.4 and 3.3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[jira] [Resolved] (KAFKA-14704) Follower should truncate before incrementing high watermark

2023-02-14 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-14704. - Fix Version/s: 3.5.0 3.4.1 3.3.3 Reviewer: Jason G

[GitHub] [kafka] showuon commented on a diff in pull request #13239: MINOR: Various cleanups in clients javadoc

2023-02-14 Thread via GitHub
showuon commented on code in PR #13239: URL: https://github.com/apache/kafka/pull/13239#discussion_r1105504621 ## clients/src/main/java/org/apache/kafka/common/KafkaFuture.java: ## @@ -127,8 +127,6 @@ public static KafkaFuture allOf(KafkaFuture... futures) { public abstrac

[GitHub] [kafka] Hangleton commented on a diff in pull request #13078: KAFKA-13999: Add ProducerCount metrics (KIP-847)

2023-02-14 Thread via GitHub
Hangleton commented on code in PR #13078: URL: https://github.com/apache/kafka/pull/13078#discussion_r1105509459 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -264,6 +265,8 @@ class ReplicaManager(val config: KafkaConfig, leaderPartitionsIterator.count(_.h

[GitHub] [kafka] mimaison commented on a diff in pull request #13239: MINOR: Various cleanups in clients javadoc

2023-02-14 Thread via GitHub
mimaison commented on code in PR #13239: URL: https://github.com/apache/kafka/pull/13239#discussion_r1105515715 ## clients/src/main/java/org/apache/kafka/common/KafkaFuture.java: ## @@ -127,8 +127,6 @@ public static KafkaFuture allOf(KafkaFuture... futures) { public abstra

[GitHub] [kafka] mimaison merged pull request #13239: MINOR: Various cleanups in clients javadoc

2023-02-14 Thread via GitHub
mimaison merged PR #13239: URL: https://github.com/apache/kafka/pull/13239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] dajac opened a new pull request, #13245: KAFKA-14704; Follower should truncate before incrementing high watermark (#13230)

2023-02-14 Thread via GitHub
dajac opened a new pull request, #13245: URL: https://github.com/apache/kafka/pull/13245 Cherry-picked from bd32d98bd7cbd8cff35d6786bc8b00061e0fedf7. When a leader becomes a follower, it is likely that it has uncommitted records in its log. When it reaches out to the leader, the leade

[GitHub] [kafka] bachmanity1 opened a new pull request, #13246: MINOR: Remove useless array size check

2023-02-14 Thread via GitHub
bachmanity1 opened a new pull request, #13246: URL: https://github.com/apache/kafka/pull/13246 ``` if (size > buffer.remaining()) throw new SchemaException("Error reading array of size " + size + ", only " + buffer.remaining() + " bytes available"); ``` Array type has the ab

[GitHub] [kafka] bachmanity1 commented on pull request #13246: MINOR: Remove useless array size check

2023-02-14 Thread via GitHub
bachmanity1 commented on PR #13246: URL: https://github.com/apache/kafka/pull/13246#issuecomment-1429404900 @cmccabe @dongjinleekr could you please have a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [kafka] armandfargeon commented on pull request #6193: [WIP] KIP-81 Bound Fetch memory usage in the consumer

2023-02-14 Thread via GitHub
armandfargeon commented on PR #6193: URL: https://github.com/apache/kafka/pull/6193#issuecomment-1429444823 Hello @mimaison, It is planned to add this PR in a future version ? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] nizhikov opened a new pull request, #13247: [WIP] KAFKA-14595 Move PartitionReassignmentState to java

2023-02-14 Thread via GitHub
nizhikov opened a new pull request, #13247: URL: https://github.com/apache/kafka/pull/13247 This PR contains changes required to move `PartitionReassignmentState` class to java code. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation -

[jira] [Commented] (KAFKA-14595) Move ReassignPartitionsCommand to tools

2023-02-14 Thread Nikolay Izhikov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17688429#comment-17688429 ] Nikolay Izhikov commented on KAFKA-14595: - Hello To reduce changes and make the

[GitHub] [kafka] cadonna commented on a diff in pull request #13025: KAFKA-14299: Fix pause and resume with state updater

2023-02-14 Thread via GitHub
cadonna commented on code in PR #13025: URL: https://github.com/apache/kafka/pull/13025#discussion_r1105612105 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/ReadOnlyTask.java: ## @@ -190,7 +190,7 @@ public void clearTaskTimeout() { @Override p

[GitHub] [kafka] cadonna commented on a diff in pull request #13127: KAFKA-14586: Moving StreamResetter to tools

2023-02-14 Thread via GitHub
cadonna commented on code in PR #13127: URL: https://github.com/apache/kafka/pull/13127#discussion_r1105688687 ## streams/src/test/java/org/apache/kafka/streams/tools/StreamsResetterTest.java: ## @@ -16,7 +16,7 @@ */ package org.apache.kafka.streams.tools; -import kafka.too

[GitHub] [kafka] cadonna commented on a diff in pull request #13127: KAFKA-14586: Moving StreamResetter to tools

2023-02-14 Thread via GitHub
cadonna commented on code in PR #13127: URL: https://github.com/apache/kafka/pull/13127#discussion_r1105688687 ## streams/src/test/java/org/apache/kafka/streams/tools/StreamsResetterTest.java: ## @@ -16,7 +16,7 @@ */ package org.apache.kafka.streams.tools; -import kafka.too

[jira] [Created] (KAFKA-14713) Kafka Streams global table startup takes too long

2023-02-14 Thread Tamas (Jira)
Tamas created KAFKA-14713: - Summary: Kafka Streams global table startup takes too long Key: KAFKA-14713 URL: https://issues.apache.org/jira/browse/KAFKA-14713 Project: Kafka Issue Type: Bug

[GitHub] [kafka] satishd commented on pull request #13235: MINOR Added assertion checks in tests for ProducerStateManager.active Producers usages.

2023-02-14 Thread via GitHub
satishd commented on PR #13235: URL: https://github.com/apache/kafka/pull/13235#issuecomment-1429693785 Thanks @showuon for your review. > Overall LGTM! But do you think we should add a null `producerStateEntry` test case? Can you be more specific about the test case you are r

[jira] [Commented] (KAFKA-14595) Move ReassignPartitionsCommand to tools

2023-02-14 Thread Nikolay Izhikov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17688492#comment-17688492 ] Nikolay Izhikov commented on KAFKA-14595: - > I move the `PartitionReassignmentSt

[jira] [Created] (KAFKA-14714) Move/Rewrite RollParams, LogAppendInfo, and LeaderHwChange to storage module.

2023-02-14 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-14714: -- Summary: Move/Rewrite RollParams, LogAppendInfo, and LeaderHwChange to storage module. Key: KAFKA-14714 URL: https://issues.apache.org/jira/browse/KAFKA-14714 Pro

[GitHub] [kafka] dajac commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to OffsetCommit API and propagate for request version >= 9

2023-02-14 Thread via GitHub
dajac commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1105846605 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -425,35 +425,72 @@ class KafkaApis(val requestChannel: RequestChannel, requestHelper.sendMaybeThrottle(r

[GitHub] [kafka] dajac commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to OffsetCommit API and propagate for request version >= 9

2023-02-14 Thread via GitHub
dajac commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1105854122 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -425,35 +425,72 @@ class KafkaApis(val requestChannel: RequestChannel, requestHelper.sendMaybeThrottle(r

[GitHub] [kafka] dajac commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to OffsetCommit API and propagate for request version >= 9

2023-02-14 Thread via GitHub
dajac commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1105855843 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -425,35 +425,72 @@ class KafkaApis(val requestChannel: RequestChannel, requestHelper.sendMaybeThrottle(r

[GitHub] [kafka] nizhikov commented on pull request #13247: [WIP] KAFKA-14595 Move value object of ReassignPartitionsCommand to java

2023-02-14 Thread via GitHub
nizhikov commented on PR #13247: URL: https://github.com/apache/kafka/pull/13247#issuecomment-1429798487 @mimaison This PR is a first part of moving `ReassignPartitionsCommand` to java and `tools` module. Can you, please, take a look? -- This is an automated message from the Ap

[jira] [Updated] (KAFKA-14696) CVE-2023-25194: Apache Kafka: Possible RCE/Denial of service attack via SASL JAAS JndiLoginModule configuration using Kafka Connect

2023-02-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison updated KAFKA-14696: --- Fix Version/s: (was: 3.4.0) > CVE-2023-25194: Apache Kafka: Possible RCE/Denial of servi

[jira] [Updated] (KAFKA-14696) CVE-2023-25194: Apache Kafka: Possible RCE/Denial of service attack via SASL JAAS JndiLoginModule configuration using Kafka Connect

2023-02-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison updated KAFKA-14696: --- Fix Version/s: 3.4.0 > CVE-2023-25194: Apache Kafka: Possible RCE/Denial of service attack v

[jira] [Resolved] (KAFKA-14696) CVE-2023-25194: Apache Kafka: Possible RCE/Denial of service attack via SASL JAAS JndiLoginModule configuration using Kafka Connect

2023-02-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-14696. Resolution: Fixed > CVE-2023-25194: Apache Kafka: Possible RCE/Denial of service attack vi

[jira] [Assigned] (KAFKA-14225) lazy val exemptSensor Could Cause Deadlock

2023-02-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison reassigned KAFKA-14225: -- Assignee: Huilin Shi > lazy val exemptSensor Could Cause Deadlock > -

[jira] [Assigned] (KAFKA-14236) ListGroups request produces too much Denied logs in authorizer

2023-02-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison reassigned KAFKA-14236: -- Assignee: Alexandre GRIFFAUT > ListGroups request produces too much Denied logs in au

[jira] [Assigned] (KAFKA-14646) SubscriptionWrapper is of an incompatible version (Kafka Streams 3.2.3 -> 3.3.2)

2023-02-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison reassigned KAFKA-14646: -- Assignee: Matthias J. Sax > SubscriptionWrapper is of an incompatible version (Kafka

[jira] [Assigned] (KAFKA-13632) MirrorMaker 2.0 NPE and Warning "Failure to commit records" for filtered records

2023-02-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison reassigned KAFKA-13632: -- Assignee: Rens Groothuijsen > MirrorMaker 2.0 NPE and Warning "Failure to commit reco

[jira] [Assigned] (KAFKA-14564) Upgrade Netty to 4.1.86.Final to fix CVEs

2023-02-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison reassigned KAFKA-14564: -- Assignee: Brendan Ribera > Upgrade Netty to 4.1.86.Final to fix CVEs > --

[jira] [Assigned] (KAFKA-14570) Problem description missing closing parenthesis symbol

2023-02-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison reassigned KAFKA-14570: -- Assignee: iamazy > Problem description missing closing parenthesis symbol > -

[jira] [Assigned] (KAFKA-14644) Process should stop after failure in raft IO thread

2023-02-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison reassigned KAFKA-14644: -- Assignee: Jason Gustafson > Process should stop after failure in raft IO thread > ---

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to OffsetCommit API and propagate for request version >= 9

2023-02-14 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1105985661 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -425,35 +425,72 @@ class KafkaApis(val requestChannel: RequestChannel, requestHelper.sendMaybeThrott

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to OffsetCommit API and propagate for request version >= 9

2023-02-14 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1105987099 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -425,35 +425,72 @@ class KafkaApis(val requestChannel: RequestChannel, requestHelper.sendMaybeThrott

[GitHub] [kafka] Hangleton commented on pull request #13240: KAFKA-14690: Add topic IDs to OffsetCommit API and propagate for request version >= 9

2023-02-14 Thread via GitHub
Hangleton commented on PR #13240: URL: https://github.com/apache/kafka/pull/13240#issuecomment-1429942230 Thanks for the review, David. I am working on adding unit tests for `OffsetCommitResponse` and the server-side handling of the request/response, and fix the bugs you have identified.

[jira] [Created] (KAFKA-14715) Reduce Fetcher#parseRecord() memory copy

2023-02-14 Thread Mickael Maison (Jira)
Mickael Maison created KAFKA-14715: -- Summary: Reduce Fetcher#parseRecord() memory copy Key: KAFKA-14715 URL: https://issues.apache.org/jira/browse/KAFKA-14715 Project: Kafka Issue Type: Impr

[jira] [Updated] (KAFKA-14395) Add config to configure client supplier for KafkaStreams

2023-02-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison updated KAFKA-14395: --- Fix Version/s: 3.5.0 > Add config to configure client supplier for KafkaStreams > --

[jira] [Commented] (KAFKA-13152) Replace "buffered.records.per.partition" & "cache.max.bytes.buffering" with "{statestore.cache}/{input.buffer}.max.bytes"

2023-02-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17688565#comment-17688565 ] Mickael Maison commented on KAFKA-13152: Is KIP-770 still in progress? If so do

[GitHub] [kafka] urbandan commented on pull request #13163: KAFKA-14653: MirrorMakerConfig using raw properties instead of post-r…

2023-02-14 Thread via GitHub
urbandan commented on PR #13163: URL: https://github.com/apache/kafka/pull/13163#issuecomment-1429992202 @C0urante Existing tests (testWorkerConfigs and testClientConfigProperties) already cover the config provider resolution (they use ${fake:secret:password} and FakeConfigProvider), I don'

[GitHub] [kafka] urbandan commented on pull request #13163: KAFKA-14653: MirrorMakerConfig using raw properties instead of post-r…

2023-02-14 Thread via GitHub
urbandan commented on PR #13163: URL: https://github.com/apache/kafka/pull/13163#issuecomment-1429994157 Rebased the change on trunk and resolved conflicts -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Updated] (KAFKA-14578) Move ConsumerPerformance to tools

2023-02-14 Thread Federico Valeri (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Federico Valeri updated KAFKA-14578: Fix Version/s: 3.5.0 > Move ConsumerPerformance to tools > ---

[GitHub] [kafka] philipnee commented on a diff in pull request #13238: KAFKA-14708: Use Java thread instead of kafka library for example purpose

2023-02-14 Thread via GitHub
philipnee commented on code in PR #13238: URL: https://github.com/apache/kafka/pull/13238#discussion_r1106052765 ## examples/src/main/java/kafka/examples/Consumer.java: ## @@ -63,33 +63,36 @@ public Consumer(final String topic, this.numMessageToConsume = numMessageToCon

[GitHub] [kafka] C0urante commented on a diff in pull request #13178: KAFKA-12468, KAFKA-13659, KAFKA-12566: Fix MM2 causing negative downstream lag

2023-02-14 Thread via GitHub
C0urante commented on code in PR #13178: URL: https://github.com/apache/kafka/pull/13178#discussion_r1106023715 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorCheckpointTask.java: ## @@ -134,9 +138,9 @@ public String version() { @Override public

[GitHub] [kafka] philipnee commented on a diff in pull request #13238: KAFKA-14708: Use Java thread instead of kafka library for example purpose

2023-02-14 Thread via GitHub
philipnee commented on code in PR #13238: URL: https://github.com/apache/kafka/pull/13238#discussion_r1106061755 ## examples/src/main/java/kafka/examples/Consumer.java: ## @@ -63,33 +63,36 @@ public Consumer(final String topic, this.numMessageToConsume = numMessageToCon

[jira] [Created] (KAFKA-14716) Connect schema does not allow struct default values

2023-02-14 Thread Daniel Urban (Jira)
Daniel Urban created KAFKA-14716: Summary: Connect schema does not allow struct default values Key: KAFKA-14716 URL: https://issues.apache.org/jira/browse/KAFKA-14716 Project: Kafka Issue Typ

[jira] [Updated] (KAFKA-14716) Connect schema does not allow struct default values

2023-02-14 Thread Daniel Urban (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Urban updated KAFKA-14716: - Description: The ConnectSchema API should allow specifying a composite (struct) default value f

[GitHub] [kafka] jolshan commented on pull request #13078: KAFKA-13999: Add ProducerCount metrics (KIP-847)

2023-02-14 Thread via GitHub
jolshan commented on PR #13078: URL: https://github.com/apache/kafka/pull/13078#issuecomment-1430066349 @anatasiavela do you mind rebasing again 😅 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] C0urante commented on pull request #13163: KAFKA-14653: MirrorMakerConfig using raw properties instead of post-r…

2023-02-14 Thread via GitHub
C0urante commented on PR #13163: URL: https://github.com/apache/kafka/pull/13163#issuecomment-1430077887 @urbandan Ah yes, apologies for missing that. You're right, we don't need any more coverage for the `MirrorMakerConfig` class. I also agree that the manual config transformations p

[GitHub] [kafka] urbandan commented on pull request #13163: KAFKA-14653: MirrorMakerConfig using raw properties instead of post-r…

2023-02-14 Thread via GitHub
urbandan commented on PR #13163: URL: https://github.com/apache/kafka/pull/13163#issuecomment-1430082523 Thanks for the reviews @C0urante! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [kafka] hachikuji merged pull request #13242: MINOR: Better logging to distinguish clean vs unclean loading times

2023-02-14 Thread via GitHub
hachikuji merged PR #13242: URL: https://github.com/apache/kafka/pull/13242 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apach

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13178: KAFKA-12468, KAFKA-13659, KAFKA-12566: Fix MM2 causing negative downstream lag

2023-02-14 Thread via GitHub
gharris1727 commented on code in PR #13178: URL: https://github.com/apache/kafka/pull/13178#discussion_r1106151042 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorCheckpointTask.java: ## @@ -134,9 +138,9 @@ public String version() { @Override publ

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13178: KAFKA-12468, KAFKA-13659, KAFKA-12566: Fix MM2 causing negative downstream lag

2023-02-14 Thread via GitHub
gharris1727 commented on code in PR #13178: URL: https://github.com/apache/kafka/pull/13178#discussion_r1106151656 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/OffsetSyncStore.java: ## @@ -16,40 +16,88 @@ */ package org.apache.kafka.connect.mirror; -impor

[GitHub] [kafka] jsancio merged pull request #13227: KAFKA-14693; Kafka node should halt instead of exit

2023-02-14 Thread via GitHub
jsancio merged PR #13227: URL: https://github.com/apache/kafka/pull/13227 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Resolved] (KAFKA-14693) KRaft Controller and ProcessExitingFaultHandler can deadlock shutdown

2023-02-14 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] José Armando García Sancio resolved KAFKA-14693. Resolution: Fixed > KRaft Controller and ProcessExitingFaultHandle

[jira] [Updated] (KAFKA-14693) KRaft Controller and ProcessExitingFaultHandler can deadlock shutdown

2023-02-14 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] José Armando García Sancio updated KAFKA-14693: --- Fix Version/s: 3.5.0 > KRaft Controller and ProcessExitingFaultHandl

[jira] [Commented] (KAFKA-14716) Connect schema does not allow struct default values

2023-02-14 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17688651#comment-17688651 ] Chris Egerton commented on KAFKA-14716: --- [~durban] is this a duplicate of KAFKA-12

[GitHub] [kafka] C0urante commented on a diff in pull request #13208: KAFKA-5756: Wait for concurrent offset flush to complete before starting next flush

2023-02-14 Thread via GitHub
C0urante commented on code in PR #13208: URL: https://github.com/apache/kafka/pull/13208#discussion_r1106203170 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/ExactlyOnceWorkerSourceTask.java: ## @@ -256,14 +257,25 @@ private void maybeBeginTransaction() {

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to OffsetCommit API and propagate for request version >= 9

2023-02-14 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1106229439 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -425,35 +425,72 @@ class KafkaApis(val requestChannel: RequestChannel, requestHelper.sendMaybeThrott

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to OffsetCommit API and propagate for request version >= 9

2023-02-14 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1106229439 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -425,35 +425,72 @@ class KafkaApis(val requestChannel: RequestChannel, requestHelper.sendMaybeThrott

[GitHub] [kafka] C0urante commented on a diff in pull request #13208: KAFKA-5756: Wait for concurrent offset flush to complete before starting next flush

2023-02-14 Thread via GitHub
C0urante commented on code in PR #13208: URL: https://github.com/apache/kafka/pull/13208#discussion_r1106236517 ## connect/runtime/src/test/java/org/apache/kafka/connect/storage/OffsetStorageWriterTest.java: ## @@ -135,38 +135,40 @@ public void testFlushFailureReplacesOffsets()

[GitHub] [kafka] C0urante commented on a diff in pull request #13208: KAFKA-5756: Wait for concurrent offset flush to complete before starting next flush

2023-02-14 Thread via GitHub
C0urante commented on code in PR #13208: URL: https://github.com/apache/kafka/pull/13208#discussion_r1106250177 ## connect/runtime/src/main/java/org/apache/kafka/connect/storage/OffsetStorageWriter.java: ## @@ -98,6 +104,24 @@ private boolean flushing() { return toFlush

[GitHub] [kafka] guozhangwang commented on a diff in pull request #13082: MINOR: Clarify docs for Streams config max.warmup.replicas.

2023-02-14 Thread via GitHub
guozhangwang commented on code in PR #13082: URL: https://github.com/apache/kafka/pull/13082#discussion_r1106267030 ## docs/streams/developer-guide/config-streams.html: ## @@ -778,10 +778,20 @@ rack.aware.assignment.tagsmax.warmup.replicas -

[jira] [Commented] (KAFKA-14713) Kafka Streams global table startup takes too long

2023-02-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17688679#comment-17688679 ] Matthias J. Sax commented on KAFKA-14713: - Sounds like a duplicate to https://is

[jira] [Commented] (KAFKA-13152) Replace "buffered.records.per.partition" & "cache.max.bytes.buffering" with "{statestore.cache}/{input.buffer}.max.bytes"

2023-02-14 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17688684#comment-17688684 ] Guozhang Wang commented on KAFKA-13152: --- Thanks [~mimaison], this is still quite r

[GitHub] [kafka] guozhangwang commented on a diff in pull request #13025: KAFKA-14299: Fix pause and resume with state updater

2023-02-14 Thread via GitHub
guozhangwang commented on code in PR #13025: URL: https://github.com/apache/kafka/pull/13025#discussion_r1102201605 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/DefaultStateUpdater.java: ## @@ -258,21 +269,27 @@ private List getTasksAndActions() {

[jira] [Created] (KAFKA-14717) KafkaStreams can' get running if the rebalance happens before StreamThread gets shutdown completely

2023-02-14 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-14717: -- Summary: KafkaStreams can' get running if the rebalance happens before StreamThread gets shutdown completely Key: KAFKA-14717 URL: https://issues.apache.org/jira/browse/KAFKA-

[GitHub] [kafka] guozhangwang commented on pull request #13025: KAFKA-14299: Fix pause and resume with state updater

2023-02-14 Thread via GitHub
guozhangwang commented on PR #13025: URL: https://github.com/apache/kafka/pull/13025#issuecomment-1430280435 @lucasbru seems I forgot to send those pending comments from last week, sorry... anyways there's only one comment that I had here: https://github.com/apache/kafka/pull/13025#discussi

[GitHub] [kafka] chia7712 opened a new pull request, #13248: KAFKA-14717 KafkaStreams can' get running if the rebalance happens be…

2023-02-14 Thread via GitHub
chia7712 opened a new pull request, #13248: URL: https://github.com/apache/kafka/pull/13248 I noticed this issue when tracing #12590 StreamThread closes the consumer before changing state to DEAD. If the partition rebalance happens quickly, the other StreamThreads can't change KafkaStrea

[GitHub] [kafka] mattwong949 commented on pull request #13206: [KAFKA-14685] Refactor logic to handle OFFSET_MOVED_TO_TIERED_STORAGE error

2023-02-14 Thread via GitHub
mattwong949 commented on PR #13206: URL: https://github.com/apache/kafka/pull/13206#issuecomment-1430302196 sorry for the delay on any open comments, I'll be able to reply to and iterate on them by tomorrow -- This is an automated message from the Apache Git Service. To respond to the mes

[jira] [Commented] (KAFKA-13152) Replace "buffered.records.per.partition" & "cache.max.bytes.buffering" with "{statestore.cache}/{input.buffer}.max.bytes"

2023-02-14 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17688701#comment-17688701 ] Mickael Maison commented on KAFKA-13152: Ok, I've added it to the release plan.

[GitHub] [kafka] C0urante commented on pull request #13163: KAFKA-14653: MirrorMakerConfig using raw properties instead of post-r…

2023-02-14 Thread via GitHub
C0urante commented on PR #13163: URL: https://github.com/apache/kafka/pull/13163#issuecomment-1430408893 Test failures appear unrelated (the newly-introduced dedicated MM2 integration tests seem to be flaking out on Jenkins; filed KAFKA-14718 to track). Merging... -- This is an automated

[jira] [Created] (KAFKA-14718) Flaky DedicatedMirrorIntegrationTest test suite

2023-02-14 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-14718: - Summary: Flaky DedicatedMirrorIntegrationTest test suite Key: KAFKA-14718 URL: https://issues.apache.org/jira/browse/KAFKA-14718 Project: Kafka Issue Type:

[GitHub] [kafka] C0urante merged pull request #13163: KAFKA-14653: MirrorMakerConfig using raw properties instead of post-r…

2023-02-14 Thread via GitHub
C0urante merged PR #13163: URL: https://github.com/apache/kafka/pull/13163 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Updated] (KAFKA-14653) MM2 should delay resolving config provider references

2023-02-14 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-14653: -- Fix Version/s: 3.5.0 > MM2 should delay resolving config provider references > ---

[jira] [Resolved] (KAFKA-10586) Full support for distributed mode in dedicated MirrorMaker 2.0 clusters

2023-02-14 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-10586. --- Resolution: Done > Full support for distributed mode in dedicated MirrorMaker 2.0 clusters >

[jira] [Updated] (KAFKA-10586) Full support for distributed mode in dedicated MirrorMaker 2.0 clusters

2023-02-14 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-10586: -- Fix Version/s: 3.5.0 > Full support for distributed mode in dedicated MirrorMaker 2.0 clusters

[jira] [Resolved] (KAFKA-10857) Mirror Maker 2 - replication not working when deploying multiple instances

2023-02-14 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-10857. --- Resolution: Duplicate > Mirror Maker 2 - replication not working when deploying multiple ins

[jira] [Resolved] (KAFKA-12893) MM2 fails to replicate if starting two+ nodes same time

2023-02-14 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-12893. --- Resolution: Duplicate > MM2 fails to replicate if starting two+ nodes same time > --

[jira] [Resolved] (KAFKA-12150) Consumer group refresh not working with clustered MM2 setup

2023-02-14 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-12150. --- Resolution: Duplicate > Consumer group refresh not working with clustered MM2 setup > --

[GitHub] [kafka] guozhangwang commented on a diff in pull request #13248: KAFKA-14717 KafkaStreams can' get running if the rebalance happens be…

2023-02-14 Thread via GitHub
guozhangwang commented on code in PR #13248: URL: https://github.com/apache/kafka/pull/13248#discussion_r1106408656 ## streams/src/test/java/org/apache/kafka/streams/integration/AdjustStreamThreadCountTest.java: ## @@ -257,6 +258,23 @@ private Thread adjustCountHelperThread(fina

[GitHub] [kafka] hachikuji commented on a diff in pull request #13192: KAFKA-14675: Extract metadata-related tasks from Fetcher into MetadataFetcher 1/4

2023-02-14 Thread via GitHub
hachikuji commented on code in PR #13192: URL: https://github.com/apache/kafka/pull/13192#discussion_r1106464011 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MetadataFetcher.java: ## @@ -0,0 +1,805 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] [kafka] hachikuji commented on a diff in pull request #13192: KAFKA-14675: Extract metadata-related tasks from Fetcher into MetadataFetcher 1/4

2023-02-14 Thread via GitHub
hachikuji commented on code in PR #13192: URL: https://github.com/apache/kafka/pull/13192#discussion_r1106467125 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MetadataFetcher.java: ## @@ -0,0 +1,805 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] [kafka] jolshan commented on pull request #12957: MINOR: Fix flaky testClientDisconnectionUpdatesRequestMetrics() (#11987)

2023-02-14 Thread via GitHub
jolshan commented on PR #12957: URL: https://github.com/apache/kafka/pull/12957#issuecomment-1430506039 Is this an issue only on 3.2? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] kirktrue commented on a diff in pull request #13192: KAFKA-14675: Extract metadata-related tasks from Fetcher into MetadataFetcher 1/4

2023-02-14 Thread via GitHub
kirktrue commented on code in PR #13192: URL: https://github.com/apache/kafka/pull/13192#discussion_r1106476284 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MetadataFetcher.java: ## @@ -0,0 +1,805 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[jira] [Updated] (KAFKA-14717) KafkaStreams can' get running if the rebalance happens before StreamThread gets shutdown completely

2023-02-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14717: Component/s: streams > KafkaStreams can' get running if the rebalance happens before Strea

[GitHub] [kafka] kirktrue commented on a diff in pull request #13192: KAFKA-14675: Extract metadata-related tasks from Fetcher into MetadataFetcher 1/4

2023-02-14 Thread via GitHub
kirktrue commented on code in PR #13192: URL: https://github.com/apache/kafka/pull/13192#discussion_r1106476882 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MetadataFetcher.java: ## @@ -0,0 +1,805 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] mjsax commented on a diff in pull request #13243: KAFKA-14491: [7/N] Enforce strict grace period for versioned stores

2023-02-14 Thread via GitHub
mjsax commented on code in PR #13243: URL: https://github.com/apache/kafka/pull/13243#discussion_r1106482112 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStore.java: ## @@ -283,7 +293,12 @@ public void init(final StateStoreContext context, fi

[GitHub] [kafka] ijuma commented on pull request #13247: [WIP] KAFKA-14595 Move value object of ReassignPartitionsCommand to java

2023-02-14 Thread via GitHub
ijuma commented on PR #13247: URL: https://github.com/apache/kafka/pull/13247#issuecomment-1430569601 Is `server-common` the right module? What requires access to these classes aside from `tools`? -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [kafka] ijuma commented on pull request #13246: KAFKA-14707: Improve array size check

2023-02-14 Thread via GitHub
ijuma commented on PR #13246: URL: https://github.com/apache/kafka/pull/13246#issuecomment-1430570552 Checking for OOM is a security issue, so we cannot do that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13243: KAFKA-14491: [7/N] Enforce strict grace period for versioned stores

2023-02-14 Thread via GitHub
vcrfxia commented on code in PR #13243: URL: https://github.com/apache/kafka/pull/13243#discussion_r1106532591 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStore.java: ## @@ -297,6 +312,12 @@ void restoreBatch(final Collection> records) {

[GitHub] [kafka] bachmanity1 commented on a diff in pull request #13246: KAFKA-14707: Improve array size check

2023-02-14 Thread via GitHub
bachmanity1 commented on code in PR #13246: URL: https://github.com/apache/kafka/pull/13246#discussion_r1106542835 ## clients/src/main/java/org/apache/kafka/common/protocol/types/ArrayOf.java: ## @@ -72,10 +74,10 @@ public Object read(ByteBuffer buffer) { else if (size

[GitHub] [kafka] ijuma commented on pull request #13205: KAFKA-14680: gradle version upgrade 7 -->> 8

2023-02-14 Thread via GitHub
ijuma commented on PR #13205: URL: https://github.com/apache/kafka/pull/13205#issuecomment-1430621318 Gradle 8 has been released. Have you been able to figure out why the build is failing? -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [kafka] mjsax commented on a diff in pull request #13243: KAFKA-14491: [7/N] Enforce strict grace period for versioned stores

2023-02-14 Thread via GitHub
mjsax commented on code in PR #13243: URL: https://github.com/apache/kafka/pull/13243#discussion_r1106567105 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStore.java: ## @@ -283,7 +293,12 @@ public void init(final StateStoreContext context, fi

  1   2   >