[GitHub] [kafka] dengziming opened a new pull request #9575: MINOR: optimize lock operation

dengziming opened a new pull request #9575: URL: https://github.com/apache/kafka/pull/9575 move lock backward. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [kafka] dengziming commented on pull request #9574: MINOR: KIP-497 Not start Controller2ChannelManager when we are using zookeeper to notify isr change

dengziming commented on pull request #9574: URL: https://github.com/apache/kafka/pull/9574#issuecomment-723376858 @abbccdda @hachikuji ,Hi, PTAL. This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [kafka] dengziming opened a new pull request #9574: MINOR: KIP-497 Not start Controller2ChannelManager when we are using zookeeper to notify isr change

dengziming opened a new pull request #9574: URL: https://github.com/apache/kafka/pull/9574 KafkaServer Start alterIsrChannelManager even when we are using zookeeper to notify isr change.   BrokerToControllerChannelManager will consume some resources when started, so we should optimiz

[GitHub] [kafka] chia7712 commented on a change in pull request #9547: KAFKA-9630; Migrate OffsetsForLeaderEpochRequest/Response to the auto-generated protocol

chia7712 commented on a change in pull request #9547: URL: https://github.com/apache/kafka/pull/9547#discussion_r519081332 ## File path: clients/src/main/resources/common/message/OffsetForLeaderEpochRequest.json ## @@ -32,13 +32,13 @@ { "name": "Topics", "type": "[]Offset

[GitHub] [kafka] chia7712 commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

chia7712 commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r519081195 ## File path: clients/src/main/resources/common/message/ProduceRequest.json ## @@ -33,21 +33,21 @@ "validVersions": "0-8", "flexibleVersions": "none"

[GitHub] [kafka] hachikuji commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

hachikuji commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r519079153 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -194,7 +107,27 @@ private ProduceRequest build(short versi

[GitHub] [kafka] chia7712 commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

chia7712 commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r519067655 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -194,7 +107,27 @@ private ProduceRequest build(short versio

[GitHub] [kafka] chia7712 commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

chia7712 commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r519066134 ## File path: clients/src/main/resources/common/message/ProduceRequest.json ## @@ -33,21 +33,21 @@ "validVersions": "0-8", "flexibleVersions": "none"

[GitHub] [kafka] ableegoldman commented on pull request #9489: MINOR: demote "Committing task offsets" log to DEBUG

ableegoldman commented on pull request #9489: URL: https://github.com/apache/kafka/pull/9489#issuecomment-723347370 Eh, I'm going back and forth on this. It's definitely valuable information to have in the absence of any other logging within the main Streams loop at the INFO level --

[GitHub] [kafka] splett2 commented on a change in pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

splett2 commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r519054824 ## File path: core/src/main/scala/kafka/network/SocketServer.scala ## @@ -1659,19 +1648,22 @@ class ConnectionQuotas(config: KafkaConfig, time: Time, metri

[GitHub] [kafka] splett2 commented on a change in pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

splett2 commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r519054824 ## File path: core/src/main/scala/kafka/network/SocketServer.scala ## @@ -1659,19 +1648,22 @@ class ConnectionQuotas(config: KafkaConfig, time: Time, metri

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9572: KAFKA-10500: Thread Cache Resizes

wcarlson5 commented on a change in pull request #9572: URL: https://github.com/apache/kafka/pull/9572#discussion_r519049317 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -589,6 +589,10 @@ private void subscribeConsumer

[GitHub] [kafka] dajac commented on pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

dajac commented on pull request #9386: URL: https://github.com/apache/kafka/pull/9386#issuecomment-723332813 @splett2 Could you rebase the PR? I'll review it on Monday. This is an automated message from the Apache Git Service

[GitHub] [kafka] splett2 opened a new pull request #9573: KAFKA-10693: Close quota managers created in tests

splett2 opened a new pull request #9573: URL: https://github.com/apache/kafka/pull/9573 We have various tests that call `QuotaFactory.instantiate()`, but don't close the returned `QuotaManagers`. This results in dangling threads while running tests. ### Committer Checklist (excluded

[jira] [Created] (KAFKA-10693) Tests instantiate QuotaManagers without closing the managers in teardown

David Mao created KAFKA-10693: - Summary: Tests instantiate QuotaManagers without closing the managers in teardown Key: KAFKA-10693 URL: https://issues.apache.org/jira/browse/KAFKA-10693 Project: Kafka

[GitHub] [kafka] hachikuji edited a comment on pull request #9563: KAFKA-10684; Avoid additional envelope copies during network transmission

hachikuji edited a comment on pull request #9563: URL: https://github.com/apache/kafka/pull/9563#issuecomment-723209437 Here are updated results for `FetchResponseBenchmark`. Trunk: ``` Benchmark (partitionCount) (topicCount) Mode Cnt

[GitHub] [kafka] hachikuji commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

hachikuji commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r518989756 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -194,7 +107,27 @@ private ProduceRequest build(short versi

[GitHub] [kafka] hachikuji commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

hachikuji commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r518984064 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceResponse.java ## @@ -204,118 +75,78 @@ public ProduceResponse(Map responses)

[GitHub] [kafka] wcarlson5 commented on pull request #9572: KAFKA-10500: Thread Cache Resizes

wcarlson5 commented on pull request #9572: URL: https://github.com/apache/kafka/pull/9572#issuecomment-723242241 @cadonna Part 2 This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] wcarlson5 opened a new pull request #9572: KAFKA-10500: Thread Cache Resizes

wcarlson5 opened a new pull request #9572: URL: https://github.com/apache/kafka/pull/9572 The thread cache can now be resized. This will go towards being able to scale the number of threads ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementati

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r518938852 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -996,6 +1082,62 @@ private boolean close(final long timeoutMs) {

[GitHub] [kafka] rajinisivaram commented on a change in pull request #8986: KAFKA-10233; Add backoff after AuthorizationExceptions in consumer

rajinisivaram commented on a change in pull request #8986: URL: https://github.com/apache/kafka/pull/8986#discussion_r518938665 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java ## @@ -460,15 +462,21 @@ boolean joinGroupIf

[GitHub] [kafka] rajinisivaram commented on a change in pull request #8986: KAFKA-10233; Add backoff after AuthorizationExceptions in consumer

rajinisivaram commented on a change in pull request #8986: URL: https://github.com/apache/kafka/pull/8986#discussion_r518937545 ## File path: clients/src/test/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinatorTest.java ## @@ -259,6 +260,22 @@ public void test

[GitHub] [kafka] jsancio commented on a change in pull request #9512: KAFKA-10394: generate snapshot

jsancio commented on a change in pull request #9512: URL: https://github.com/apache/kafka/pull/9512#discussion_r518916392 ## File path: raft/src/main/java/org/apache/kafka/snapshot/SnapshotReader.java ## @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] ableegoldman commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

ableegoldman commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r518913514 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -996,6 +1082,62 @@ private boolean close(final long timeoutMs) {

[jira] [Commented] (KAFKA-10688) Handle accidental truncation of repartition topics as exceptional failure

[ https://issues.apache.org/jira/browse/KAFKA-10688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17227545#comment-17227545 ] Guozhang Wang commented on KAFKA-10688: --- No we don't actually :) In StreamThread w

[GitHub] [kafka] hachikuji edited a comment on pull request #9563: KAFKA-10684; Avoid additional envelope copies during network transmission

hachikuji edited a comment on pull request #9563: URL: https://github.com/apache/kafka/pull/9563#issuecomment-723209437 Here are updated results for `FetchResponseBenchmark`. Trunk: ``` Benchmark (partitionCount) (topicCount) Mode Cnt

[GitHub] [kafka] hachikuji commented on pull request #9563: KAFKA-10684; Avoid additional envelope copies during network transmission

hachikuji commented on pull request #9563: URL: https://github.com/apache/kafka/pull/9563#issuecomment-723209437 Here are updated benchmarks for `FetchResponseBenchmark`. Trunk: ``` Benchmark (partitionCount) (topicCount) Mode Cnt

[GitHub] [kafka] hachikuji commented on pull request #9563: KAFKA-10684; Avoid additional envelope copies during network transmission

hachikuji commented on pull request #9563: URL: https://github.com/apache/kafka/pull/9563#issuecomment-723207603 @chia7712 Thanks for the reviews. I pushed an update to address your comments. I was a little concerned about the garbage created from the `MessageSize` objects, so I changed th

[jira] [Commented] (KAFKA-5381) ERROR Uncaught exception in scheduled task 'delete-expired-consumer-offsets' (kafka.utils.KafkaScheduler)

[ https://issues.apache.org/jira/browse/KAFKA-5381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17227530#comment-17227530 ] Brajesh Kumar commented on KAFKA-5381: --- We have seen this issue with Kafka 2.3.1.

[GitHub] [kafka] hachikuji commented on a change in pull request #9563: KAFKA-10684; Avoid additional envelope copies during network transmission

hachikuji commented on a change in pull request #9563: URL: https://github.com/apache/kafka/pull/9563#discussion_r518885497 ## File path: clients/src/main/java/org/apache/kafka/common/requests/EnvelopeRequest.java ## @@ -91,4 +91,14 @@ public AbstractResponse getErrorResponse(

[GitHub] [kafka] hachikuji commented on a change in pull request #9563: KAFKA-10684; Avoid additional envelope copies during network transmission

hachikuji commented on a change in pull request #9563: URL: https://github.com/apache/kafka/pull/9563#discussion_r518885497 ## File path: clients/src/main/java/org/apache/kafka/common/requests/EnvelopeRequest.java ## @@ -91,4 +91,14 @@ public AbstractResponse getErrorResponse(

[GitHub] [kafka] hachikuji commented on a change in pull request #9512: KAFKA-10394: generate snapshot

hachikuji commented on a change in pull request #9512: URL: https://github.com/apache/kafka/pull/9512#discussion_r518882604 ## File path: raft/src/main/java/org/apache/kafka/snapshot/SnapshotReader.java ## @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[jira] [Resolved] (KAFKA-10393) Message for fetch snapshot and fetch

[ https://issues.apache.org/jira/browse/KAFKA-10393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio resolved KAFKA-10393. Resolution: Duplicate > Message for fetch snapshot and fetch > ---

[jira] [Commented] (KAFKA-10678) Re-deploying Streams app causes rebalance and task migration

[ https://issues.apache.org/jira/browse/KAFKA-10678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17227507#comment-17227507 ] Bradley Peterson commented on KAFKA-10678: -- Sophie, thank you again for your th

[jira] [Created] (KAFKA-10692) Rename broker master key config for KIP 681

Tom Bentley created KAFKA-10692: --- Summary: Rename broker master key config for KIP 681 Key: KAFKA-10692 URL: https://issues.apache.org/jira/browse/KAFKA-10692 Project: Kafka Issue Type: Sub-tas

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r518850419 ## File path: streams/src/main/resources/common/message/SubscriptionInfoData.json ## @@ -57,6 +57,11 @@ "name": "uniqueField", "versions": "

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r518842747 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -282,6 +283,17 @@ public boolean isRunning() {

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r518840602 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/ReferenceContainer.java ## @@ -30,7 +30,7 @@ public Admi

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r518840121 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsRebalanceListener.java ## @@ -60,6 +60,11 @@ public void onPartit

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r518838586 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -996,6 +1082,62 @@ private boolean close(final long timeoutMs) {

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r518838421 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -996,6 +1082,62 @@ private boolean close(final long timeoutMs) {

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r518837250 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -567,10 +589,34 @@ void runLoop() {

[jira] [Assigned] (KAFKA-4748) Need a way to shutdown all workers in a Streams application at the same time

[ https://issues.apache.org/jira/browse/KAFKA-4748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walker Carlson reassigned KAFKA-4748: - Assignee: Walker Carlson > Need a way to shutdown all workers in a Streams application a

[GitHub] [kafka] mimaison merged pull request #9565: MINOR: Move upgraded docs from site to Kafka docs

mimaison merged pull request #9565: URL: https://github.com/apache/kafka/pull/9565 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] cadonna commented on a change in pull request #9508: KAFKA-10648: Add Prefix Scan support to State Stores

cadonna commented on a change in pull request #9508: URL: https://github.com/apache/kafka/pull/9508#discussion_r518682502 ## File path: streams/src/main/java/org/apache/kafka/streams/state/ReadOnlyKeyValueStore.java ## @@ -107,4 +108,18 @@ * @throws InvalidStateStoreExce

[GitHub] [kafka] scanterog commented on pull request #9545: [mm2] Allow Checkpoints for consumers using static partition assignments

scanterog commented on pull request #9545: URL: https://github.com/apache/kafka/pull/9545#issuecomment-723063889 @ryannedolan friendly ping. Would love your feedback on this one. This is an automated message from the Apache G

[GitHub] [kafka] ghmulti commented on pull request #9489: MINOR: demote "Committing task offsets" log to DEBUG

ghmulti commented on pull request #9489: URL: https://github.com/apache/kafka/pull/9489#issuecomment-723038111 hello @ableegoldman, I wonder why did you close the pr? This is an automated message from the Apache Git Service.

[jira] [Issue Comment Deleted] (KAFKA-10629) TopologyTestDriver should not require a Properties arg

[ https://issues.apache.org/jira/browse/KAFKA-10629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bovetti updated KAFKA-10629: --- Comment: was deleted (was: Wrong URL, please fix with [https://github.com/apache/kafka/pull/

[GitHub] [kafka] mimaison commented on pull request #9565: MINOR: Move upgraded docs from site to Kafka docs

mimaison commented on pull request #9565: URL: https://github.com/apache/kafka/pull/9565#issuecomment-723006249 Thanks @bbejeck ! I've pushed an update This is an automated message from the Apache Git Service. To respond to t

[GitHub] [kafka] dengziming closed pull request #7826: Kafka-9275: Update "reason" string for rebalance for faster investigation when rebalance

dengziming closed pull request #7826: URL: https://github.com/apache/kafka/pull/7826 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] dengziming commented on pull request #7826: Kafka-9275: Update "reason" string for rebalance for faster investigation when rebalance

dengziming commented on pull request #7826: URL: https://github.com/apache/kafka/pull/7826#issuecomment-722971595 Hi, I am closing the pr as no one is interested. we can reopen if anyone is interested. This is an automated m

[jira] [Issue Comment Deleted] (KAFKA-9275) Print assignment and IP address in the log message when consumer leaves/removed from the group

[ https://issues.apache.org/jira/browse/KAFKA-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dengziming updated KAFKA-9275: -- Comment: was deleted (was: https://github.com/apache/kafka/pull/7826) > Print assignment and IP addres

[jira] [Commented] (KAFKA-9275) Print assignment and IP address in the log message when consumer leaves/removed from the group

[ https://issues.apache.org/jira/browse/KAFKA-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17227259#comment-17227259 ] dengziming commented on KAFKA-9275: --- https://github.com/apache/kafka/pull/7826 > Print

[GitHub] [kafka] rajinisivaram merged pull request #9567: MINOR: Always return partitions with diverging epochs in fetch response

rajinisivaram merged pull request #9567: URL: https://github.com/apache/kafka/pull/9567 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] rajinisivaram commented on pull request #9567: MINOR: Always return partitions with diverging epochs in fetch response

rajinisivaram commented on pull request #9567: URL: https://github.com/apache/kafka/pull/9567#issuecomment-722957915 @hachikuji @chia7712 Thanks for the review, merging to trunk and 2.7. This is an automated message from the

[jira] [Commented] (KAFKA-10688) Handle accidental truncation of repartition topics as exceptional failure

[ https://issues.apache.org/jira/browse/KAFKA-10688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17227236#comment-17227236 ] Bruno Cadonna commented on KAFKA-10688: --- [~guozhang], Thank you for the proposal.

[GitHub] [kafka] dengziming commented on pull request #8552: MINOR: Fix some java docs and move repeated `if condition` to the method

dengziming commented on pull request #8552: URL: https://github.com/apache/kafka/pull/8552#issuecomment-722949912 also ping @chia7712 to have a look This is an automated message from the Apache Git Service. To respond to the

[jira] [Resolved] (KAFKA-10673) ConnectionQuotas should cache interbroker listener name

[ https://issues.apache.org/jira/browse/KAFKA-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-10673. - Resolution: Fixed > ConnectionQuotas should cache interbroker listener name > --

[GitHub] [kafka] dajac merged pull request #9555: KAFKA-10673: Cache inter broker listener name used in connection quotas

dajac merged pull request #9555: URL: https://github.com/apache/kafka/pull/9555 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] edenhill commented on a change in pull request #9569: KAFKA-10687: make ProduceRespone only returns INVALID_PRODUCER_EPOCH

edenhill commented on a change in pull request #9569: URL: https://github.com/apache/kafka/pull/9569#discussion_r518582162 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -609,6 +609,12 @@ class KafkaApis(val requestChannel: RequestChannel, mergedR