[jira] [Commented] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

2020-07-10 Thread Sean Guo (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155245#comment-17155245 ] Sean Guo commented on KAFKA-10134: -- [~guozhang] Looks like the latest change bring back

[jira] [Updated] (KAFKA-10254) 100% cpu usage by kafkaConsumer poll , when broker can‘t be connect

2020-07-10 Thread xiaotong.wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiaotong.wang updated KAFKA-10254: -- Fix Version/s: 2.5.1 > 100% cpu usage by kafkaConsumer poll , when broker can‘t be connect >

[jira] [Commented] (KAFKA-3720) Remove BufferExhaustedException from doSend() in KafkaProducer

2020-07-10 Thread Geert Pante (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155279#comment-17155279 ] Geert Pante commented on KAFKA-3720: [~sliebau], [~mgharat] ??An open question is w

[GitHub] [kafka] dajac commented on pull request #9001: KAFKA-10028: Implement KIP-584 write path

2020-07-10 Thread GitBox
dajac commented on pull request #9001: URL: https://github.com/apache/kafka/pull/9001#issuecomment-656589433 @kowshik I just noticed that you haven't updated the code which creates the `ApiVersionsResponse` in `SaslServerAuthenticator`. Is it intentional or something left to be done? ---

[GitHub] [kafka] rajinisivaram commented on pull request #8979: KAFKA-10223; Use NOT_LEADER_OR_FOLLOWER instead of non-retriable REPLICA_NOT_AVAILABLE for consumers

2020-07-10 Thread GitBox
rajinisivaram commented on pull request #8979: URL: https://github.com/apache/kafka/pull/8979#issuecomment-656594318 @ijuma @hachikuji Updated PR as discussed. This is an automated message from the Apache Git Service. To resp

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-07-10 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r452754165 ## File path: core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala ## @@ -239,9 +239,13 @@ class GroupMetadataManager(brokerId: Int,

[GitHub] [kafka] chia7712 commented on a change in pull request #8657: KAFKA-8334 Make sure the thread which tries to complete delayed reque…

2020-07-10 Thread GitBox
chia7712 commented on a change in pull request #8657: URL: https://github.com/apache/kafka/pull/8657#discussion_r452755236 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -65,13 +65,24 @@ import scala.compat.java8.OptionConverters._ /* * Result meta

[GitHub] [kafka] mlex commented on pull request #7540: Eagerly unmap during resize

2020-07-10 Thread GitBox
mlex commented on pull request #7540: URL: https://github.com/apache/kafka/pull/7540#issuecomment-656635524 We are running into a similar issue and see lots of references to already deleted index files held by the kafka process. Is there a reason why the `safeForceUnmap ` shouldn't be call

[GitHub] [kafka] tombentley opened a new pull request #9007: KAFKA-10120: Deprecate DescribeLogDirsResult.all() and .values()

2020-07-10 Thread GitBox
tombentley opened a new pull request #9007: URL: https://github.com/apache/kafka/pull/9007 As per KIP-621. Also added some tests in KafkaAdminClientTest This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] tombentley commented on pull request #9007: KAFKA-10120: Deprecate DescribeLogDirsResult.all() and .values()

2020-07-10 Thread GitBox
tombentley commented on pull request #9007: URL: https://github.com/apache/kafka/pull/9007#issuecomment-656690684 @cmccabe, @omkreddy, @mimaison, @dajac, @dongjinleekr since you all voted on the KIP feel free to review. This

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r452889643 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -768,9 +770,30 @@ void runOnce() { private

[GitHub] [kafka] vvcephei commented on pull request #8993: KAFKA-10173: Use SmokeTest for upgrade system tests (#8938)

2020-07-10 Thread GitBox
vvcephei commented on pull request #8993: URL: https://github.com/apache/kafka/pull/8993#issuecomment-656721207 Rebased, resolved conflict, and re-ran all Streams tests: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/4023/console

[GitHub] [kafka] cmccabe commented on a change in pull request #9002: MINOR: Add ApiMessageTypeGenerator

2020-07-10 Thread GitBox
cmccabe commented on a change in pull request #9002: URL: https://github.com/apache/kafka/pull/9002#discussion_r452902273 ## File path: generator/src/main/java/org/apache/kafka/message/MessageGenerator.java ## @@ -267,10 +283,10 @@ public static void main(String[] args) throws

[GitHub] [kafka] cmccabe commented on a change in pull request #9002: MINOR: Add ApiMessageTypeGenerator

2020-07-10 Thread GitBox
cmccabe commented on a change in pull request #9002: URL: https://github.com/apache/kafka/pull/9002#discussion_r452903067 ## File path: generator/src/main/java/org/apache/kafka/message/MessageGenerator.java ## @@ -169,20 +185,20 @@ public static void processDirectories(String

[GitHub] [kafka] vvcephei commented on pull request #8996: KAFKA-10249: don't try to read un-checkpointed offsets of in-memory stores

2020-07-10 Thread GitBox
vvcephei commented on pull request #8996: URL: https://github.com/apache/kafka/pull/8996#issuecomment-656731407 merged to trunk and 2.6 This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [kafka] vvcephei commented on pull request #8971: MINOR: prune the metadata upgrade test matrix

2020-07-10 Thread GitBox
vvcephei commented on pull request #8971: URL: https://github.com/apache/kafka/pull/8971#issuecomment-656733910 Backported to 2.5 This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Assigned] (KAFKA-10215) MockProcessorContext doesn't work with SessionStores

2020-07-10 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler reassigned KAFKA-10215: Assignee: (was: John Roesler) > MockProcessorContext doesn't work with SessionStores

[jira] [Created] (KAFKA-10265) FetchRequest and FetchResponse should use the generated message classes

2020-07-10 Thread David Arthur (Jira)
David Arthur created KAFKA-10265: Summary: FetchRequest and FetchResponse should use the generated message classes Key: KAFKA-10265 URL: https://issues.apache.org/jira/browse/KAFKA-10265 Project: Kafk

[GitHub] [kafka] abbccdda commented on a change in pull request #9002: MINOR: Add ApiMessageTypeGenerator

2020-07-10 Thread GitBox
abbccdda commented on a change in pull request #9002: URL: https://github.com/apache/kafka/pull/9002#discussion_r452927978 ## File path: generator/src/main/java/org/apache/kafka/message/MessageGenerator.java ## @@ -169,20 +185,20 @@ public static void processDirectories(String

[jira] [Commented] (KAFKA-10265) FetchRequest and FetchResponse should use the generated message classes

2020-07-10 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1718#comment-1718 ] Tom Bentley commented on KAFKA-10265: - Dupe of https://issues.apache.org/jira/browse

[jira] [Commented] (KAFKA-10265) FetchRequest and FetchResponse should use the generated message classes

2020-07-10 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155566#comment-17155566 ] Mickael Maison commented on KAFKA-10265: Feel free to grab https://issues.apache

[jira] [Commented] (KAFKA-10265) FetchRequest and FetchResponse should use the generated message classes

2020-07-10 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155575#comment-17155575 ] David Arthur commented on KAFKA-10265: -- Ah, thanks -- I'll grab that one. I have a

[jira] [Resolved] (KAFKA-10265) FetchRequest and FetchResponse should use the generated message classes

2020-07-10 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur resolved KAFKA-10265. -- Resolution: Duplicate > FetchRequest and FetchResponse should use the generated message classe

[jira] [Assigned] (KAFKA-9629) Replace Fetch request/response with automated protocol

2020-07-10 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur reassigned KAFKA-9629: --- Assignee: David Arthur (was: Mickael Maison) > Replace Fetch request/response with automate

[GitHub] [kafka] mjsax commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
mjsax commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r452939323 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -768,9 +770,30 @@ void runOnce() { private vo

[GitHub] [kafka] mjsax commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
mjsax commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r452939323 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -768,9 +770,30 @@ void runOnce() { private vo

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r452948608 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -768,9 +770,30 @@ void runOnce() { private

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r452949230 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -193,6 +196,26 @@ private void closeAndRevive(fina

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r452951286 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -359,11 +359,8 @@ public static StreamThread crea

[GitHub] [kafka] vvcephei commented on pull request #8913: KAFKA-10191 fix flaky StreamsOptimizedTest

2020-07-10 Thread GitBox
vvcephei commented on pull request #8913: URL: https://github.com/apache/kafka/pull/8913#issuecomment-656791489 Thanks, @ableegoldman and @ijuma ; I've cherry-picked it to 2.6. This is an automated message from the Apache Git

[GitHub] [kafka] vvcephei commented on pull request #8913: KAFKA-10191 fix flaky StreamsOptimizedTest

2020-07-10 Thread GitBox
vvcephei commented on pull request #8913: URL: https://github.com/apache/kafka/pull/8913#issuecomment-656792870 cc @rhauch (I've picked this into the 2.6 branch) This is an automated message from the Apache Git Service. To re

[GitHub] [kafka] dajac commented on a change in pull request #9007: KAFKA-10120: Deprecate DescribeLogDirsResult.all() and .values()

2020-07-10 Thread GitBox
dajac commented on a change in pull request #9007: URL: https://github.com/apache/kafka/pull/9007#discussion_r452893782 ## File path: core/src/main/scala/kafka/admin/LogDirsCommand.scala ## @@ -64,7 +63,7 @@ object LogDirsCommand { "logDirs" -> logDirInfos.

[GitHub] [kafka] guozhangwang commented on pull request #8997: MINOR: Improve log4j for per-consumer assignment

2020-07-10 Thread GitBox
guozhangwang commented on pull request #8997: URL: https://github.com/apache/kafka/pull/8997#issuecomment-656798091 > That said, I'd be +1 on adding logs to also explicitly point out the stateless and stateful tasks. But we can always figure that out in other ways so it's only more of a ni

[GitHub] [kafka] ableegoldman commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
ableegoldman commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r452984743 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -359,11 +359,8 @@ public static StreamThread

[GitHub] [kafka] mumrah opened a new pull request #9008: KAFKA-9629 Use generated protocol for Fetch API

2020-07-10 Thread GitBox
mumrah opened a new pull request #9008: URL: https://github.com/apache/kafka/pull/9008 This change makes use of the generated protocols for FetchRequest and FetchResponse. The main challenge here was how to allow the transferrable bytes of the record set to be directly sent to the outgoing

[GitHub] [kafka] ijuma commented on pull request #9008: KAFKA-9629 Use generated protocol for Fetch API

2020-07-10 Thread GitBox
ijuma commented on pull request #9008: URL: https://github.com/apache/kafka/pull/9008#issuecomment-656809268 Thanks for the PR. Since this affects the fetch path, let's make sure we benchmark this. cc @lbradstreet This is a

[GitHub] [kafka] mjsax commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
mjsax commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r452995040 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -359,11 +359,8 @@ public static StreamThread create(

[GitHub] [kafka] mjsax commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
mjsax commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r452995771 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -359,11 +359,8 @@ public static StreamThread create(

[GitHub] [kafka] mjsax commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
mjsax commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r452996564 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -767,38 +766,65 @@ void runOnce() { } pr

[GitHub] [kafka] abbccdda commented on a change in pull request #8986: KAFKA-10233; Add backoff after AuthorizationExceptions in consumer

2020-07-10 Thread GitBox
abbccdda commented on a change in pull request #8986: URL: https://github.com/apache/kafka/pull/8986#discussion_r452997443 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java ## @@ -460,15 +463,21 @@ boolean joinGroupIfNeede

[GitHub] [kafka] guozhangwang commented on pull request #9005: KAFKA-10263: Do not assign standby for revoking stateless tasks

2020-07-10 Thread GitBox
guozhangwang commented on pull request #9005: URL: https://github.com/apache/kafka/pull/9005#issuecomment-656814245 > Awesome, thanks for the fix! Build has a checkstyleTest violation but otherwise this looks good Fixed the checkstyle. --

[jira] [Commented] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

2020-07-10 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155659#comment-17155659 ] Guozhang Wang commented on KAFKA-10134: --- Hmm, interesting. What setup are you usin

[GitHub] [kafka] ableegoldman commented on pull request #8997: MINOR: Improve log4j for per-consumer assignment

2020-07-10 Thread GitBox
ableegoldman commented on pull request #8997: URL: https://github.com/apache/kafka/pull/8997#issuecomment-656831568 Thanks @guozhangwang , LGTM This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [kafka] ableegoldman commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
ableegoldman commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453027126 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -193,6 +196,26 @@ private void closeAndRevive(

[GitHub] [kafka] vvcephei commented on a change in pull request #9005: KAFKA-10263: Do not assign standby for revoking stateless tasks

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #9005: URL: https://github.com/apache/kafka/pull/9005#discussion_r453027992 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignorTest.java ## @@ -1402,9 +1404,13 @@ public void

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453034571 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -95,7 +96,7 @@ * | | Assigned

[GitHub] [kafka] vvcephei commented on pull request #8913: KAFKA-10191 fix flaky StreamsOptimizedTest

2020-07-10 Thread GitBox
vvcephei commented on pull request #8913: URL: https://github.com/apache/kafka/pull/8913#issuecomment-656849232 And cherry-picked to 2.5 as well. This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [kafka] cmccabe commented on pull request #8998: MINOR; KafkaAdminClient#describeLogDirs should not fail all the futures when only one call fails

2020-07-10 Thread GitBox
cmccabe commented on pull request #8998: URL: https://github.com/apache/kafka/pull/8998#issuecomment-656853033 LGTM pending jenkins This is an automated message from the Apache Git Service. To respond to the message, please l

[jira] [Commented] (KAFKA-10253) Kafka Connect gets into an infinite rebalance loop

2020-07-10 Thread Konstantin Lalafaryan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155701#comment-17155701 ] Konstantin Lalafaryan commented on KAFKA-10253: --- We can reproduce the same

[GitHub] [kafka] mjsax commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
mjsax commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453070214 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -95,7 +96,7 @@ * | | Assigned (3

[GitHub] [kafka] ableegoldman commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
ableegoldman commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453073305 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -95,7 +96,7 @@ * | | Assi

[GitHub] [kafka] mimaison commented on pull request #8295: KAFKA-9627: Replace ListOffset request/response with automated protocol

2020-07-10 Thread GitBox
mimaison commented on pull request #8295: URL: https://github.com/apache/kafka/pull/8295#issuecomment-656894169 Thanks @dajac for the feedback. I've pushed an update This is an automated message from the Apache Git Service. T

[GitHub] [kafka] mjsax commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
mjsax commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453082191 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -95,7 +96,7 @@ * | | Assigned (3

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453086734 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -359,11 +359,8 @@ public static StreamThread crea

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453087493 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -767,38 +766,65 @@ void runOnce() { }

[GitHub] [kafka] kowshik commented on pull request #9001: KAFKA-10028: Implement KIP-584 write path

2020-07-10 Thread GitBox
kowshik commented on pull request #9001: URL: https://github.com/apache/kafka/pull/9001#issuecomment-656902905 @dajac Thank you for taking a look! IIUC you are referring to these lines: https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/security/

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453088861 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -193,6 +196,26 @@ private void closeAndRevive(fina

[GitHub] [kafka] kowshik edited a comment on pull request #9001: KAFKA-10028: Implement KIP-584 write path

2020-07-10 Thread GitBox
kowshik edited a comment on pull request #9001: URL: https://github.com/apache/kafka/pull/9001#issuecomment-656902905 @dajac Thank you for taking a look! IIUC you are referring to these lines: https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/se

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453089755 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -95,7 +96,7 @@ * | | Assigned

[GitHub] [kafka] vvcephei commented on pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on pull request #8994: URL: https://github.com/apache/kafka/pull/8994#issuecomment-656904545 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [kafka] dajac commented on pull request #8998: MINOR; KafkaAdminClient#describeLogDirs should not fail all the futures when only one call fails

2020-07-10 Thread GitBox
dajac commented on pull request #8998: URL: https://github.com/apache/kafka/pull/8998#issuecomment-656905640 All jenkins builds are green. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] hachikuji commented on pull request #9008: KAFKA-9629 Use generated protocol for Fetch API

2020-07-10 Thread GitBox
hachikuji commented on pull request #9008: URL: https://github.com/apache/kafka/pull/9008#issuecomment-656906624 I agree that some benchmarks would be useful. One of the key differences is how the `MultiRecordsSend` gets constructed, so that is probably one thing. Potentially we are faster

[GitHub] [kafka] kowshik edited a comment on pull request #9001: KAFKA-10028: Implement KIP-584 write path

2020-07-10 Thread GitBox
kowshik edited a comment on pull request #9001: URL: https://github.com/apache/kafka/pull/9001#issuecomment-656902905 @dajac Thank you for taking a look! IIUC you are referring to these lines: https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/se

[jira] [Commented] (KAFKA-10262) StateDirectory is not thread-safe

2020-07-10 Thread Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155773#comment-17155773 ] Sophie Blee-Goldman commented on KAFKA-10262: - I think we can just do someth

[jira] [Comment Edited] (KAFKA-10262) StateDirectory is not thread-safe

2020-07-10 Thread Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17155773#comment-17155773 ] Sophie Blee-Goldman edited comment on KAFKA-10262 at 7/10/20, 10:17 PM: --

[GitHub] [kafka] mjsax commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
mjsax commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453105805 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -760,44 +763,62 @@ void runOnce() { try {

[GitHub] [kafka] mjsax commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
mjsax commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453106606 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -193,6 +195,34 @@ private void closeAndRevive(final M

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453107700 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -760,44 +763,62 @@ void runOnce() { try

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453108030 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -193,6 +195,34 @@ private void closeAndRevive(fina

[GitHub] [kafka] mjsax commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
mjsax commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453109741 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamThreadTest.java ## @@ -1177,6 +1180,109 @@ public void shouldNotClose

[GitHub] [kafka] mjsax commented on pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
mjsax commented on pull request #8994: URL: https://github.com/apache/kafka/pull/8994#issuecomment-656928347 Just two last comments/questions. Overall LGTM. This is an automated message from the Apache Git Service. To respond

[jira] [Created] (KAFKA-10266) Fix connector configs in docs to mention the correct default value inherited from worker configs

2020-07-10 Thread Konstantine Karantasis (Jira)
Konstantine Karantasis created KAFKA-10266: -- Summary: Fix connector configs in docs to mention the correct default value inherited from worker configs Key: KAFKA-10266 URL: https://issues.apache.org/jira/

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453121093 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamThreadTest.java ## @@ -1177,6 +1180,109 @@ public void shouldNotCl

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453121551 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -760,44 +763,62 @@ void runOnce() { try

[GitHub] [kafka] JimGalasyn opened a new pull request #9009: DOCS-4810: Document how timestamps are computed for aggregations and joins

2020-07-10 Thread GitBox
JimGalasyn opened a new pull request #9009: URL: https://github.com/apache/kafka/pull/9009 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [kafka] guozhangwang merged pull request #9005: KAFKA-10263: Do not assign standby for revoking stateless tasks

2020-07-10 Thread GitBox
guozhangwang merged pull request #9005: URL: https://github.com/apache/kafka/pull/9005 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] mjsax opened a new pull request #9010: KAFKA-10262: Ensure that creating task directory is thread safe

2020-07-10 Thread GitBox
mjsax opened a new pull request #9010: URL: https://github.com/apache/kafka/pull/9010 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing str

[jira] [Assigned] (KAFKA-10262) StateDirectory is not thread-safe

2020-07-10 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-10262: --- Assignee: Matthias J. Sax > StateDirectory is not thread-safe > ---

[GitHub] [kafka] mjsax commented on pull request #9010: KAFKA-10262: Ensure that creating task directory is thread safe

2020-07-10 Thread GitBox
mjsax commented on pull request #9010: URL: https://github.com/apache/kafka/pull/9010#issuecomment-656942987 Call for review @ableegoldman @vvcephei This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] abbccdda commented on pull request #9002: MINOR: Add ApiMessageTypeGenerator

2020-07-10 Thread GitBox
abbccdda commented on pull request #9002: URL: https://github.com/apache/kafka/pull/9002#issuecomment-656947686 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [kafka] apovzner commented on a change in pull request #8768: KAFKA-10023: Enforce broker-wide and per-listener connection creation…

2020-07-10 Thread GitBox
apovzner commented on a change in pull request #8768: URL: https://github.com/apache/kafka/pull/8768#discussion_r453130277 ## File path: core/src/test/scala/unit/kafka/network/ConnectionQuotasTest.scala ## @@ -329,15 +613,46 @@ class ConnectionQuotasTest { } // this met

[GitHub] [kafka] apovzner commented on a change in pull request #8768: KAFKA-10023: Enforce broker-wide and per-listener connection creation…

2020-07-10 Thread GitBox
apovzner commented on a change in pull request #8768: URL: https://github.com/apache/kafka/pull/8768#discussion_r453131010 ## File path: core/src/test/scala/unit/kafka/network/ConnectionQuotasTest.scala ## @@ -302,21 +312,295 @@ class ConnectionQuotasTest { } // a

[GitHub] [kafka] mjsax commented on a change in pull request #9010: KAFKA-10262: Ensure that creating task directory is thread safe

2020-07-10 Thread GitBox
mjsax commented on a change in pull request #9010: URL: https://github.com/apache/kafka/pull/9010#discussion_r453131515 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StateDirectoryTest.java ## @@ -507,4 +510,51 @@ public void shouldLockGloba

[GitHub] [kafka] mjsax commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
mjsax commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453131671 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -760,44 +763,62 @@ void runOnce() { try {

[GitHub] [kafka] abbccdda commented on pull request #8923: KAFKA-6435: KIP-623 Add internal topics option to streamResetter

2020-07-10 Thread GitBox
abbccdda commented on pull request #8923: URL: https://github.com/apache/kafka/pull/8923#issuecomment-656951683 @JoelWee Could we rebase this work? This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [kafka] junrao commented on a change in pull request #8672: KAFKA-10002; Improve performances of StopReplicaRequest with large number of partitions to be deleted

2020-07-10 Thread GitBox
junrao commented on a change in pull request #8672: URL: https://github.com/apache/kafka/pull/8672#discussion_r453132769 ## File path: core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala ## @@ -2058,13 +2070,37 @@ class ReplicaManagerTest { val partition = repli

[jira] [Updated] (KAFKA-10262) StateDirectory is not thread-safe

2020-07-10 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-10262: Priority: Blocker (was: Major) > StateDirectory is not thread-safe >

[GitHub] [kafka] ableegoldman commented on a change in pull request #9010: KAFKA-10262: Ensure that creating task directory is thread safe

2020-07-10 Thread GitBox
ableegoldman commented on a change in pull request #9010: URL: https://github.com/apache/kafka/pull/9010#discussion_r453132871 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java ## @@ -47,10 +47,10 @@ public class StateDirecto

[GitHub] [kafka] vvcephei commented on a change in pull request #9010: KAFKA-10262: Ensure that creating task directory is thread safe

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #9010: URL: https://github.com/apache/kafka/pull/9010#discussion_r453132853 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StateDirectoryTest.java ## @@ -507,4 +510,51 @@ public void shouldLockGl

[GitHub] [kafka] vvcephei commented on a change in pull request #9010: KAFKA-10262: Ensure that creating task directory is thread safe

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #9010: URL: https://github.com/apache/kafka/pull/9010#discussion_r453133360 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StateDirectoryTest.java ## @@ -507,4 +510,51 @@ public void shouldLockGl

[GitHub] [kafka] vvcephei commented on a change in pull request #9010: KAFKA-10262: Ensure that creating task directory is thread safe

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #9010: URL: https://github.com/apache/kafka/pull/9010#discussion_r453133360 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StateDirectoryTest.java ## @@ -507,4 +510,51 @@ public void shouldLockGl

[GitHub] [kafka] ableegoldman commented on pull request #9010: KAFKA-10262: Ensure that creating task directory is thread safe

2020-07-10 Thread GitBox
ableegoldman commented on pull request #9010: URL: https://github.com/apache/kafka/pull/9010#issuecomment-656953025 Oh nevermind, you literally just did. Ignore my last 🙂 This is an automated message from the Apache Git Serv

[GitHub] [kafka] vvcephei commented on pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on pull request #8994: URL: https://github.com/apache/kafka/pull/8994#issuecomment-656952992 Huh, one of those tests hit: ``` Exception in thread "stream-thread-test-87bf53a8-54f2-485f-a4b6-acdbec0a8b3d-StreamThread-1" java.util.ConcurrentModificationException

[GitHub] [kafka] mjsax commented on a change in pull request #9010: KAFKA-10262: Ensure that creating task directory is thread safe

2020-07-10 Thread GitBox
mjsax commented on a change in pull request #9010: URL: https://github.com/apache/kafka/pull/9010#discussion_r453134258 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java ## @@ -47,10 +47,10 @@ public class StateDirectory {

[GitHub] [kafka] abbccdda commented on a change in pull request #8997: MINOR: Improve log4j for per-consumer assignment

2020-07-10 Thread GitBox
abbccdda commented on a change in pull request #8997: URL: https://github.com/apache/kafka/pull/8997#discussion_r453144756 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java ## @@ -1113,6 +1135,7 @@ private boolean ad

[jira] [Created] (KAFKA-10267) [Documentation] | Correction in kafka-console-producer command

2020-07-10 Thread Hemant Girase (Jira)
Hemant Girase created KAFKA-10267: - Summary: [Documentation] | Correction in kafka-console-producer command Key: KAFKA-10267 URL: https://issues.apache.org/jira/browse/KAFKA-10267 Project: Kafka

[GitHub] [kafka] lbradstreet commented on pull request #9008: KAFKA-9629 Use generated protocol for Fetch API

2020-07-10 Thread GitBox
lbradstreet commented on pull request #9008: URL: https://github.com/apache/kafka/pull/9008#issuecomment-656988853 I agree that it’d be great to have a benchmark on both the request and response side. This is an automated m

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453155624 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamThreadTest.java ## @@ -1177,6 +1180,109 @@ public void shouldNotCl

[GitHub] [kafka] vvcephei commented on a change in pull request #8994: KAFKA-10247: Correctly reset state when task is corrupted

2020-07-10 Thread GitBox
vvcephei commented on a change in pull request #8994: URL: https://github.com/apache/kafka/pull/8994#discussion_r453155949 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -648,7 +651,9 @@ void runOnce() { // o