[GitHub] [kafka] dajac opened a new pull request #9903: KAFKA-12204; Implement DescribeCluster API in the broker

2021-01-15 Thread GitBox
dajac opened a new pull request #9903: URL: https://github.com/apache/kafka/pull/9903 This PR implements the DescribeCluster API in the broker. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build

[jira] [Created] (KAFKA-12210) AdminClient should use DescribeCluster API when available

2021-01-15 Thread David Jacot (Jira)
David Jacot created KAFKA-12210: --- Summary: AdminClient should use DescribeCluster API when available Key: KAFKA-12210 URL: https://issues.apache.org/jira/browse/KAFKA-12210 Project: Kafka Issue

[jira] [Created] (KAFKA-12211) NoSuchFileException will be thrown if hasPersistentStores is false when creating stateDir

2021-01-15 Thread Luke Chen (Jira)
Luke Chen created KAFKA-12211: - Summary: NoSuchFileException will be thrown if hasPersistentStores is false when creating stateDir Key: KAFKA-12211 URL: https://issues.apache.org/jira/browse/KAFKA-12211 P

[GitHub] [kafka] showuon opened a new pull request #9904: KAFKA-12211: don't change perm for base/state dir when no persistent store

2021-01-15 Thread GitBox
showuon opened a new pull request #9904: URL: https://github.com/apache/kafka/pull/9904 We improved the state directory folder/file permission setting in #9583 . But we forgot to consider one situation: if user doesn't have Persistent Stores, we won't create base dir and state dir. And if

[GitHub] [kafka] showuon commented on a change in pull request #9904: KAFKA-12211: don't change perm for base/state dir when no persistent store

2021-01-15 Thread GitBox
showuon commented on a change in pull request #9904: URL: https://github.com/apache/kafka/pull/9904#discussion_r558097187 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java ## @@ -94,27 +94,32 @@ public StateDirectory(final Str

[GitHub] [kafka] showuon commented on pull request #9904: KAFKA-12211: don't change perm for base/state dir when no persistent store

2021-01-15 Thread GitBox
showuon commented on pull request #9904: URL: https://github.com/apache/kafka/pull/9904#issuecomment-760772022 @lct45 @ableegoldman @cadonna , please help review this PR. Thanks. This is an automated message from the Apache G

[jira] [Created] (KAFKA-12212) Bump Metadata API version to remove `ClusterAuthorizedOperations` fields

2021-01-15 Thread David Jacot (Jira)
David Jacot created KAFKA-12212: --- Summary: Bump Metadata API version to remove `ClusterAuthorizedOperations` fields Key: KAFKA-12212 URL: https://issues.apache.org/jira/browse/KAFKA-12212 Project: Kafka

[jira] [Updated] (KAFKA-12211) NoSuchFileException will be thrown if hasPersistentStores is false when creating stateDir

2021-01-15 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-12211: -- Fix Version/s: (was: 2.8.0) > NoSuchFileException will be thrown if hasPersistentStores is false w

[jira] [Updated] (KAFKA-12211) NoSuchFileException will be thrown if hasPersistentStores is false when creating stateDir

2021-01-15 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-12211: -- Affects Version/s: (was: 2.8.0) 2.7.0 > NoSuchFileException will be thrown

[GitHub] [kafka] dajac opened a new pull request #9905: KAFKA-12210; AdminClient should use DescribeCluster API when available

2021-01-15 Thread GitBox
dajac opened a new pull request #9905: URL: https://github.com/apache/kafka/pull/9905 This PR updates the AdminClient to use the DescribeCluster API when available. The clients fails back to the Metadata API otherwise. ### Committer Checklist (excluded from commit message) - [ ] V

[GitHub] [kafka] g1geordie opened a new pull request #9906: KAFKA-10885 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of…

2021-01-15 Thread GitBox
g1geordie opened a new pull request #9906: URL: https://github.com/apache/kafka/pull/9906 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of ignored test cases I add different Parameter Source to prevent assume method. if method uses the source only itself

[GitHub] [kafka] g1geordie commented on pull request #9906: KAFKA-10885 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of…

2021-01-15 Thread GitBox
g1geordie commented on pull request #9906: URL: https://github.com/apache/kafka/pull/9906#issuecomment-760784911 There are no ignore test in those tests . @chia7712 can you check whether it satisfies your expectation or not ? ---

[GitHub] [kafka] cadonna commented on a change in pull request #9888: KAFKA-12194: use stateListener to catch each state change

2021-01-15 Thread GitBox
cadonna commented on a change in pull request #9888: URL: https://github.com/apache/kafka/pull/9888#discussion_r558149927 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/AdjustStreamThreadCountTest.java ## @@ -90,24 +91,61 @@ public void setup() {

[GitHub] [kafka] akamensky commented on pull request #7884: [KAFKA-8522] Streamline tombstone and transaction marker removal

2021-01-15 Thread GitBox
akamensky commented on pull request #7884: URL: https://github.com/apache/kafka/pull/7884#issuecomment-760814414 @ConcurrencyPractitioner @junrao this PR has been stale since April 2020. When would it be ready to merge? We are hitting this issue and it causes insanely long startup times in

[GitHub] [kafka] showuon commented on pull request #9733: KAFKA-10017: fix 2 issues in EosBetaUpgradeIntegrationTest

2021-01-15 Thread GitBox
showuon commented on pull request #9733: URL: https://github.com/apache/kafka/pull/9733#issuecomment-760898836 There's a jenkins timeout build error in jdk8. Made a minor change to trigger jenkins build again to make sure it's infra's issue, not my change -

[GitHub] [kafka] tombentley commented on pull request #9847: KAFKA-10703: Better handling and doc for config defaults of topics

2021-01-15 Thread GitBox
tombentley commented on pull request #9847: URL: https://github.com/apache/kafka/pull/9847#issuecomment-760914291 @chia7712 thanks for taking a look. It's true that I should have fixed this on the broker, so it works for non-Java clients too. It turns out that the three RPCs are remarkably

[jira] [Commented] (KAFKA-12199) Migrate connect:runtime module to JUnit 5

2021-01-15 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17266032#comment-17266032 ] Ismael Juma commented on KAFKA-12199: - We have already picked Mockito as our mocking

[jira] [Commented] (KAFKA-12199) Migrate connect:runtime module to JUnit 5

2021-01-15 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17266034#comment-17266034 ] Chia-Ping Tsai commented on KAFKA-12199: Thanks for your response. I plan to fil

[GitHub] [kafka] dajac commented on a change in pull request #9430: KAFKA-5235: GetOffsetShell: support for multiple topics and consumer configuration override

2021-01-15 Thread GitBox
dajac commented on a change in pull request #9430: URL: https://github.com/apache/kafka/pull/9430#discussion_r558158660 ## File path: core/src/main/scala/kafka/tools/GetOffsetShell.scala ## @@ -33,15 +35,26 @@ object GetOffsetShell { def main(args: Array[String]): Unit = {

[GitHub] [kafka] dengziming commented on pull request #9890: KAFKA-12198: Migrate connect:json module to JUnit 5

2021-01-15 Thread GitBox
dengziming commented on pull request #9890: URL: https://github.com/apache/kafka/pull/9890#issuecomment-760942669 > @dengziming Could you fix the conflicting files? thanks! Sorry for the late reply, done! This is an au

[GitHub] [kafka] cadonna commented on a change in pull request #9508: KAFKA-10648: Add Prefix Scan support to State Stores

2021-01-15 Thread GitBox
cadonna commented on a change in pull request #9508: URL: https://github.com/apache/kafka/pull/9508#discussion_r558201109 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java ## @@ -229,6 +232,13 @@ public V delete(final K key)

[GitHub] [kafka] ijuma commented on a change in pull request #9906: KAFKA-10885 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of…

2021-01-15 Thread GitBox
ijuma commented on a change in pull request #9906: URL: https://github.com/apache/kafka/pull/9906#discussion_r558311926 ## File path: clients/src/test/java/org/apache/kafka/common/record/MemoryRecordsBuilderTest.java ## @@ -72,19 +70,28 @@ public String toString() {

[GitHub] [kafka] ijuma merged pull request #9897: MINOR: Remove incorrect comment in metadata module build

2021-01-15 Thread GitBox
ijuma merged pull request #9897: URL: https://github.com/apache/kafka/pull/9897 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] ijuma merged pull request #9896: KAFKA-12206: o.a.k.common.Uuid should implement Comparable

2021-01-15 Thread GitBox
ijuma merged pull request #9896: URL: https://github.com/apache/kafka/pull/9896 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] dengziming opened a new pull request #9907: KAFKA-12197: Migrate connect:transforms module to JUnit 5

2021-01-15 Thread GitBox
dengziming opened a new pull request #9907: URL: https://github.com/apache/kafka/pull/9907 *More detailed description of your change* As title *Summary of testing strategy (including rationale)* QA ### Committer Checklist (excluded from commit message) - [ ] Verify des

[GitHub] [kafka] ijuma commented on a change in pull request #9894: KAFKA-12202 Migrate connect:mirror module to JUnit 5

2021-01-15 Thread GitBox
ijuma commented on a change in pull request #9894: URL: https://github.com/apache/kafka/pull/9894#discussion_r558314026 ## File path: connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorSourceTaskTest.java ## @@ -58,26 +58,26 @@ public void testSerde() { pu

[GitHub] [kafka] ijuma commented on pull request #9877: KAFKA-8460: consume records right after awaitAssignment

2021-01-15 Thread GitBox
ijuma commented on pull request #9877: URL: https://github.com/apache/kafka/pull/9877#issuecomment-760952738 Not sure what the original purpose of the test was, but this changes what it's testing, right? Before producing was happening after subscription and now before. --

[GitHub] [kafka] ijuma commented on pull request #9877: KAFKA-8460: consume records right after awaitAssignment

2021-01-15 Thread GitBox
ijuma commented on pull request #9877: URL: https://github.com/apache/kafka/pull/9877#issuecomment-760953842 Can we just bump `max.poll.interval.ms`? It doesn't seem like there's any value in having that be low in this test.

[GitHub] [kafka] ijuma commented on pull request #9877: KAFKA-8460: consume records right after awaitAssignment

2021-01-15 Thread GitBox
ijuma commented on pull request #9877: URL: https://github.com/apache/kafka/pull/9877#issuecomment-760954112 Also, thanks so much for diagnosing this! Very awesome. :) This is an automated message from the Apache Git Service.

[GitHub] [kafka] ijuma commented on pull request #9872: KAFKA-10759: ARM support for Kafka

2021-01-15 Thread GitBox
ijuma commented on pull request #9872: URL: https://github.com/apache/kafka/pull/9872#issuecomment-760954971 Do we have to run the full test suite or would it be sufficient to run the unit tests only? This is an automated me

[GitHub] [kafka] ijuma commented on a change in pull request #9865: Fill in the 2.8 release note for Authorizer

2021-01-15 Thread GitBox
ijuma commented on a change in pull request #9865: URL: https://github.com/apache/kafka/pull/9865#discussion_r558321382 ## File path: docs/upgrade.html ## @@ -20,6 +20,28 @@

[GitHub] [kafka] dengziming commented on pull request #9820: MINOR: Make data in FetchSnapshotReq and FetchSnapshotResp private

2021-01-15 Thread GitBox
dengziming commented on pull request #9820: URL: https://github.com/apache/kafka/pull/9820#issuecomment-760960679 @jsancio @hachikuji , Hi, PTAL. This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [kafka] ijuma commented on a change in pull request #9855: KAFKA-7341 Migrate core module to JUnit 5

2021-01-15 Thread GitBox
ijuma commented on a change in pull request #9855: URL: https://github.com/apache/kafka/pull/9855#discussion_r558338474 ## File path: core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala ## @@ -1228,10 +1223,10 @@ class PlaintextConsumerTest extends BaseConsume

[jira] [Created] (KAFKA-12213) Kafka Streams aggregation Initializer to accept record key

2021-01-15 Thread Piotr Fras (Jira)
Piotr Fras created KAFKA-12213: -- Summary: Kafka Streams aggregation Initializer to accept record key Key: KAFKA-12213 URL: https://issues.apache.org/jira/browse/KAFKA-12213 Project: Kafka Issue

[GitHub] [kafka] jCalamari opened a new pull request #9908: KAFKA-12213: Kafka Streams aggregation Initializer to accept record key

2021-01-15 Thread GitBox
jCalamari opened a new pull request #9908: URL: https://github.com/apache/kafka/pull/9908 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* As per title and ji

[jira] [Updated] (KAFKA-12213) Kafka Streams aggregation Initializer to accept record key

2021-01-15 Thread Piotr Fras (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Fras updated KAFKA-12213: --- Issue Type: New Feature (was: Bug) > Kafka Streams aggregation Initializer to accept record key > -

[jira] [Updated] (KAFKA-12213) Kafka Streams aggregation Initializer to accept record key

2021-01-15 Thread Piotr Fras (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Fras updated KAFKA-12213: --- Priority: Minor (was: Major) > Kafka Streams aggregation Initializer to accept record key > ---

[GitHub] [kafka] 17hao commented on pull request #9891: MINOR: Remind user index file is empty when dumping LogSegment index file

2021-01-15 Thread GitBox
17hao commented on pull request #9891: URL: https://github.com/apache/kafka/pull/9891#issuecomment-761026242 hi @chia7712 , I added an ut, PTAL. This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [kafka] cadonna commented on a change in pull request #9895: KAFKA-9924: Add docs for RocksDB properties-based metrics

2021-01-15 Thread GitBox
cadonna commented on a change in pull request #9895: URL: https://github.com/apache/kafka/pull/9895#discussion_r558404376 ## File path: docs/ops.html ## @@ -2032,17 +2032,25 @@ RocksDB Metrics - All of the following metrics have a recording level of debug. - The metrics are

[GitHub] [kafka] g1geordie commented on a change in pull request #9906: KAFKA-10885 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of…

2021-01-15 Thread GitBox
g1geordie commented on a change in pull request #9906: URL: https://github.com/apache/kafka/pull/9906#discussion_r558406657 ## File path: clients/src/test/java/org/apache/kafka/common/record/MemoryRecordsBuilderTest.java ## @@ -72,19 +70,28 @@ public String toString() {

[GitHub] [kafka] g1geordie commented on a change in pull request #9906: KAFKA-10885 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of…

2021-01-15 Thread GitBox
g1geordie commented on a change in pull request #9906: URL: https://github.com/apache/kafka/pull/9906#discussion_r558406657 ## File path: clients/src/test/java/org/apache/kafka/common/record/MemoryRecordsBuilderTest.java ## @@ -72,19 +70,28 @@ public String toString() {

[GitHub] [kafka] cadonna commented on a change in pull request #9848: KAFKA-10357: Extract setup of repartition topics from Streams partition assignor

2021-01-15 Thread GitBox
cadonna commented on a change in pull request #9848: URL: https://github.com/apache/kafka/pull/9848#discussion_r558414235 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/RepartitionTopics.java ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] dajac commented on pull request #9820: MINOR: Make data in FetchSnapshotReq and FetchSnapshotResp private

2021-01-15 Thread GitBox
dajac commented on pull request #9820: URL: https://github.com/apache/kafka/pull/9820#issuecomment-761038874 @dengziming Could you address the checkstyle issues? ``` [2021-01-15T14:11:08.434Z] [ant:checkstyle] [ERROR] /home/jenkins/workspace/Kafka_kafka-pr_PR-9820/clients/src/main/jav

[jira] [Updated] (KAFKA-12213) Kafka Streams aggregation Initializer to accept record key

2021-01-15 Thread Piotr Fras (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Piotr Fras updated KAFKA-12213: --- Description: Sometimes Kafka record key contains useful information for creating a zero object in a

[GitHub] [kafka] ijuma commented on a change in pull request #9906: KAFKA-10885 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of…

2021-01-15 Thread GitBox
ijuma commented on a change in pull request #9906: URL: https://github.com/apache/kafka/pull/9906#discussion_r558416241 ## File path: clients/src/test/java/org/apache/kafka/common/record/MemoryRecordsBuilderTest.java ## @@ -72,19 +70,28 @@ public String toString() {

[GitHub] [kafka] cadonna commented on a change in pull request #9848: KAFKA-10357: Extract setup of repartition topics from Streams partition assignor

2021-01-15 Thread GitBox
cadonna commented on a change in pull request #9848: URL: https://github.com/apache/kafka/pull/9848#discussion_r558419237 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java ## @@ -471,135 +470,22 @@ private boolean ch

[jira] [Assigned] (KAFKA-12193) Re-resolve IPs when a client is disconnected

2021-01-15 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot reassigned KAFKA-12193: --- Assignee: Bob Barrett > Re-resolve IPs when a client is disconnected >

[GitHub] [kafka] g1geordie opened a new pull request #9909: KAFKA-12196 Migrate connect:api module to JUnit 5

2021-01-15 Thread GitBox
g1geordie opened a new pull request #9909: URL: https://github.com/apache/kafka/pull/9909 - change import - assertThrows replaces try-catch This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [kafka] chia7712 merged pull request #9884: MINOR: remove unused flag 'hasIdempotentRecords'

2021-01-15 Thread GitBox
chia7712 merged pull request #9884: URL: https://github.com/apache/kafka/pull/9884 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] cadonna commented on a change in pull request #9848: KAFKA-10357: Extract setup of repartition topics from Streams partition assignor

2021-01-15 Thread GitBox
cadonna commented on a change in pull request #9848: URL: https://github.com/apache/kafka/pull/9848#discussion_r558439866 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/RepartitionTopics.java ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] chia7712 commented on a change in pull request #9906: KAFKA-10885 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of…

2021-01-15 Thread GitBox
chia7712 commented on a change in pull request #9906: URL: https://github.com/apache/kafka/pull/9906#discussion_r558430765 ## File path: clients/src/test/java/org/apache/kafka/common/record/MemoryRecordsTest.java ## @@ -201,38 +218,37 @@ public void testHasRoomForMethod(Args a

[GitHub] [kafka] cadonna commented on a change in pull request #9848: KAFKA-10357: Extract setup of repartition topics from Streams partition assignor

2021-01-15 Thread GitBox
cadonna commented on a change in pull request #9848: URL: https://github.com/apache/kafka/pull/9848#discussion_r558440845 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/RepartitionTopicsTest.java ## @@ -0,0 +1,434 @@ +/* + * Licensed to the Ap

[GitHub] [kafka] chia7712 commented on a change in pull request #9855: KAFKA-7341 Migrate core module to JUnit 5

2021-01-15 Thread GitBox
chia7712 commented on a change in pull request #9855: URL: https://github.com/apache/kafka/pull/9855#discussion_r558455483 ## File path: core/src/test/scala/unit/kafka/utils/TestUtils.scala ## @@ -459,41 +459,9 @@ object TestUtils extends Logging { * Check that the buffer c

[GitHub] [kafka] hachikuji commented on a change in pull request #9871: KAFKA-12161; Support raft observers with optional id

2021-01-15 Thread GitBox
hachikuji commented on a change in pull request #9871: URL: https://github.com/apache/kafka/pull/9871#discussion_r558456316 ## File path: raft/src/main/java/org/apache/kafka/raft/QuorumState.java ## @@ -196,7 +203,15 @@ public void initialize(OffsetAndEpoch logEndOffsetAndEpoc

[jira] [Updated] (KAFKA-12190) Failure on Windows due to an UnsupportedOperationException when StateDirectory sets file permissions

2021-01-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12190: Affects Version/s: 2.7.1 > Failure on Windows due to an UnsupportedOperationException when

[jira] [Commented] (KAFKA-12190) Failure on Windows due to an UnsupportedOperationException when StateDirectory sets file permissions

2021-01-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17266222#comment-17266222 ] Matthias J. Sax commented on KAFKA-12190: - Note that `2.7.0` release is not affe

[jira] [Updated] (KAFKA-12190) Failure on Windows due to an UnsupportedOperationException when StateDirectory sets file permissions

2021-01-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12190: Fix Version/s: 2.7.1 > Failure on Windows due to an UnsupportedOperationException when >

[GitHub] [kafka] g1geordie commented on pull request #9906: KAFKA-10885 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of…

2021-01-15 Thread GitBox
g1geordie commented on pull request #9906: URL: https://github.com/apache/kafka/pull/9906#issuecomment-761078614 @chia7712 thanks for all suggestions This is an automated message from the Apache Git Service. To respond to th

[GitHub] [kafka] guozhangwang commented on pull request #9681: MINOR: Fix flaky test shouldQueryOnlyActivePartitionStoresByDefault

2021-01-15 Thread GitBox
guozhangwang commented on pull request #9681: URL: https://github.com/apache/kafka/pull/9681#issuecomment-761078960 Cherry-picked to 2.7 This is an automated message from the Apache Git Service. To respond to the message, ple

[jira] [Updated] (KAFKA-12190) Failure on Windows due to an UnsupportedOperationException when StateDirectory sets file permissions

2021-01-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12190: Fix Version/s: 2.6.2 > Failure on Windows due to an UnsupportedOperationException when >

[GitHub] [kafka] guozhangwang commented on pull request #9873: MINOR: Fix flaky test shouldQuerySpecificActivePartitionStores

2021-01-15 Thread GitBox
guozhangwang commented on pull request #9873: URL: https://github.com/apache/kafka/pull/9873#issuecomment-761079743 Cherry-picked to 2.7 This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [kafka] chia7712 commented on a change in pull request #9855: KAFKA-7341 Migrate core module to JUnit 5

2021-01-15 Thread GitBox
chia7712 commented on a change in pull request #9855: URL: https://github.com/apache/kafka/pull/9855#discussion_r558465841 ## File path: core/src/test/scala/unit/kafka/server/AlterUserScramCredentialsRequestTest.scala ## @@ -43,20 +41,27 @@ import scala.jdk.CollectionConverter

[GitHub] [kafka] smccauliff opened a new pull request #9910: KAFKA-10877

2021-01-15 Thread GitBox
smccauliff opened a new pull request #9910: URL: https://github.com/apache/kafka/pull/9910 This moves the Logging trait into companion objects so the loggers are only instantiated once. Classes import the methods from their companion objects. Classes that were using inheritance to gain a

[jira] [Commented] (KAFKA-10877) Instantiating loggers for every FetchContext causes low request handler idle pool ratio.

2021-01-15 Thread Sean McCauliff (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17266247#comment-17266247 ] Sean McCauliff commented on KAFKA-10877: PR added > Instantiating loggers for e

[GitHub] [kafka] chia7712 commented on a change in pull request #9855: KAFKA-7341 Migrate core module to JUnit 5

2021-01-15 Thread GitBox
chia7712 commented on a change in pull request #9855: URL: https://github.com/apache/kafka/pull/9855#discussion_r558473729 ## File path: core/src/test/scala/unit/kafka/server/FetchRequestMaxBytesTest.scala ## @@ -58,11 +58,13 @@ class FetchRequestMaxBytesTest extends BaseReques

[GitHub] [kafka] smccauliff commented on pull request #9910: KAFKA-10877

2021-01-15 Thread GitBox
smccauliff commented on pull request #9910: URL: https://github.com/apache/kafka/pull/9910#issuecomment-761096689 @gitlw This is the more comprehensive change I was discussing with you. This is an automated message from the A

[GitHub] [kafka] ijuma commented on a change in pull request #9871: KAFKA-12161; Support raft observers with optional id

2021-01-15 Thread GitBox
ijuma commented on a change in pull request #9871: URL: https://github.com/apache/kafka/pull/9871#discussion_r558481270 ## File path: raft/src/main/java/org/apache/kafka/raft/QuorumState.java ## @@ -196,7 +203,15 @@ public void initialize(OffsetAndEpoch logEndOffsetAndEpoch) t

[GitHub] [kafka] hachikuji commented on a change in pull request #9600: KAFKA-10674: Controller API version bond with forwardable APIs

2021-01-15 Thread GitBox
hachikuji commented on a change in pull request #9600: URL: https://github.com/apache/kafka/pull/9600#discussion_r558469889 ## File path: clients/src/test/java/org/apache/kafka/clients/NodeApiVersionsTest.java ## @@ -125,13 +126,6 @@ public void testUsableVersionCalculationNoK

[GitHub] [kafka] ijuma commented on a change in pull request #9906: KAFKA-10885 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of…

2021-01-15 Thread GitBox
ijuma commented on a change in pull request #9906: URL: https://github.com/apache/kafka/pull/9906#discussion_r558484400 ## File path: clients/src/test/java/org/apache/kafka/common/record/MemoryRecordsBuilderTest.java ## @@ -72,19 +70,28 @@ public String toString() {

[GitHub] [kafka] abbccdda commented on a change in pull request #9600: KAFKA-10674: Controller API version bond with forwardable APIs

2021-01-15 Thread GitBox
abbccdda commented on a change in pull request #9600: URL: https://github.com/apache/kafka/pull/9600#discussion_r558484623 ## File path: clients/src/test/java/org/apache/kafka/clients/NodeApiVersionsTest.java ## @@ -125,13 +126,6 @@ public void testUsableVersionCalculationNoKn

[GitHub] [kafka] ijuma commented on a change in pull request #9906: KAFKA-10885 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of…

2021-01-15 Thread GitBox
ijuma commented on a change in pull request #9906: URL: https://github.com/apache/kafka/pull/9906#discussion_r558486354 ## File path: clients/src/test/java/org/apache/kafka/common/record/MemoryRecordsBuilderTest.java ## @@ -72,19 +70,28 @@ public String toString() {

[GitHub] [kafka] hachikuji commented on pull request #9819: [WIP] KAFKA-10694: Implement zero copy for FetchSnapshot

2021-01-15 Thread GitBox
hachikuji commented on pull request #9819: URL: https://github.com/apache/kafka/pull/9819#issuecomment-761119104 @dengziming @jsancio I'll give my take what I think we should do here, but let's try to agree before we go any further. Basically I am not really convinced that we need a new ty

[GitHub] [kafka] hachikuji commented on a change in pull request #9600: KAFKA-10674: Controller API version bond with forwardable APIs

2021-01-15 Thread GitBox
hachikuji commented on a change in pull request #9600: URL: https://github.com/apache/kafka/pull/9600#discussion_r558529073 ## File path: core/src/test/scala/unit/kafka/server/AbstractApiVersionsRequestTest.scala ## @@ -25,6 +27,15 @@ import scala.jdk.CollectionConverters._

[GitHub] [kafka] chia7712 merged pull request #9890: KAFKA-12198: Migrate connect:json module to JUnit 5

2021-01-15 Thread GitBox
chia7712 merged pull request #9890: URL: https://github.com/apache/kafka/pull/9890 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] chia7712 commented on pull request #9909: KAFKA-12196 Migrate connect:api module to JUnit 5

2021-01-15 Thread GitBox
chia7712 commented on pull request #9909: URL: https://github.com/apache/kafka/pull/9909#issuecomment-761135068 @g1geordie Please fix conflicting files :) This is an automated message from the Apache Git Service. To respond t

[GitHub] [kafka] mumrah commented on pull request #9715: Upstream ApisUtils from kip-500

2021-01-15 Thread GitBox
mumrah commented on pull request #9715: URL: https://github.com/apache/kafka/pull/9715#issuecomment-761171619 The test failures were due to unhappy mocks in KafkaApisTest. It looks rather tedious to fix the mocks so I ended up moving two of the methods I had put into RequestChannel back in

[GitHub] [kafka] vvcephei opened a new pull request #9911: MINOR: Fix StreamsOptimizedTest

2021-01-15 Thread GitBox
vvcephei opened a new pull request #9911: URL: https://github.com/apache/kafka/pull/9911 We have seen recent system test timeouts associated with this test. Analysis revealed an excessive amount of time spent searching for test conditions in the logs. This change addresses the i

[GitHub] [kafka] vvcephei commented on pull request #9911: MINOR: Fix StreamsOptimizedTest

2021-01-15 Thread GitBox
vvcephei commented on pull request #9911: URL: https://github.com/apache/kafka/pull/9911#issuecomment-761176712 In detail, here's my analysis of a recent timeout: Looking at the debug log for the test, there seem to be some remote commands that really burn the clock. Consider this se

[jira] [Updated] (KAFKA-12211) NoSuchFileException will be thrown if hasPersistentStores is false when creating stateDir

2021-01-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12211: Component/s: streams > NoSuchFileException will be thrown if hasPersistentStores is false

[jira] [Updated] (KAFKA-12213) Kafka Streams aggregation Initializer to accept record key

2021-01-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12213: Labels: needs-kip (was: ) > Kafka Streams aggregation Initializer to accept record key >

[GitHub] [kafka] vvcephei commented on a change in pull request #9911: MINOR: Fix StreamsOptimizedTest

2021-01-15 Thread GitBox
vvcephei commented on a change in pull request #9911: URL: https://github.com/apache/kafka/pull/9911#discussion_r558572032 ## File path: tests/kafkatest/tests/streams/streams_optimized_test.py ## @@ -65,30 +66,41 @@ def test_upgrade_optimized_topology(self): processo

[GitHub] [kafka] vvcephei commented on pull request #9911: MINOR: Fix StreamsOptimizedTest

2021-01-15 Thread GitBox
vvcephei commented on pull request #9911: URL: https://github.com/apache/kafka/pull/9911#issuecomment-761180464 I confirmed this change works both positively and negatively: ``` SESSION REPORT (ALL

[jira] [Commented] (KAFKA-12213) Kafka Streams aggregation Initializer to accept record key

2021-01-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17266335#comment-17266335 ] Matthias J. Sax commented on KAFKA-12213: - [~MonCalamari] – thanks for creating

[GitHub] [kafka] ijuma merged pull request #9715: Upstream ApisUtils from kip-500

2021-01-15 Thread GitBox
ijuma merged pull request #9715: URL: https://github.com/apache/kafka/pull/9715 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] mjsax commented on pull request #9908: KAFKA-12213: Kafka Streams aggregation Initializer to accept record key

2021-01-15 Thread GitBox
mjsax commented on pull request #9908: URL: https://github.com/apache/kafka/pull/9908#issuecomment-761181578 Thanks for the PR @jCalamari - I left a more detailed comment on the Jira ticket. We cannot just merge public API changes without a KIP. There is KIP-149 and KIP-159 that overlap w

[GitHub] [kafka] mjsax edited a comment on pull request #9908: KAFKA-12213: Kafka Streams aggregation Initializer to accept record key

2021-01-15 Thread GitBox
mjsax edited a comment on pull request #9908: URL: https://github.com/apache/kafka/pull/9908#issuecomment-761181578 Thanks for the PR @jCalamari - I left a more detailed comment on the Jira ticket. We cannot just merge public API changes without a KIP. (Also your PR would be provide backw

[GitHub] [kafka] mjsax edited a comment on pull request #9908: KAFKA-12213: Kafka Streams aggregation Initializer to accept record key

2021-01-15 Thread GitBox
mjsax edited a comment on pull request #9908: URL: https://github.com/apache/kafka/pull/9908#issuecomment-761181578 Thanks for the PR @jCalamari - I left a more detailed comment on the Jira ticket. We cannot just merge public API changes without a KIP. (Also your PR would not provide API

[GitHub] [kafka] jsancio commented on pull request #9819: [WIP] KAFKA-10694: Implement zero copy for FetchSnapshot

2021-01-15 Thread GitBox
jsancio commented on pull request #9819: URL: https://github.com/apache/kafka/pull/9819#issuecomment-761183234 Replying to https://github.com/apache/kafka/pull/9819#issuecomment-761119104 Thanks @hachikuji for thinking about this and for giving a detail reply. For the long term solut

[GitHub] [kafka] vvcephei commented on pull request #9911: MINOR: Fix StreamsOptimizedTest

2021-01-15 Thread GitBox
vvcephei commented on pull request #9911: URL: https://github.com/apache/kafka/pull/9911#issuecomment-761183334 Hey @bbejeck , I don't suppose you have time for a review of this, since you had come context? This is an automa

[GitHub] [kafka] cmccabe merged pull request #9900: KAFKA-12208: Rename AdminManager to ZkAdminManager

2021-01-15 Thread GitBox
cmccabe merged pull request #9900: URL: https://github.com/apache/kafka/pull/9900 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[jira] [Resolved] (KAFKA-12208) Rename AdminManager to ZkAdminManager

2021-01-15 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-12208. -- Fix Version/s: 2.8.0 Resolution: Fixed > Rename AdminManager to ZkAdminManager > --

[jira] [Assigned] (KAFKA-12208) Rename AdminManager to ZkAdminManager

2021-01-15 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe reassigned KAFKA-12208: Assignee: Colin McCabe > Rename AdminManager to ZkAdminManager >

[GitHub] [kafka] cmccabe commented on a change in pull request #9883: MINOR: Generalize server startup to make way for KIP-500

2021-01-15 Thread GitBox
cmccabe commented on a change in pull request #9883: URL: https://github.com/apache/kafka/pull/9883#discussion_r558590756 ## File path: core/src/main/scala/kafka/server/KafkaConfig.scala ## @@ -1453,6 +1456,23 @@ class KafkaConfig(val props: java.util.Map[_, _], doLog: Boolean

[GitHub] [kafka] cmccabe commented on pull request #9883: MINOR: Generalize server startup to make way for KIP-500

2021-01-15 Thread GitBox
cmccabe commented on pull request #9883: URL: https://github.com/apache/kafka/pull/9883#issuecomment-761195905 LGTM. I had one small comment. Thanks, @hachikuji This is an automated message from the Apache Git Servic

[GitHub] [kafka] hachikuji opened a new pull request #9912: MINOR: Let TestRaftRequestHandler use RequestHandlerHelper

2021-01-15 Thread GitBox
hachikuji opened a new pull request #9912: URL: https://github.com/apache/kafka/pull/9912 Since we factored out `RequestHandlerHelper`, we may as well make use of it in `TestRaftRequestHandler`. ### Committer Checklist (excluded from commit message) - [ ] Verify design and impleme

[GitHub] [kafka] ableegoldman commented on a change in pull request #9848: KAFKA-10357: Extract setup of repartition topics from Streams partition assignor

2021-01-15 Thread GitBox
ableegoldman commented on a change in pull request #9848: URL: https://github.com/apache/kafka/pull/9848#discussion_r558592503 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java ## @@ -471,135 +470,22 @@ private boole

[GitHub] [kafka] ableegoldman commented on pull request #9848: KAFKA-10357: Extract setup of repartition topics from Streams partition assignor

2021-01-15 Thread GitBox
ableegoldman commented on pull request #9848: URL: https://github.com/apache/kafka/pull/9848#issuecomment-761198037 @cadonna looks like `checkstyleTest` is failing This is an automated message from the Apache Git Service. To

[GitHub] [kafka] ableegoldman commented on a change in pull request #9895: KAFKA-9924: Add docs for RocksDB properties-based metrics

2021-01-15 Thread GitBox
ableegoldman commented on a change in pull request #9895: URL: https://github.com/apache/kafka/pull/9895#discussion_r558598465 ## File path: docs/ops.html ## @@ -2032,17 +2032,26 @@ RocksDB Metrics - All of the following metrics have a recording level of debug. - The metrics

[jira] [Created] (KAFKA-12214) Generated code does not include UUID or struct fields in its toString output

2021-01-15 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-12214: Summary: Generated code does not include UUID or struct fields in its toString output Key: KAFKA-12214 URL: https://issues.apache.org/jira/browse/KAFKA-12214 Project:

[GitHub] [kafka] jsancio commented on a change in pull request #9881: MINOR: Initialize QuorumState lazily in RaftClient.initialize()

2021-01-15 Thread GitBox
jsancio commented on a change in pull request #9881: URL: https://github.com/apache/kafka/pull/9881#discussion_r558601210 ## File path: core/src/main/scala/kafka/raft/RaftManager.scala ## @@ -168,30 +166,20 @@ class KafkaRaftManager[T]( } private def buildRaftClient():

  1   2   >