[GitHub] [kafka] tang7526 opened a new pull request #10012: MINOR: Refactor return statement and log info

2021-02-01 Thread GitBox
tang7526 opened a new pull request #10012: URL: https://github.com/apache/kafka/pull/10012 - Invoke method only conditionally. - No need to call "toString()" method as formatting and string conversion is done by the Formatter. before ```java log.info("Successfully validate

[GitHub] [kafka] showuon commented on a change in pull request #9777: KAFKA-7940: wait until we've got the expected partition size

2021-02-01 Thread GitBox
showuon commented on a change in pull request #9777: URL: https://github.com/apache/kafka/pull/9777#discussion_r567650065 ## File path: core/src/test/scala/unit/kafka/utils/TestUtils.scala ## @@ -885,6 +891,24 @@ object TestUtils extends Logging { ), "Timed out waiting for

[GitHub] [kafka] showuon commented on pull request #9777: KAFKA-7940: wait until we've got the expected partition size

2021-02-01 Thread GitBox
showuon commented on pull request #9777: URL: https://github.com/apache/kafka/pull/9777#issuecomment-770691251 @hachikuji , thanks for the comments. I've updated: 1. rename the method: `waitUntilMetadataIsPropagatedWithExpectedSize` -> `waitForAllPartitionMetadata` `waitUntilMetada

[GitHub] [kafka] showuon commented on a change in pull request #10010: MINOR: AbstractConfigTest.testClassConfigs should reset the class loa…

2021-02-01 Thread GitBox
showuon commented on a change in pull request #10010: URL: https://github.com/apache/kafka/pull/10010#discussion_r567662275 ## File path: clients/src/test/java/org/apache/kafka/common/config/AbstractConfigTest.java ## @@ -276,52 +277,46 @@ public RestrictedClassLoader() {

[GitHub] [kafka] showuon commented on a change in pull request #10010: MINOR: AbstractConfigTest.testClassConfigs should reset the class loa…

2021-02-01 Thread GitBox
showuon commented on a change in pull request #10010: URL: https://github.com/apache/kafka/pull/10010#discussion_r567662357 ## File path: clients/src/test/java/org/apache/kafka/common/config/AbstractConfigTest.java ## @@ -276,52 +277,46 @@ public RestrictedClassLoader() {

[GitHub] [kafka] showuon commented on a change in pull request #10010: MINOR: AbstractConfigTest.testClassConfigs should reset the class loa…

2021-02-01 Thread GitBox
showuon commented on a change in pull request #10010: URL: https://github.com/apache/kafka/pull/10010#discussion_r567662443 ## File path: clients/src/test/java/org/apache/kafka/common/config/AbstractConfigTest.java ## @@ -276,52 +277,46 @@ public RestrictedClassLoader() {

[GitHub] [kafka] chia7712 commented on a change in pull request #9769: KAFKA-10774; Support Describe topic using topic IDs

2021-02-01 Thread GitBox
chia7712 commented on a change in pull request #9769: URL: https://github.com/apache/kafka/pull/9769#discussion_r567655022 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/TopicListing.java ## @@ -17,24 +17,36 @@ package org.apache.kafka.clients.admin; +

[GitHub] [kafka] chia7712 commented on pull request #10010: MINOR: AbstractConfigTest.testClassConfigs should reset the class loa…

2021-02-01 Thread GitBox
chia7712 commented on pull request #10010: URL: https://github.com/apache/kafka/pull/10010#issuecomment-770703994 @showuon Thanks for your reviews. I have addressed your comment by https://github.com/apache/kafka/pull/10010/commits/dac031f228c8e8db02de2f17e14f43a96db11b8e

[jira] [Assigned] (KAFKA-12257) Consumer mishandles topics deleted and recreated with the same name

2021-02-01 Thread lqjacklee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lqjacklee reassigned KAFKA-12257: - Assignee: lqjacklee > Consumer mishandles topics deleted and recreated with the same name > --

[GitHub] [kafka] rajinisivaram commented on pull request #9769: KAFKA-10774; Support Describe topic using topic IDs

2021-02-01 Thread GitBox
rajinisivaram commented on pull request #9769: URL: https://github.com/apache/kafka/pull/9769#issuecomment-770724994 @dengziming Have you tried any system tests with this PR? @jolshan ran a system tests build on this branch twice and found quite a few tests had failed. It is not clear whet

[GitHub] [kafka] dengziming commented on a change in pull request #9769: KAFKA-10774; Support Describe topic using topic IDs

2021-02-01 Thread GitBox
dengziming commented on a change in pull request #9769: URL: https://github.com/apache/kafka/pull/9769#discussion_r567718317 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -1191,8 +1192,35 @@ class KafkaApis(val requestChannel: RequestChannel, val met

[GitHub] [kafka] dengziming commented on a change in pull request #9769: KAFKA-10774; Support Describe topic using topic IDs

2021-02-01 Thread GitBox
dengziming commented on a change in pull request #9769: URL: https://github.com/apache/kafka/pull/9769#discussion_r567719703 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -1191,8 +1192,35 @@ class KafkaApis(val requestChannel: RequestChannel, val met

[GitHub] [kafka] dengziming commented on pull request #9769: KAFKA-10774; Support Describe topic using topic IDs

2021-02-01 Thread GitBox
dengziming commented on pull request #9769: URL: https://github.com/apache/kafka/pull/9769#issuecomment-770761700 @rajinisivaram I haven't run any system tests and I will try to run some system tests later. This is an autom

[GitHub] [kafka] dengziming removed a comment on pull request #9769: KAFKA-10774; Support Describe topic using topic IDs

2021-02-01 Thread GitBox
dengziming removed a comment on pull request #9769: URL: https://github.com/apache/kafka/pull/9769#issuecomment-770761700 @rajinisivaram I haven't run any system tests and I will try to run some system tests later. This is

[GitHub] [kafka] dengziming commented on pull request #9769: KAFKA-10774; Support Describe topic using topic IDs

2021-02-01 Thread GitBox
dengziming commented on pull request #9769: URL: https://github.com/apache/kafka/pull/9769#issuecomment-770763376 @rajinisivaram I haven't run any system tests and I will try to run some system tests later. Should I run the 2 python script locally and analyze the logs? --

[GitHub] [kafka] rajinisivaram commented on pull request #9769: KAFKA-10774; Support Describe topic using topic IDs

2021-02-01 Thread GitBox
rajinisivaram commented on pull request #9769: URL: https://github.com/apache/kafka/pull/9769#issuecomment-770860477 @dengziming Yes, if you are able to run those two tests locally, that will be good. The failures looked similar with consumer failing to consume, so I guess may be only one

[GitHub] [kafka] showuon commented on a change in pull request #9777: KAFKA-7940: wait until we've got the expected partition size

2021-02-01 Thread GitBox
showuon commented on a change in pull request #9777: URL: https://github.com/apache/kafka/pull/9777#discussion_r567837343 ## File path: core/src/test/scala/unit/kafka/admin/AddPartitionsTest.scala ## @@ -137,12 +137,12 @@ class AddPartitionsTest extends BaseRequestTest { a

[jira] [Created] (KAFKA-12262) New session keys are never distributed when follower with key becomes leader

2021-02-01 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-12262: - Summary: New session keys are never distributed when follower with key becomes leader Key: KAFKA-12262 URL: https://issues.apache.org/jira/browse/KAFKA-12262 Projec

[GitHub] [kafka] rondagostino commented on pull request #10009: MINOR: Introduce ProducerIdGenerator trait

2021-02-01 Thread GitBox
rondagostino commented on pull request #10009: URL: https://github.com/apache/kafka/pull/10009#issuecomment-770914261 Only test failure is a known flaky test: https://issues.apache.org/jira/browse/KAFKA-7940 This is an autom

[GitHub] [kafka] rondagostino opened a new pull request #10013: MINOR: Remove ZK dependency for __transaction_state partition count

2021-02-01 Thread GitBox
rondagostino opened a new pull request #10013: URL: https://github.com/apache/kafka/pull/10013 `TransactionStateManager` currently uses ZooKeeper to obtain any non-default value for the `__transaction_state` partition count. ZooKeeper will not be available when the broker uses a Raft-base

[GitHub] [kafka] rondagostino commented on pull request #10013: MINOR: Remove ZK dependency for __transaction_state partition count

2021-02-01 Thread GitBox
rondagostino commented on pull request #10013: URL: https://github.com/apache/kafka/pull/10013#issuecomment-770924558 The combination of this PR and https://github.com/apache/kafka/pull/10009/ will remove hard ZooKeeper dependencies from the `TransactionCoordinator` and `TransactionStateMa

[GitHub] [kafka] feyman2016 commented on pull request #9739: KAFKA-10636 Bypass log validation for writes to raft log

2021-02-01 Thread GitBox
feyman2016 commented on pull request #9739: URL: https://github.com/apache/kafka/pull/9739#issuecomment-770938213 Fixed the KafkaMetadataLogTest, let's see if jenkins still complains~ This is an automated message from the Apa

[GitHub] [kafka] C0urante opened a new pull request #10014: KAFKA-12252 and KAFKA-12262: Fix key rotation when leadership changes

2021-02-01 Thread GitBox
C0urante opened a new pull request #10014: URL: https://github.com/apache/kafka/pull/10014 Address [KAFKA-12252](https://issues.apache.org/jira/browse/KAFKA-12252) and [KAFKA-12262](https://issues.apache.org/jira/browse/KAFKA-12262). - To address `KAFKA-12262`, all workers now track

[GitHub] [kafka] C0urante commented on pull request #10014: KAFKA-12252 and KAFKA-12262: Fix key rotation when leadership changes

2021-02-01 Thread GitBox
C0urante commented on pull request #10014: URL: https://github.com/apache/kafka/pull/10014#issuecomment-770951057 @gharris1727 @ncliang would either of you like to take a look? This is an automated message from the Apache Git

[jira] [Updated] (KAFKA-12262) New session keys are never distributed when follower with key becomes leader

2021-02-01 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-12262: -- Component/s: KafkaConnect > New session keys are never distributed when follower with key beco

[GitHub] [kafka] rondagostino commented on a change in pull request #10008: MINOR: Remove ZK dependency for __consumer_offsets partition count

2021-02-01 Thread GitBox
rondagostino commented on a change in pull request #10008: URL: https://github.com/apache/kafka/pull/10008#discussion_r567943552 ## File path: core/src/main/scala/kafka/server/KafkaServer.scala ## @@ -318,8 +318,9 @@ class KafkaServer( /* start group coordinator */

[GitHub] [kafka] ijuma commented on a change in pull request #10013: MINOR: Remove ZK dependency for __transaction_state partition count

2021-02-01 Thread GitBox
ijuma commented on a change in pull request #10013: URL: https://github.com/apache/kafka/pull/10013#discussion_r567952139 ## File path: core/src/main/scala/kafka/coordinator/transaction/TransactionCoordinator.scala ## @@ -30,6 +30,13 @@ import org.apache.kafka.common.record.Re

[GitHub] [kafka] mumrah commented on a change in pull request #9986: JUnit extensions for integration tests

2021-02-01 Thread GitBox
mumrah commented on a change in pull request #9986: URL: https://github.com/apache/kafka/pull/9986#discussion_r567956186 ## File path: core/src/test/java/kafka/test/ClusterInstance.java ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mo

[GitHub] [kafka] mumrah commented on a change in pull request #9986: JUnit extensions for integration tests

2021-02-01 Thread GitBox
mumrah commented on a change in pull request #9986: URL: https://github.com/apache/kafka/pull/9986#discussion_r567956723 ## File path: build.gradle ## @@ -1011,7 +1011,10 @@ project(':core') { } test { java { -srcDirs = ["src/generated/java", "src/test/

[GitHub] [kafka] mumrah commented on a change in pull request #9986: JUnit extensions for integration tests

2021-02-01 Thread GitBox
mumrah commented on a change in pull request #9986: URL: https://github.com/apache/kafka/pull/9986#discussion_r567957941 ## File path: core/src/test/java/kafka/test/annotation/ClusterProperty.java ## @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] [kafka] rondagostino commented on a change in pull request #10013: MINOR: Remove ZK dependency for __transaction_state partition count

2021-02-01 Thread GitBox
rondagostino commented on a change in pull request #10013: URL: https://github.com/apache/kafka/pull/10013#discussion_r567962340 ## File path: core/src/main/scala/kafka/coordinator/transaction/TransactionCoordinator.scala ## @@ -30,6 +30,13 @@ import org.apache.kafka.common.re

[GitHub] [kafka] ijuma commented on a change in pull request #10013: MINOR: Remove ZK dependency for __transaction_state partition count

2021-02-01 Thread GitBox
ijuma commented on a change in pull request #10013: URL: https://github.com/apache/kafka/pull/10013#discussion_r567966872 ## File path: core/src/main/scala/kafka/coordinator/transaction/TransactionCoordinator.scala ## @@ -30,6 +30,13 @@ import org.apache.kafka.common.record.Re

[GitHub] [kafka] ijuma commented on a change in pull request #10013: MINOR: Remove ZK dependency for __transaction_state partition count

2021-02-01 Thread GitBox
ijuma commented on a change in pull request #10013: URL: https://github.com/apache/kafka/pull/10013#discussion_r567967281 ## File path: core/src/main/scala/kafka/coordinator/transaction/TransactionCoordinator.scala ## @@ -30,6 +30,13 @@ import org.apache.kafka.common.record.Re

[GitHub] [kafka] mumrah commented on a change in pull request #9986: JUnit extensions for integration tests

2021-02-01 Thread GitBox
mumrah commented on a change in pull request #9986: URL: https://github.com/apache/kafka/pull/9986#discussion_r567970806 ## File path: core/src/test/java/kafka/test/ClusterForEachTest.java ## @@ -0,0 +1,112 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] C0urante commented on pull request #9987: KAFKA-10895: Gracefully handle invalid JAAS configs

2021-02-01 Thread GitBox
C0urante commented on pull request #9987: URL: https://github.com/apache/kafka/pull/9987#issuecomment-770994329 Ping @kkonstantine this needs to be merged before any release with https://github.com/apache/kafka/pull/9806 is put out.

[GitHub] [kafka] C0urante edited a comment on pull request #9987: KAFKA-10895: Gracefully handle invalid JAAS configs

2021-02-01 Thread GitBox
C0urante edited a comment on pull request #9987: URL: https://github.com/apache/kafka/pull/9987#issuecomment-770994329 Ping @kkonstantine this needs to be merged before any release with the changes from https://github.com/apache/kafka/pull/9806 is put out.

[GitHub] [kafka] mumrah commented on pull request #9992: KAFKA-10759 Add ARM build stage

2021-02-01 Thread GitBox
mumrah commented on pull request #9992: URL: https://github.com/apache/kafka/pull/9992#issuecomment-770999448 Looks like one of the tests got hung up and the build timed out: ``` 16:04:09 SocketServerTest > testStagedListenerShutdownWhenConnectionQueueIsFull() STARTED 16:04:09

[jira] [Commented] (KAFKA-10340) Source connectors should report error when trying to produce records to non-existent topics instead of hanging forever

2021-02-01 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276498#comment-17276498 ] Chris Egerton commented on KAFKA-10340: --- The logging changes are certainly an impr

[jira] [Updated] (KAFKA-10340) Source connectors should report error when trying to produce records to non-existent topics instead of hanging forever

2021-02-01 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-10340: -- Fix Version/s: (was: 2.6.1) (was: 2.7.0) > Source connectors should

[jira] [Reopened] (KAFKA-10340) Source connectors should report error when trying to produce records to non-existent topics instead of hanging forever

2021-02-01 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton reopened KAFKA-10340: --- Assignee: Chris Egerton (was: Luke Chen) > Source connectors should report error when try

[GitHub] [kafka] cmccabe commented on a change in pull request #10008: MINOR: Remove ZK dependency for __consumer_offsets partition count

2021-02-01 Thread GitBox
cmccabe commented on a change in pull request #10008: URL: https://github.com/apache/kafka/pull/10008#discussion_r568009986 ## File path: core/src/main/scala/kafka/server/KafkaServer.scala ## @@ -318,8 +318,9 @@ class KafkaServer( /* start group coordinator */

[jira] [Updated] (KAFKA-12262) New session keys are never distributed when follower with key becomes leader

2021-02-01 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-12262: -- Description: The expiration time for session keys [starts at {{Long.MAX_VALUE}}|https://githu

[GitHub] [kafka] cmccabe commented on a change in pull request #10008: MINOR: Remove ZK dependency for __consumer_offsets partition count

2021-02-01 Thread GitBox
cmccabe commented on a change in pull request #10008: URL: https://github.com/apache/kafka/pull/10008#discussion_r568009986 ## File path: core/src/main/scala/kafka/server/KafkaServer.scala ## @@ -318,8 +318,9 @@ class KafkaServer( /* start group coordinator */

[GitHub] [kafka] cmccabe commented on a change in pull request #10013: MINOR: Remove ZK dependency for __transaction_state partition count

2021-02-01 Thread GitBox
cmccabe commented on a change in pull request #10013: URL: https://github.com/apache/kafka/pull/10013#discussion_r568011675 ## File path: core/src/main/scala/kafka/coordinator/transaction/TransactionCoordinator.scala ## @@ -30,6 +30,13 @@ import org.apache.kafka.common.record.

[GitHub] [kafka] hachikuji commented on a change in pull request #10006: KAFKA-12245; Fix flaky `FetcherTest.testEarlierOffsetResetArrivesLate`

2021-02-01 Thread GitBox
hachikuji commented on a change in pull request #10006: URL: https://github.com/apache/kafka/pull/10006#discussion_r568013407 ## File path: clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java ## @@ -1745,51 +1742,78 @@ public void testSeekWithInF

[jira] [Created] (KAFKA-12263) Improve MockClient RequestMatcher interface

2021-02-01 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-12263: --- Summary: Improve MockClient RequestMatcher interface Key: KAFKA-12263 URL: https://issues.apache.org/jira/browse/KAFKA-12263 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-12263) Improve MockClient RequestMatcher interface

2021-02-01 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-12263: Description: MockClient has a RequestMatcher interface which is used to verify that a req

[GitHub] [kafka] cmccabe commented on pull request #10009: MINOR: Introduce ProducerIdGenerator trait

2021-02-01 Thread GitBox
cmccabe commented on pull request #10009: URL: https://github.com/apache/kafka/pull/10009#issuecomment-771029381 This one looks good to me... This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [kafka] cmccabe commented on a change in pull request #9901: KAFKA-12209: Add the timeline data structures for the KIP-631 controller

2021-02-01 Thread GitBox
cmccabe commented on a change in pull request #9901: URL: https://github.com/apache/kafka/pull/9901#discussion_r568027824 ## File path: metadata/src/main/java/org/apache/kafka/timeline/SnapshottableHashTable.java ## @@ -0,0 +1,449 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] [kafka] cmccabe commented on a change in pull request #9901: KAFKA-12209: Add the timeline data structures for the KIP-631 controller

2021-02-01 Thread GitBox
cmccabe commented on a change in pull request #9901: URL: https://github.com/apache/kafka/pull/9901#discussion_r568028002 ## File path: metadata/src/main/java/org/apache/kafka/timeline/SnapshottableHashTable.java ## @@ -0,0 +1,449 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] [kafka] cmccabe commented on a change in pull request #9901: KAFKA-12209: Add the timeline data structures for the KIP-631 controller

2021-02-01 Thread GitBox
cmccabe commented on a change in pull request #9901: URL: https://github.com/apache/kafka/pull/9901#discussion_r568028493 ## File path: metadata/src/main/java/org/apache/kafka/timeline/SnapshottableHashTable.java ## @@ -0,0 +1,449 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] [kafka] cmccabe commented on a change in pull request #9901: KAFKA-12209: Add the timeline data structures for the KIP-631 controller

2021-02-01 Thread GitBox
cmccabe commented on a change in pull request #9901: URL: https://github.com/apache/kafka/pull/9901#discussion_r568029191 ## File path: metadata/src/main/java/org/apache/kafka/timeline/SnapshottableHashTable.java ## @@ -0,0 +1,449 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] [kafka] cmccabe commented on a change in pull request #9901: KAFKA-12209: Add the timeline data structures for the KIP-631 controller

2021-02-01 Thread GitBox
cmccabe commented on a change in pull request #9901: URL: https://github.com/apache/kafka/pull/9901#discussion_r568030332 ## File path: metadata/src/main/java/org/apache/kafka/timeline/SnapshotRegistry.java ## @@ -0,0 +1,247 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [kafka] cmccabe commented on a change in pull request #9901: KAFKA-12209: Add the timeline data structures for the KIP-631 controller

2021-02-01 Thread GitBox
cmccabe commented on a change in pull request #9901: URL: https://github.com/apache/kafka/pull/9901#discussion_r568030679 ## File path: metadata/src/main/java/org/apache/kafka/timeline/SnapshotRegistry.java ## @@ -0,0 +1,247 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [kafka] hachikuji merged pull request #9739: KAFKA-10636 Bypass log validation for writes to raft log

2021-02-01 Thread GitBox
hachikuji merged pull request #9739: URL: https://github.com/apache/kafka/pull/9739 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Resolved] (KAFKA-10636) Bypass log validation for writes to raft log

2021-02-01 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-10636. - Resolution: Fixed > Bypass log validation for writes to raft log > -

[GitHub] [kafka] aloknnikhil commented on a change in pull request #9994: KAFKA-12248: Add BrokerHeartbeat/BrokerRegistration RPCs for KIP-500

2021-02-01 Thread GitBox
aloknnikhil commented on a change in pull request #9994: URL: https://github.com/apache/kafka/pull/9994#discussion_r568033553 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/ApiKeys.java ## @@ -98,7 +98,9 @@ UPDATE_FEATURES(ApiMessageType.UPDATE_FEATUR

[GitHub] [kafka] jolshan commented on a change in pull request #9944: KAFKA-10580: Add topic ID support to Fetch request

2021-02-01 Thread GitBox
jolshan commented on a change in pull request #9944: URL: https://github.com/apache/kafka/pull/9944#discussion_r568033865 ## File path: clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java ## @@ -358,6 +452,15 @@ public int sessionId() { .

[GitHub] [kafka] aloknnikhil commented on a change in pull request #9994: KAFKA-12248: Add BrokerHeartbeat/BrokerRegistration RPCs for KIP-500

2021-02-01 Thread GitBox
aloknnikhil commented on a change in pull request #9994: URL: https://github.com/apache/kafka/pull/9994#discussion_r568033646 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -232,6 +232,9 @@ class KafkaApis(val requestChannel: RequestChannel, case

[GitHub] [kafka] jolshan commented on a change in pull request #9944: KAFKA-10580: Add topic ID support to Fetch request

2021-02-01 Thread GitBox
jolshan commented on a change in pull request #9944: URL: https://github.com/apache/kafka/pull/9944#discussion_r568034891 ## File path: clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java ## @@ -358,6 +452,15 @@ public int sessionId() { .

[GitHub] [kafka] hachikuji merged pull request #9777: KAFKA-7940: wait until we've got the expected partition size

2021-02-01 Thread GitBox
hachikuji merged pull request #9777: URL: https://github.com/apache/kafka/pull/9777 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] rondagostino commented on a change in pull request #10013: MINOR: Remove ZK dependency for __transaction_state partition count

2021-02-01 Thread GitBox
rondagostino commented on a change in pull request #10013: URL: https://github.com/apache/kafka/pull/10013#discussion_r568046183 ## File path: core/src/main/scala/kafka/coordinator/transaction/TransactionCoordinator.scala ## @@ -30,6 +30,13 @@ import org.apache.kafka.common.re

[GitHub] [kafka] rondagostino closed pull request #10013: MINOR: Remove ZK dependency for __transaction_state partition count

2021-02-01 Thread GitBox
rondagostino closed pull request #10013: URL: https://github.com/apache/kafka/pull/10013 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [kafka] rondagostino commented on pull request #10013: MINOR: Remove ZK dependency for __transaction_state partition count

2021-02-01 Thread GitBox
rondagostino commented on pull request #10013: URL: https://github.com/apache/kafka/pull/10013#issuecomment-771063821 Closing this PR -- I consolidated these changes into https://github.com/apache/kafka/pull/10008 so we can review them together. ---

[jira] [Created] (KAFKA-12264) JMH BaseRecordBatchBenchmark ZSTD error

2021-02-01 Thread Jeff Kim (Jira)
Jeff Kim created KAFKA-12264: Summary: JMH BaseRecordBatchBenchmark ZSTD error Key: KAFKA-12264 URL: https://issues.apache.org/jira/browse/KAFKA-12264 Project: Kafka Issue Type: Bug R

[GitHub] [kafka] junrao commented on a change in pull request #9901: KAFKA-12209: Add the timeline data structures for the KIP-631 controller

2021-02-01 Thread GitBox
junrao commented on a change in pull request #9901: URL: https://github.com/apache/kafka/pull/9901#discussion_r568039786 ## File path: metadata/src/main/java/org/apache/kafka/timeline/SnapshotRegistry.java ## @@ -0,0 +1,247 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] aloknnikhil commented on pull request #9994: KAFKA-12248: Add BrokerHeartbeat/BrokerRegistration RPCs for KIP-500

2021-02-01 Thread GitBox
aloknnikhil commented on pull request #9994: URL: https://github.com/apache/kafka/pull/9994#issuecomment-771079401 > Left a couple comments. You will need to update RequestConvertToJson as well. Probably we should have some tests in RequestResponseTest as well. Done. -

[GitHub] [kafka] aloknnikhil commented on a change in pull request #9996: KAFKA-12249: KIP-500 - Add client-side Decommission Broker RPC

2021-02-01 Thread GitBox
aloknnikhil commented on a change in pull request #9996: URL: https://github.com/apache/kafka/pull/9996#discussion_r568065511 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/ApiKeys.java ## @@ -98,7 +98,8 @@ UPDATE_FEATURES(ApiMessageType.UPDATE_FEATUR

[GitHub] [kafka] vvcephei commented on a change in pull request #9997: KAFKA-9274: Add timeout handling for `StreamPartitioner`

2021-02-01 Thread GitBox
vvcephei commented on a change in pull request #9997: URL: https://github.com/apache/kafka/pull/9997#discussion_r568067987 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/RecordCollectorImpl.java ## @@ -111,13 +112,16 @@ public void initialize(

[GitHub] [kafka] aloknnikhil commented on a change in pull request #9996: KAFKA-12249: KIP-500 - Add client-side Decommission Broker RPC

2021-02-01 Thread GitBox
aloknnikhil commented on a change in pull request #9996: URL: https://github.com/apache/kafka/pull/9996#discussion_r568069107 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/Admin.java ## @@ -1449,6 +1449,39 @@ default DescribeFeaturesResult describeFeatures(

[GitHub] [kafka] aloknnikhil commented on a change in pull request #9996: KAFKA-12249: KIP-500 - Add client-side Decommission Broker RPC

2021-02-01 Thread GitBox
aloknnikhil commented on a change in pull request #9996: URL: https://github.com/apache/kafka/pull/9996#discussion_r568070217 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java ## @@ -4503,6 +4506,47 @@ void handleFailure(Throwable throwab

[GitHub] [kafka] aloknnikhil commented on a change in pull request #9996: KAFKA-12249: KIP-500 - Add client-side Decommission Broker RPC

2021-02-01 Thread GitBox
aloknnikhil commented on a change in pull request #9996: URL: https://github.com/apache/kafka/pull/9996#discussion_r568072205 ## File path: clients/src/main/resources/common/message/DecommissionBrokerResponse.json ## @@ -0,0 +1,28 @@ +// Licensed to the Apache Software Foundat

[GitHub] [kafka] aloknnikhil commented on a change in pull request #9996: KAFKA-12249: KIP-500 - Add client-side Decommission Broker RPC

2021-02-01 Thread GitBox
aloknnikhil commented on a change in pull request #9996: URL: https://github.com/apache/kafka/pull/9996#discussion_r568074009 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -3296,6 +3298,13 @@ class KafkaApis(val requestChannel: RequestChannel, handle

[GitHub] [kafka] jolshan commented on a change in pull request #9944: KAFKA-10580: Add topic ID support to Fetch request

2021-02-01 Thread GitBox
jolshan commented on a change in pull request #9944: URL: https://github.com/apache/kafka/pull/9944#discussion_r568078414 ## File path: clients/src/main/java/org/apache/kafka/clients/MetadataCache.java ## @@ -130,13 +149,30 @@ MetadataCache mergeWith(String newClusterId,

[GitHub] [kafka] vvcephei commented on a change in pull request #9997: KAFKA-9274: Add timeout handling for `StreamPartitioner`

2021-02-01 Thread GitBox
vvcephei commented on a change in pull request #9997: URL: https://github.com/apache/kafka/pull/9997#discussion_r568069417 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/RecordCollectorImpl.java ## @@ -111,13 +112,16 @@ public void initialize(

[GitHub] [kafka] hachikuji merged pull request #10006: KAFKA-12245; Fix flaky `FetcherTest.testEarlierOffsetResetArrivesLate`

2021-02-01 Thread GitBox
hachikuji merged pull request #10006: URL: https://github.com/apache/kafka/pull/10006 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[jira] [Resolved] (KAFKA-12245) fix flay org.apache.kafka.clients.consumer.internals.FetcherTest.testEarlierOffsetResetArrivesLate

2021-02-01 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-12245. - Resolution: Fixed > fix flay > org.apache.kafka.clients.consumer.internals.FetcherTest.

[GitHub] [kafka] hachikuji opened a new pull request #10015: MINOR: Factor out controller node provider `BrokerToControllerChannelManager`

2021-02-01 Thread GitBox
hachikuji opened a new pull request #10015: URL: https://github.com/apache/kafka/pull/10015 This patch factors out a trait to allow for other ways to provide the controller `Node` object to `BrokerToControllerChannelManager`. In KIP-500, the controller will be provided from the Raft client

[GitHub] [kafka] hachikuji commented on pull request #9994: KAFKA-12248: Add BrokerHeartbeat/BrokerRegistration RPCs for KIP-500

2021-02-01 Thread GitBox
hachikuji commented on pull request #9994: URL: https://github.com/apache/kafka/pull/9994#issuecomment-771137129 @aloknnikhil Looks like we will need to fix some failures in `ApiKeysTest` before we can merge this. I suspect that the test does not like the throttle time fields. One way we c

[GitHub] [kafka] mumrah commented on a change in pull request #10009: MINOR: Introduce ProducerIdGenerator trait

2021-02-01 Thread GitBox
mumrah commented on a change in pull request #10009: URL: https://github.com/apache/kafka/pull/10009#discussion_r568133942 ## File path: core/src/main/scala/kafka/coordinator/transaction/TransactionCoordinator.scala ## @@ -615,7 +615,7 @@ class TransactionCoordinator(brokerId:

[GitHub] [kafka] cmccabe commented on a change in pull request #9901: KAFKA-12209: Add the timeline data structures for the KIP-631 controller

2021-02-01 Thread GitBox
cmccabe commented on a change in pull request #9901: URL: https://github.com/apache/kafka/pull/9901#discussion_r568134210 ## File path: metadata/src/main/java/org/apache/kafka/timeline/SnapshottableHashTable.java ## @@ -0,0 +1,449 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] [kafka] mumrah commented on a change in pull request #10009: MINOR: Introduce ProducerIdGenerator trait

2021-02-01 Thread GitBox
mumrah commented on a change in pull request #10009: URL: https://github.com/apache/kafka/pull/10009#discussion_r568134453 ## File path: core/src/main/scala/kafka/coordinator/transaction/TransactionCoordinator.scala ## @@ -615,7 +615,7 @@ class TransactionCoordinator(brokerId:

[GitHub] [kafka] C0urante opened a new pull request #10016: KAFKA-10340: Proactively close producer when cancelling source tasks

2021-02-01 Thread GitBox
C0urante opened a new pull request #10016: URL: https://github.com/apache/kafka/pull/10016 [Jira](https://issues.apache.org/jira/browse/KAFKA-10340) When a source task produces records for a topic that doesn't exist on the Kafka cluster and automatic topic creation is disabled on the

[GitHub] [kafka] C0urante commented on pull request #10016: KAFKA-10340: Proactively close producer when cancelling source tasks

2021-02-01 Thread GitBox
C0urante commented on pull request #10016: URL: https://github.com/apache/kafka/pull/10016#issuecomment-771154039 @gharris1727 @ncliang @sajanaW would any of you have time to take a look? This is an automated message from the

[GitHub] [kafka] mumrah commented on a change in pull request #10008: MINOR: Remove ZK dependency for coordinator topics' partition counts

2021-02-01 Thread GitBox
mumrah commented on a change in pull request #10008: URL: https://github.com/apache/kafka/pull/10008#discussion_r568142924 ## File path: core/src/main/scala/kafka/coordinator/transaction/TransactionStateManager.scala ## @@ -96,7 +108,20 @@ class TransactionStateManager(brokerI

[GitHub] [kafka] abbccdda opened a new pull request #10017: KAFKA-12260: Avoid hitting NPE for partitionsFor

2021-02-01 Thread GitBox
abbccdda opened a new pull request #10017: URL: https://github.com/apache/kafka/pull/10017 As mentioned in the ticket, returning `null` for `consumer#partitionsFor` is not a good client side agreement. Addressing this problem by returning the empty list instead. ### Committer Checkl

[GitHub] [kafka] mumrah commented on a change in pull request #10008: MINOR: Remove ZK dependency for coordinator topics' partition counts

2021-02-01 Thread GitBox
mumrah commented on a change in pull request #10008: URL: https://github.com/apache/kafka/pull/10008#discussion_r568144003 ## File path: core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala ## @@ -78,7 +78,20 @@ class GroupMetadataManager(brokerId: Int, pr

[GitHub] [kafka] mumrah commented on pull request #10008: MINOR: Remove ZK dependency for coordinator topics' partition counts

2021-02-01 Thread GitBox
mumrah commented on pull request #10008: URL: https://github.com/apache/kafka/pull/10008#issuecomment-771166135 What is the motivation for making transactionTopicPartitionCount and groupMetadataTopicPartitionCount lazy? Is it because in the KIP-500 world, we may not yet have this metadata

[GitHub] [kafka] mumrah commented on a change in pull request #10008: MINOR: Remove ZK dependency for coordinator topics' partition counts

2021-02-01 Thread GitBox
mumrah commented on a change in pull request #10008: URL: https://github.com/apache/kafka/pull/10008#discussion_r568149064 ## File path: core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala ## @@ -78,7 +78,20 @@ class GroupMetadataManager(brokerId: Int, pr

[GitHub] [kafka] cmccabe commented on a change in pull request #9901: KAFKA-12209: Add the timeline data structures for the KIP-631 controller

2021-02-01 Thread GitBox
cmccabe commented on a change in pull request #9901: URL: https://github.com/apache/kafka/pull/9901#discussion_r568157113 ## File path: metadata/src/main/java/org/apache/kafka/timeline/SnapshottableHashTable.java ## @@ -0,0 +1,449 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] [kafka] ableegoldman commented on a change in pull request #9997: KAFKA-9274: Add timeout handling for `StreamPartitioner`

2021-02-01 Thread GitBox
ableegoldman commented on a change in pull request #9997: URL: https://github.com/apache/kafka/pull/9997#discussion_r568158224 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -714,19 +720,34 @@ public boolean process(final

[GitHub] [kafka] junrao commented on a change in pull request #9901: KAFKA-12209: Add the timeline data structures for the KIP-631 controller

2021-02-01 Thread GitBox
junrao commented on a change in pull request #9901: URL: https://github.com/apache/kafka/pull/9901#discussion_r568172947 ## File path: metadata/src/main/java/org/apache/kafka/timeline/SnapshottableHashTable.java ## @@ -0,0 +1,453 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [kafka] mumrah opened a new pull request #10018: Upstream MetadataImage and related classes

2021-02-01 Thread GitBox
mumrah opened a new pull request #10018: URL: https://github.com/apache/kafka/pull/10018 Mostly a wholesale copy of the files from confluentinc/kip-500, with a few fixes for the test classes This is an automated message from

[GitHub] [kafka] gharris1727 commented on a change in pull request #10014: KAFKA-12252 and KAFKA-12262: Fix key rotation when leadership changes

2021-02-01 Thread GitBox
gharris1727 commented on a change in pull request #10014: URL: https://github.com/apache/kafka/pull/10014#discussion_r568174791 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java ## @@ -381,7 +381,7 @@ public void ti

[GitHub] [kafka] hachikuji merged pull request #10015: MINOR: Factor out controller node provider `BrokerToControllerChannelManager`

2021-02-01 Thread GitBox
hachikuji merged pull request #10015: URL: https://github.com/apache/kafka/pull/10015 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [kafka] hachikuji edited a comment on pull request #9996: KAFKA-12249: KIP-500 - Add client-side Decommission Broker RPC

2021-02-01 Thread GitBox
hachikuji edited a comment on pull request #9996: URL: https://github.com/apache/kafka/pull/9996#issuecomment-770067513 Note also the test failure `RequestConvertToJsonTest.testAllResponseTypesHandled`. This is an automated

[GitHub] [kafka] gharris1727 commented on a change in pull request #10016: KAFKA-10340: Proactively close producer when cancelling source tasks

2021-02-01 Thread GitBox
gharris1727 commented on a change in pull request #10016: URL: https://github.com/apache/kafka/pull/10016#discussion_r568190708 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java ## @@ -202,6 +198,8 @@ public void removeMetrics()

[GitHub] [kafka] hachikuji commented on a change in pull request #9996: KAFKA-12249: KIP-500 - Add client-side Decommission Broker RPC

2021-02-01 Thread GitBox
hachikuji commented on a change in pull request #9996: URL: https://github.com/apache/kafka/pull/9996#discussion_r568200014 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/ApiKeys.java ## @@ -98,7 +98,8 @@ UPDATE_FEATURES(ApiMessageType.UPDATE_FEATURES

[GitHub] [kafka] cmccabe commented on a change in pull request #9901: KAFKA-12209: Add the timeline data structures for the KIP-631 controller

2021-02-01 Thread GitBox
cmccabe commented on a change in pull request #9901: URL: https://github.com/apache/kafka/pull/9901#discussion_r568201202 ## File path: metadata/src/main/java/org/apache/kafka/timeline/SnapshottableHashTable.java ## @@ -0,0 +1,453 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] [kafka] rondagostino opened a new pull request #10019: MINOR: Introduce KafkaBroker trait for use in dynamic reconfiguration

2021-02-01 Thread GitBox
rondagostino opened a new pull request #10019: URL: https://github.com/apache/kafka/pull/10019 Dynamic broker reconfiguration needs to occur for both ZooKeeper-based brokers and brokers that use a Raft-based metadata quorum. `DynamicBrokerConfig` currently operates on `KafkaServer`, but i

  1   2   >