[GitHub] [kafka] serjchebotarev commented on pull request #9028: KAFKA-10035: Safer conversion of consumer timeout parameters

2020-08-12 Thread GitBox
serjchebotarev commented on pull request #9028: URL: https://github.com/apache/kafka/pull/9028#issuecomment-672683714 Looks like no issues related to the changes in this PR. @mjsax could you please suggest what would be the next thing to do on this one? ---

[jira] [Assigned] (KAFKA-10390) kafka-server-stop lookup is not specific enough and may kill other processes

2020-08-12 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen reassigned KAFKA-10390: - Assignee: Luke Chen > kafka-server-stop lookup is not specific enough and may kill other proces

[GitHub] [kafka] rajinisivaram opened a new pull request #9168: MINOR: Exclude scala-library from jacksonModuleScala for compile

2020-08-12 Thread GitBox
rajinisivaram opened a new pull request #9168: URL: https://github.com/apache/kafka/pull/9168 We are forcing runtime scala-library dependency to be the one used by Kafka in https://github.com/apache/kafka/commit/3fdf1523a369e9e2f03e215f39c0ed987b2a16c5, but we still seem to include the ve

[GitHub] [kafka] cadonna commented on a change in pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
cadonna commented on a change in pull request #9098: URL: https://github.com/apache/kafka/pull/9098#discussion_r469095757 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java ## @@ -181,26 +181,42 @@ void openDB(final ProcessorContext

[GitHub] [kafka] chia7712 commented on pull request #9162: MINOR: refactor Log#read and Log#fetchOffsetByTimestamp to avoid using "return" keyword in nested an…

2020-08-12 Thread GitBox
chia7712 commented on pull request #9162: URL: https://github.com/apache/kafka/pull/9162#issuecomment-672737657 I profiled one of our use case to trace the ```NonLocalReturnException```. ```Log#read``` and ```Log#fetchOffsetByTimestamp``` produces most of ```NonLocalReturnException``` so I

[jira] [Created] (KAFKA-10392) kafka-producer-perf-test.sh includes messages successfully delivered when acks=-1

2020-08-12 Thread perryzou (Jira)
perryzou created KAFKA-10392: Summary: kafka-producer-perf-test.sh includes messages successfully delivered when acks=-1 Key: KAFKA-10392 URL: https://issues.apache.org/jira/browse/KAFKA-10392 Project: Ka

[jira] [Updated] (KAFKA-10307) Topology cycles in KTableKTableForeignKeyInnerJoinMultiIntegrationTest#shouldInnerJoinMultiPartitionQueryable

2020-08-12 Thread feyman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feyman updated KAFKA-10307: --- Attachment: repartition_calc.jpg > Topology cycles in > KTableKTableForeignKeyInnerJoinMultiIntegrationTest

[jira] [Commented] (KAFKA-10307) Topology cycles in KTableKTableForeignKeyInnerJoinMultiIntegrationTest#shouldInnerJoinMultiPartitionQueryable

2020-08-12 Thread feyman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176224#comment-17176224 ] feyman commented on KAFKA-10307: [~vvcephei] Thanks for letting me know that we can have

[jira] [Commented] (KAFKA-8362) LogCleaner gets stuck after partition move between log directories

2020-08-12 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176257#comment-17176257 ] Luke Chen commented on KAFKA-8362: -- Currently, we'll append a tombstone record into the

[jira] [Assigned] (KAFKA-10392) kafka-producer-perf-test.sh includes messages successfully delivered when acks=-1

2020-08-12 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen reassigned KAFKA-10392: - Assignee: Luke Chen > kafka-producer-perf-test.sh includes messages successfully delivered when

[GitHub] [kafka] jeqo commented on a change in pull request #9137: KAFKA-9929: Support reverse iterator on KeyValueStore

2020-08-12 Thread GitBox
jeqo commented on a change in pull request #9137: URL: https://github.com/apache/kafka/pull/9137#discussion_r469225368 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBRangeIterator.java ## @@ -29,32 +29,41 @@ // comparator to be pluggab

[GitHub] [kafka] showuon commented on a change in pull request #9149: KAFKA-10340: improve the logging to help user know what is going on

2020-08-12 Thread GitBox
showuon commented on a change in pull request #9149: URL: https://github.com/apache/kafka/pull/9149#discussion_r467713932 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java ## @@ -363,6 +363,7 @@ private boolean sendRecords() {

[GitHub] [kafka] ijuma commented on a change in pull request #9168: MINOR: Exclude scala-library from jacksonModuleScala for compile

2020-08-12 Thread GitBox
ijuma commented on a change in pull request #9168: URL: https://github.com/apache/kafka/pull/9168#discussion_r469256355 ## File path: build.gradle ## @@ -716,7 +716,9 @@ project(':core') { dependencies { compile project(':clients') compile libs.jacksonDatabind -

[GitHub] [kafka] ijuma commented on pull request #9162: MINOR: refactor Log#read and Log#fetchOffsetByTimestamp to avoid using "return" keyword in nested an…

2020-08-12 Thread GitBox
ijuma commented on pull request #9162: URL: https://github.com/apache/kafka/pull/9162#issuecomment-672882880 @chia7712 what are the other cases where we do this in `Log`? This is an automated message from the Apache Git Servi

[GitHub] [kafka] bbejeck commented on pull request #9108: KAFKA-9273: Extract testShouldAutoShutdownOnIncompleteMetadata from S…

2020-08-12 Thread GitBox
bbejeck commented on pull request #9108: URL: https://github.com/apache/kafka/pull/9108#issuecomment-672884942 Ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [kafka] chia7712 commented on pull request #9162: MINOR: refactor Log#read and Log#fetchOffsetByTimestamp to avoid using "return" keyword in nested an…

2020-08-12 Thread GitBox
chia7712 commented on pull request #9162: URL: https://github.com/apache/kafka/pull/9162#issuecomment-672891234 > what are the other cases where we do this in Log? The use case I mentioned is a heavy IO-load task of my (team) application. I didn't trace other potential hot methods fo

[GitHub] [kafka] chia7712 opened a new pull request #9169: MINOR: the scheduler used to perform rebalance should have thread prefix

2020-08-12 Thread GitBox
chia7712 opened a new pull request #9169: URL: https://github.com/apache/kafka/pull/9169 There is already a scheduler using default thread prefix so the others should define different thread name. Otherwise, it is hard to distinguish them by jvm profiler. ### Committer Checklist (ex

[GitHub] [kafka] ijuma commented on pull request #9162: MINOR: refactor Log#read and Log#fetchOffsetByTimestamp to avoid using "return" keyword in nested an…

2020-08-12 Thread GitBox
ijuma commented on pull request #9162: URL: https://github.com/apache/kafka/pull/9162#issuecomment-672907940 Yeah, I would fix all the cases in this file at once if possible. This is an automated message from the Apache Git S

[GitHub] [kafka] ijuma commented on a change in pull request #7222: KAFKA-8806 Reduce calls to validateOffsetsIfNeeded

2020-08-12 Thread GitBox
ijuma commented on a change in pull request #7222: URL: https://github.com/apache/kafka/pull/7222#discussion_r469307691 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/SubscriptionState.java ## @@ -639,26 +643,34 @@ public synchronized OffsetRes

[GitHub] [kafka] vvcephei commented on pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
vvcephei commented on pull request #9098: URL: https://github.com/apache/kafka/pull/9098#issuecomment-672912500 Ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] jeqo commented on a change in pull request #9137: KAFKA-9929: Support reverse iterator on KeyValueStore

2020-08-12 Thread GitBox
jeqo commented on a change in pull request #9137: URL: https://github.com/apache/kafka/pull/9137#discussion_r469314938 ## File path: streams/src/test/java/org/apache/kafka/streams/state/internals/AbstractKeyValueStoreTest.java ## @@ -422,6 +503,21 @@ public void shouldNotThro

[GitHub] [kafka] jeqo commented on a change in pull request #9137: KAFKA-9929: Support reverse iterator on KeyValueStore

2020-08-12 Thread GitBox
jeqo commented on a change in pull request #9137: URL: https://github.com/apache/kafka/pull/9137#discussion_r469315300 ## File path: streams/src/test/java/org/apache/kafka/streams/state/internals/CachingKeyValueStoreTest.java ## @@ -294,13 +305,27 @@ public void shouldIterateO

[GitHub] [kafka] jeqo commented on a change in pull request #9137: KAFKA-9929: Support reverse iterator on KeyValueStore

2020-08-12 Thread GitBox
jeqo commented on a change in pull request #9137: URL: https://github.com/apache/kafka/pull/9137#discussion_r469315828 ## File path: streams/src/test/java/org/apache/kafka/streams/state/internals/CachingKeyValueStoreTest.java ## @@ -339,12 +366,24 @@ public void shouldThrowIf

[GitHub] [kafka] rajinisivaram commented on pull request #9168: MINOR: Exclude scala-library from jacksonModuleScala for compile

2020-08-12 Thread GitBox
rajinisivaram commented on pull request #9168: URL: https://github.com/apache/kafka/pull/9168#issuecomment-672922930 @ijuma I have updated the PR using `configurations.all` that you suggested. Thank you! This is an automated

[GitHub] [kafka] vvcephei commented on a change in pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
vvcephei commented on a change in pull request #9098: URL: https://github.com/apache/kafka/pull/9098#discussion_r469327500 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/metrics/RocksDBMetricsRecorder.java ## @@ -187,37 +206,39 @@ public void reco

[GitHub] [kafka] hachikuji merged pull request #9166: KAFKA-10388; Fix struct conversion logic for tagged structures

2020-08-12 Thread GitBox
hachikuji merged pull request #9166: URL: https://github.com/apache/kafka/pull/9166 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Resolved] (KAFKA-10388) Casting errors in tagged struct conversion

2020-08-12 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-10388. - Resolution: Fixed > Casting errors in tagged struct conversion > ---

[GitHub] [kafka] chia7712 commented on pull request #9162: MINOR: refactor Log to get rid of "return" in nested anonymous function

2020-08-12 Thread GitBox
chia7712 commented on pull request #9162: URL: https://github.com/apache/kafka/pull/9162#issuecomment-672957838 > Yeah, I would fix all the cases in this file at once if possible. sure This is an automated message from

[GitHub] [kafka] abbccdda commented on pull request #8549: KAFKA-9911: Add new PRODUCER_FENCED error code

2020-08-12 Thread GitBox
abbccdda commented on pull request #8549: URL: https://github.com/apache/kafka/pull/8549#issuecomment-672958592 All 3 pass, with minor exception due to ssh channel This is an automated message from the Apache Git Service. To

[GitHub] [kafka] abbccdda merged pull request #8549: KAFKA-9911: Add new PRODUCER_FENCED error code

2020-08-12 Thread GitBox
abbccdda merged pull request #8549: URL: https://github.com/apache/kafka/pull/8549 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] vvcephei commented on pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
vvcephei commented on pull request #9098: URL: https://github.com/apache/kafka/pull/9098#issuecomment-672975608 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] vvcephei commented on pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
vvcephei commented on pull request #9098: URL: https://github.com/apache/kafka/pull/9098#issuecomment-672975857 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [kafka] vvcephei commented on pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
vvcephei commented on pull request #9098: URL: https://github.com/apache/kafka/pull/9098#issuecomment-672975770 Ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] vvcephei commented on pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
vvcephei commented on pull request #9098: URL: https://github.com/apache/kafka/pull/9098#issuecomment-672977276 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] vvcephei commented on pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
vvcephei commented on pull request #9098: URL: https://github.com/apache/kafka/pull/9098#issuecomment-672977429 Well, I guess I'll just run them locally. This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] guozhangwang commented on a change in pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
guozhangwang commented on a change in pull request #9098: URL: https://github.com/apache/kafka/pull/9098#discussion_r469388654 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java ## @@ -181,26 +181,42 @@ void openDB(final ProcessorCon

[jira] [Updated] (KAFKA-9911) Implement new producer fenced error

2020-08-12 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-9911: --- Fix Version/s: 2.7.0 > Implement new producer fenced error > --- > >

[jira] [Assigned] (KAFKA-9911) Implement new producer fenced error

2020-08-12 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen reassigned KAFKA-9911: -- Assignee: Boyang Chen > Implement new producer fenced error >

[jira] [Resolved] (KAFKA-9911) Implement new producer fenced error

2020-08-12 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen resolved KAFKA-9911. Resolution: Fixed > Implement new producer fenced error > --- > >

[jira] [Updated] (KAFKA-9911) Implement new producer fenced error

2020-08-12 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-9911: --- Component/s: producer > Implement new producer fenced error > --- >

[jira] [Commented] (KAFKA-8362) LogCleaner gets stuck after partition move between log directories

2020-08-12 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176476#comment-17176476 ] Ming Liu commented on KAFKA-8362: - In that case, we need to modify *allCleanerCheckpoints

[GitHub] [kafka] guozhangwang commented on pull request #9170: KAFKA-10391: Overwrite checkpoint in task corruption to remove corrupted partitions

2020-08-12 Thread GitBox
guozhangwang commented on pull request #9170: URL: https://github.com/apache/kafka/pull/9170#issuecomment-672992402 Successful system tests: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/4121/ https://jenkins.confluent.io/job/system-test-kafka-branch-builder/41

[GitHub] [kafka] guozhangwang opened a new pull request #9170: KAFKA-10391: Overwrite checkpoint in task corruption to remove corrupted partitions

2020-08-12 Thread GitBox
guozhangwang opened a new pull request #9170: URL: https://github.com/apache/kafka/pull/9170 In order to do this, I also removed the optimization such that once enforced checkpoint is set to true, we always checkpoint unless the state stores are not initialized at all (i.e. the snapshot is

[jira] [Commented] (KAFKA-10307) Topology cycles in KTableKTableForeignKeyInnerJoinMultiIntegrationTest#shouldInnerJoinMultiPartitionQueryable

2020-08-12 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176482#comment-17176482 ] John Roesler commented on KAFKA-10307: -- Hey [~feyman] , thanks for the reply! IIRC

[GitHub] [kafka] rondagostino commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-12 Thread GitBox
rondagostino commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r469410480 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java ## @@ -4071,6 +4081,168 @@ void handleFailure(Throwable throw

[GitHub] [kafka] guozhangwang commented on pull request #9170: KAFKA-10391: Overwrite checkpoint in task corruption to remove corrupted partitions

2020-08-12 Thread GitBox
guozhangwang commented on pull request #9170: URL: https://github.com/apache/kafka/pull/9170#issuecomment-67281 cc @ableegoldman @abbccdda for reviews This is an automated message from the Apache Git Service. To respond t

[GitHub] [kafka] rondagostino commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-12 Thread GitBox
rondagostino commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r469421523 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java ## @@ -4071,6 +4081,168 @@ void handleFailure(Throwable throw

[GitHub] [kafka] cadonna commented on a change in pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
cadonna commented on a change in pull request #9098: URL: https://github.com/apache/kafka/pull/9098#discussion_r469422110 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java ## @@ -181,26 +181,42 @@ void openDB(final ProcessorContext

[GitHub] [kafka] ableegoldman commented on pull request #9171: MINOR: add ableegoldman and cadonna to whitelist

2020-08-12 Thread GitBox
ableegoldman commented on pull request #9171: URL: https://github.com/apache/kafka/pull/9171#issuecomment-673010059 cc @vvcephei @guozhangwang This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [kafka] ableegoldman opened a new pull request #9171: MINOR: add ableegoldman and cadonna to whitelist

2020-08-12 Thread GitBox
ableegoldman opened a new pull request #9171: URL: https://github.com/apache/kafka/pull/9171 So we don't have to bug the committers to kick off tests 😄 This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] guozhangwang merged pull request #9171: MINOR: add ableegoldman and cadonna to whitelist

2020-08-12 Thread GitBox
guozhangwang merged pull request #9171: URL: https://github.com/apache/kafka/pull/9171 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Created] (KAFKA-10393) KIP-630: messafe for fetch snapshot and fetch

2020-08-12 Thread Jose Armando Garcia Sancio (Jira)
Jose Armando Garcia Sancio created KAFKA-10393: -- Summary: KIP-630: messafe for fetch snapshot and fetch Key: KAFKA-10393 URL: https://issues.apache.org/jira/browse/KAFKA-10393 Project: Kaf

[GitHub] [kafka] hachikuji commented on a change in pull request #9100: Add AlterISR RPC and use it for ISR modifications

2020-08-12 Thread GitBox
hachikuji commented on a change in pull request #9100: URL: https://github.com/apache/kafka/pull/9100#discussion_r469388095 ## File path: core/src/main/scala/kafka/cluster/Partition.scala ## @@ -748,7 +755,7 @@ class Partition(val topicPartition: TopicPartition, leaderLogI

[GitHub] [kafka] ableegoldman commented on a change in pull request #9170: KAFKA-10391: Overwrite checkpoint in task corruption to remove corrupted partitions

2020-08-12 Thread GitBox
ableegoldman commented on a change in pull request #9170: URL: https://github.com/apache/kafka/pull/9170#discussion_r469450345 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamTaskTest.java ## @@ -1609,12 +1609,11 @@ public void shouldRetu

[GitHub] [kafka] ableegoldman commented on pull request #9028: KAFKA-10035: Safer conversion of consumer timeout parameters

2020-08-12 Thread GitBox
ableegoldman commented on pull request #9028: URL: https://github.com/apache/kafka/pull/9028#issuecomment-673036155 @serjchebotarev he's out of office this week, just fyi. This is an automated message from the Apache Git Ser

[GitHub] [kafka] rondagostino commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-12 Thread GitBox
rondagostino commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r469457068 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java ## @@ -4071,6 +4081,168 @@ void handleFailure(Throwable throw

[GitHub] [kafka] mumrah commented on a change in pull request #9100: Add AlterISR RPC and use it for ISR modifications

2020-08-12 Thread GitBox
mumrah commented on a change in pull request #9100: URL: https://github.com/apache/kafka/pull/9100#discussion_r469460462 ## File path: core/src/main/scala/kafka/server/AlterIsrChannelManager.scala ## @@ -0,0 +1,132 @@ +package kafka.server + +import java.util +import java.util.

[jira] [Commented] (KAFKA-10357) Handle accidental deletion of repartition-topics as exceptional failure

2020-08-12 Thread Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176535#comment-17176535 ] Sophie Blee-Goldman commented on KAFKA-10357: - I think the elegant way to sh

[GitHub] [kafka] mumrah commented on a change in pull request #9100: Add AlterISR RPC and use it for ISR modifications

2020-08-12 Thread GitBox
mumrah commented on a change in pull request #9100: URL: https://github.com/apache/kafka/pull/9100#discussion_r469467881 ## File path: core/src/main/scala/kafka/cluster/Partition.scala ## @@ -1210,28 +1218,20 @@ class Partition(val topicPartition: TopicPartition, } }

[GitHub] [kafka] serjchebotarev commented on pull request #9028: KAFKA-10035: Safer conversion of consumer timeout parameters

2020-08-12 Thread GitBox
serjchebotarev commented on pull request #9028: URL: https://github.com/apache/kafka/pull/9028#issuecomment-673049249 @ableegoldman alright, thanks. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] d8tltanc commented on a change in pull request #8846: KAFKA-9800: [KIP-580] Client Exponential Backoff Implementation

2020-08-12 Thread GitBox
d8tltanc commented on a change in pull request #8846: URL: https://github.com/apache/kafka/pull/8846#discussion_r469471723 ## File path: clients/src/main/java/org/apache/kafka/common/utils/GeometricProgression.java ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] d8tltanc commented on a change in pull request #8846: KAFKA-9800: [KIP-580] Client Exponential Backoff Implementation

2020-08-12 Thread GitBox
d8tltanc commented on a change in pull request #8846: URL: https://github.com/apache/kafka/pull/8846#discussion_r469472239 ## File path: clients/src/main/java/org/apache/kafka/common/utils/GeometricProgression.java ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] d8tltanc commented on a change in pull request #8846: KAFKA-9800: [KIP-580] Client Exponential Backoff Implementation

2020-08-12 Thread GitBox
d8tltanc commented on a change in pull request #8846: URL: https://github.com/apache/kafka/pull/8846#discussion_r469473196 ## File path: clients/src/main/java/org/apache/kafka/clients/Metadata.java ## @@ -74,22 +76,28 @@ private final ClusterResourceListeners clusterResour

[GitHub] [kafka] d8tltanc commented on a change in pull request #8846: KAFKA-9800: [KIP-580] Client Exponential Backoff Implementation

2020-08-12 Thread GitBox
d8tltanc commented on a change in pull request #8846: URL: https://github.com/apache/kafka/pull/8846#discussion_r468736687 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java ## @@ -721,6 +730,11 @@ protected Node curNode() { r

[GitHub] [kafka] d8tltanc commented on a change in pull request #8846: KAFKA-9800: [KIP-580] Client Exponential Backoff Implementation

2020-08-12 Thread GitBox
d8tltanc commented on a change in pull request #8846: URL: https://github.com/apache/kafka/pull/8846#discussion_r468736687 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java ## @@ -721,6 +730,11 @@ protected Node curNode() { r

[jira] [Updated] (KAFKA-10393) KIP-630: message for fetch snapshot and fetch

2020-08-12 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio updated KAFKA-10393: --- Summary: KIP-630: message for fetch snapshot and fetch (was: KIP-630

[GitHub] [kafka] rondagostino commented on a change in pull request #9032: KAFKA-10259: KIP-554 Broker-side SCRAM Config API

2020-08-12 Thread GitBox
rondagostino commented on a change in pull request #9032: URL: https://github.com/apache/kafka/pull/9032#discussion_r469476195 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java ## @@ -4071,6 +4081,168 @@ void handleFailure(Throwable throw

[jira] [Created] (KAFKA-10394) Internal API for generating snapshots

2020-08-12 Thread Jose Armando Garcia Sancio (Jira)
Jose Armando Garcia Sancio created KAFKA-10394: -- Summary: Internal API for generating snapshots Key: KAFKA-10394 URL: https://issues.apache.org/jira/browse/KAFKA-10394 Project: Kafka

[jira] [Updated] (KAFKA-10393) Message for fetch snapshot and fetch

2020-08-12 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio updated KAFKA-10393: --- Summary: Message for fetch snapshot and fetch (was: KIP-630: message

[GitHub] [kafka] kkonstantine opened a new pull request #9172: KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect

2020-08-12 Thread GitBox
kkonstantine opened a new pull request #9172: URL: https://github.com/apache/kafka/pull/9172 Addition of configs for custom topic creation with KIP-158 created a regression when transformation configs are also included in the configuration of a source connector. To experience the i

[GitHub] [kafka] kkonstantine commented on pull request #9172: KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect

2020-08-12 Thread GitBox
kkonstantine commented on pull request #9172: URL: https://github.com/apache/kafka/pull/9172#issuecomment-673065882 I'll be adding a few more tests. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] kkonstantine commented on pull request #9172: KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect

2020-08-12 Thread GitBox
kkonstantine commented on pull request #9172: URL: https://github.com/apache/kafka/pull/9172#issuecomment-673067506 cc @rhauch @wicknicks This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [kafka] ableegoldman opened a new pull request #9173: KAFKA-10122: Consumer should allow heartbeat during rebalance

2020-08-12 Thread GitBox
ableegoldman opened a new pull request #9173: URL: https://github.com/apache/kafka/pull/9173 We launched a Streams application reading from a single 3000-partition topic and saw continuous rebalancing. Digging into the logs, every time the leader sent a SyncGroup request it would discover

[GitHub] [kafka] abbccdda commented on a change in pull request #9170: KAFKA-10391: Overwrite checkpoint in task corruption to remove corrupted partitions

2020-08-12 Thread GitBox
abbccdda commented on a change in pull request #9170: URL: https://github.com/apache/kafka/pull/9170#discussion_r469509680 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -193,13 +193,17 @@ private void closeAndRevive(fin

[GitHub] [kafka] abbccdda commented on pull request #8846: KAFKA-9800: [KIP-580] Client Exponential Backoff Implementation

2020-08-12 Thread GitBox
abbccdda commented on pull request #8846: URL: https://github.com/apache/kafka/pull/8846#issuecomment-673088220 @d8tltanc Thanks for the great work! I could resume reviewing this PR sometime next week, do you mind rebasing it? --

[GitHub] [kafka] vvcephei commented on pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
vvcephei commented on pull request #9098: URL: https://github.com/apache/kafka/pull/9098#issuecomment-673090023 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Commented] (KAFKA-10357) Handle accidental deletion of repartition-topics as exceptional failure

2020-08-12 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176582#comment-17176582 ] Guozhang Wang commented on KAFKA-10357: --- I've thought about relaying on the commit

[GitHub] [kafka] vvcephei commented on pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
vvcephei commented on pull request #9098: URL: https://github.com/apache/kafka/pull/9098#issuecomment-673090128 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] vvcephei commented on pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
vvcephei commented on pull request #9098: URL: https://github.com/apache/kafka/pull/9098#issuecomment-673090192 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] guozhangwang commented on pull request #9170: KAFKA-10391: Overwrite checkpoint in task corruption to remove corrupted partitions

2020-08-12 Thread GitBox
guozhangwang commented on pull request #9170: URL: https://github.com/apache/kafka/pull/9170#issuecomment-673090664 > Do you have any theories as to why this was causing `StreamsUpgradeTest.test_app_upgrade` to be flaky? Do we really get that many TaskCorruptedExceptions in this system tes

[GitHub] [kafka] guozhangwang commented on a change in pull request #9170: KAFKA-10391: Overwrite checkpoint in task corruption to remove corrupted partitions

2020-08-12 Thread GitBox
guozhangwang commented on a change in pull request #9170: URL: https://github.com/apache/kafka/pull/9170#discussion_r469517595 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -193,13 +193,17 @@ private void closeAndRevive

[GitHub] [kafka] guozhangwang commented on a change in pull request #9170: KAFKA-10391: Overwrite checkpoint in task corruption to remove corrupted partitions

2020-08-12 Thread GitBox
guozhangwang commented on a change in pull request #9170: URL: https://github.com/apache/kafka/pull/9170#discussion_r469518409 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StateManagerUtil.java ## @@ -58,6 +58,9 @@ static boolean checkpointN

[GitHub] [kafka] vvcephei removed a comment on pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
vvcephei removed a comment on pull request #9098: URL: https://github.com/apache/kafka/pull/9098#issuecomment-673100824 @cadonna : ``` > Task :streams:checkstyleMain [ant:checkstyle] [ERROR] /home/confluent/kafka/streams/src/generated/java/org/apache/kafka/streams/InteractiveQ

[GitHub] [kafka] vvcephei commented on pull request #9098: KAFKA-9924: Prepare RocksDB and metrics for RocksDB properties recording

2020-08-12 Thread GitBox
vvcephei commented on pull request #9098: URL: https://github.com/apache/kafka/pull/9098#issuecomment-673100824 @cadonna : ``` > Task :streams:checkstyleMain [ant:checkstyle] [ERROR] /home/confluent/kafka/streams/src/generated/java/org/apache/kafka/streams/InteractiveQueryPart

[GitHub] [kafka] d8tltanc commented on a change in pull request #8846: KAFKA-9800: [KIP-580] Client Exponential Backoff Implementation

2020-08-12 Thread GitBox
d8tltanc commented on a change in pull request #8846: URL: https://github.com/apache/kafka/pull/8846#discussion_r468736687 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java ## @@ -721,6 +730,11 @@ protected Node curNode() { r

[GitHub] [kafka] d8tltanc commented on pull request #8846: KAFKA-9800: [KIP-580] Client Exponential Backoff Implementation

2020-08-12 Thread GitBox
d8tltanc commented on pull request #8846: URL: https://github.com/apache/kafka/pull/8846#issuecomment-673114249 @abbccdda Thanks for the review offers! I'll rebase this PR. This is an automated message from the Apache Git Ser

[jira] [Updated] (KAFKA-10387) Cannot include SMT configs with source connector that uses topic.creation.* properties

2020-08-12 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-10387: -- Priority: Critical (was: Major) > Cannot include SMT configs with source connector that uses

[GitHub] [kafka] rhauch commented on a change in pull request #9172: KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect

2020-08-12 Thread GitBox
rhauch commented on a change in pull request #9172: URL: https://github.com/apache/kafka/pull/9172#discussion_r469574130 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/integration/TransformationIntegrationTest.java ## @@ -318,4 +323,61 @@ public void test

[GitHub] [kafka] kkonstantine commented on pull request #9172: KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect

2020-08-12 Thread GitBox
kkonstantine commented on pull request #9172: URL: https://github.com/apache/kafka/pull/9172#issuecomment-673136551 According to https://status.apache.org the jenkins queue has grown too much and the infra team is investigating how to resolve the issues. Thus, the tests haven't run yet.

[GitHub] [kafka] kkonstantine commented on a change in pull request #9172: KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect

2020-08-12 Thread GitBox
kkonstantine commented on a change in pull request #9172: URL: https://github.com/apache/kafka/pull/9172#discussion_r469577371 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SourceConnectorConfig.java ## @@ -129,7 +129,8 @@ public SourceConnectorC

[GitHub] [kafka] kkonstantine commented on a change in pull request #9172: KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect

2020-08-12 Thread GitBox
kkonstantine commented on a change in pull request #9172: URL: https://github.com/apache/kafka/pull/9172#discussion_r469578195 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/integration/TransformationIntegrationTest.java ## @@ -318,4 +323,61 @@ public voi

[GitHub] [kafka] ableegoldman commented on a change in pull request #9094: KAFKA-10054: KIP-613, add TRACE-level e2e latency metrics

2020-08-12 Thread GitBox
ableegoldman commented on a change in pull request #9094: URL: https://github.com/apache/kafka/pull/9094#discussion_r469582284 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/metrics/ProcessorNodeMetrics.java ## @@ -289,6 +294,25 @@ public stat

[GitHub] [kafka] ableegoldman commented on a change in pull request #9094: KAFKA-10054: KIP-613, add TRACE-level e2e latency metrics

2020-08-12 Thread GitBox
ableegoldman commented on a change in pull request #9094: URL: https://github.com/apache/kafka/pull/9094#discussion_r469582284 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/metrics/ProcessorNodeMetrics.java ## @@ -289,6 +294,25 @@ public stat

[GitHub] [kafka] ableegoldman commented on pull request #9173: KAFKA-10122: Consumer should allow heartbeat during rebalance

2020-08-12 Thread GitBox
ableegoldman commented on pull request #9173: URL: https://github.com/apache/kafka/pull/9173#issuecomment-673143739 Closing this since I didn't see the existing PR for this bugfix: https://github.com/apache/kafka/pull/8834 T

[GitHub] [kafka] ableegoldman closed pull request #9173: KAFKA-10122: Consumer should allow heartbeat during rebalance

2020-08-12 Thread GitBox
ableegoldman closed pull request #9173: URL: https://github.com/apache/kafka/pull/9173 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] rajinisivaram commented on pull request #9168: MINOR: Ensure same version of scala library is used for compile and at runtime

2020-08-12 Thread GitBox
rajinisivaram commented on pull request #9168: URL: https://github.com/apache/kafka/pull/9168#issuecomment-673144455 @ijuma Thanks for the review, streams test failures not related. Merging to trunk, 2.6, 2,5 and 2.4. This i

[GitHub] [kafka] rajinisivaram merged pull request #9168: MINOR: Ensure same version of scala library is used for compile and at runtime

2020-08-12 Thread GitBox
rajinisivaram merged pull request #9168: URL: https://github.com/apache/kafka/pull/9168 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] rhauch commented on a change in pull request #9172: KAFKA-10387: Fix inclusion of transformation configs when topic creation is enabled in Connect

2020-08-12 Thread GitBox
rhauch commented on a change in pull request #9172: URL: https://github.com/apache/kafka/pull/9172#discussion_r469586764 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/integration/TransformationIntegrationTest.java ## @@ -318,4 +323,61 @@ public void test

[GitHub] [kafka] svudutala commented on pull request #7898: KAFKA-9366: Change log4j dependency into log4j2

2020-08-12 Thread GitBox
svudutala commented on pull request #7898: URL: https://github.com/apache/kafka/pull/7898#issuecomment-673145549 @dongjinleekr What is timeline for merging this PR and making this upgrade available? This is an automated mes

[GitHub] [kafka] ableegoldman commented on a change in pull request #8834: MINOR: Do not disable heartbeat during Rebalance

2020-08-12 Thread GitBox
ableegoldman commented on a change in pull request #8834: URL: https://github.com/apache/kafka/pull/8834#discussion_r469587219 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java ## @@ -528,7 +528,6 @@ public void onFailure(

  1   2   >