[jira] [Commented] (KAFKA-4628) Support KTable/GlobalKTable Joins

2020-11-03 Thread Hartmut Armbruster (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225245#comment-17225245 ] Hartmut Armbruster commented on KAFKA-4628: --- Many Thanks, so how about the conc

[GitHub] [kafka] dengziming opened a new pull request #9546: MONOR: optimeze PartitionStateMachine handleStateChanges

2020-11-03 Thread GitBox
dengziming opened a new pull request #9546: URL: https://github.com/apache/kafka/pull/9546 The logic of state change to OfflinePartition or NonExistentPartition are identical, so just merge them. If fact, the logic of the case NewPartition is also the same, but have different log format

[GitHub] [kafka] junrao commented on a change in pull request #7751: KAFKA-7987: Reinitialize ZookeeperClient after auth failures

2020-11-03 Thread GitBox
junrao commented on a change in pull request #7751: URL: https://github.com/apache/kafka/pull/7751#discussion_r516157977 ## File path: core/src/main/scala/kafka/zookeeper/ZooKeeperClient.scala ## @@ -81,7 +85,8 @@ class ZooKeeperClient(connectString: String, private val zNod

[GitHub] [kafka] hachikuji commented on a change in pull request #9539: KAFKA-10634: Adding LeaderId to Voters list in LeaderChangeMessage

2020-11-03 Thread GitBox
hachikuji commented on a change in pull request #9539: URL: https://github.com/apache/kafka/pull/9539#discussion_r516215666 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -317,6 +317,9 @@ private void appendLeaderChangeMessage(LeaderState stat

[GitHub] [kafka] tombentley commented on pull request #9433: KAFKA-10607: Consistent behaviour for response errorCounts()

2020-11-03 Thread GitBox
tombentley commented on pull request #9433: URL: https://github.com/apache/kafka/pull/9433#issuecomment-720421263 @chia7712 fixed, thanks. This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [kafka] junrao commented on a change in pull request #5346: Kafka 6432: make index lookup more cache friendly

2020-11-03 Thread GitBox
junrao commented on a change in pull request #5346: URL: https://github.com/apache/kafka/pull/5346#discussion_r516177126 ## File path: core/src/main/scala/kafka/log/AbstractIndex.scala ## @@ -44,9 +44,67 @@ abstract class AbstractIndex[K, V](@volatile var file: File, val baseO

[GitHub] [kafka] rajinisivaram commented on pull request #7751: KAFKA-7987: Reinitialize ZookeeperClient after auth failures

2020-11-03 Thread GitBox
rajinisivaram commented on pull request #7751: URL: https://github.com/apache/kafka/pull/7751#issuecomment-720401081 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] vamossagar12 commented on a change in pull request #9539: KAFKA-10634: Adding LeaderId to Voters list in LeaderChangeMessage

2020-11-03 Thread GitBox
vamossagar12 commented on a change in pull request #9539: URL: https://github.com/apache/kafka/pull/9539#discussion_r516438813 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -317,6 +317,9 @@ private void appendLeaderChangeMessage(LeaderState s

[GitHub] [kafka] wcarlson5 commented on pull request #9543: makes the Stream thread list resizable

2020-11-03 Thread GitBox
wcarlson5 commented on pull request #9543: URL: https://github.com/apache/kafka/pull/9543#issuecomment-720787984 @cadonna First part to make the streamThreads list resizable This is an automated message from the Apache Git Se

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-11-03 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r516288558 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] abbccdda closed pull request #8677: KAFKA-9999: Make internal topic creation error non-fatal

2020-11-03 Thread GitBox
abbccdda closed pull request #8677: URL: https://github.com/apache/kafka/pull/8677 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] Iskuskov opened a new pull request #9541: KAFKA-10675: Add schema name to ConnectSchema.validateValue() error message

2020-11-03 Thread GitBox
Iskuskov opened a new pull request #9541: URL: https://github.com/apache/kafka/pull/9541 The following error message ```java org.apache.kafka.connect.errors.DataException: Invalid Java object for schema type INT64: class java.lang.Long for field: "moderate_time" ``` can be confu

[GitHub] [kafka] dengziming commented on pull request #9546: MONOR: optimeze PartitionStateMachine handleStateChanges

2020-11-03 Thread GitBox
dengziming commented on pull request #9546: URL: https://github.com/apache/kafka/pull/9546#issuecomment-720905982 @chia7712 , Hi, PTLA. This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [kafka] bbejeck closed pull request #9542: MINOR: Update trunk docs to match site docs

2020-11-03 Thread GitBox
bbejeck closed pull request #9542: URL: https://github.com/apache/kafka/pull/9542 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [kafka] omkreddy commented on a change in pull request #9540: KAFKA-10669: Make CurrentLeaderEpoch field ignorable and set MaxNumOffsets field default to 1

2020-11-03 Thread GitBox
omkreddy commented on a change in pull request #9540: URL: https://github.com/apache/kafka/pull/9540#discussion_r516103337 ## File path: core/src/test/scala/unit/kafka/server/ListOffsetsRequestTest.scala ## @@ -143,7 +143,13 @@ class ListOffsetsRequestTest extends BaseRequestTe

[GitHub] [kafka] kkonstantine commented on a change in pull request #6820: MINOR: move connectorConfig to AbstractHerder

2020-11-03 Thread GitBox
kkonstantine commented on a change in pull request #6820: URL: https://github.com/apache/kafka/pull/6820#discussion_r516445120 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java ## @@ -751,14 +751,8 @@ public Void ca

[GitHub] [kafka] chia7712 merged pull request #9535: MINOR: remove redundant return statement

2020-11-03 Thread GitBox
chia7712 merged pull request #9535: URL: https://github.com/apache/kafka/pull/9535 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] jsancio commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-11-03 Thread GitBox
jsancio commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r516172571 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -1017,12 +1028,9 @@ private boolean handleFetchResponse(

[GitHub] [kafka] chia7712 commented on a change in pull request #7641: MINOR: Add a unit test for GroupMetadataManager#offsetExpiredSensor

2020-11-03 Thread GitBox
chia7712 commented on a change in pull request #7641: URL: https://github.com/apache/kafka/pull/7641#discussion_r516399560 ## File path: core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala ## @@ -791,52 +797,52 @@ class GroupMetadataManager(brokerId: Int,

[GitHub] [kafka] dengziming commented on a change in pull request #9531: KAFKA-10661; Add new resigned state for graceful shutdown/initialization

2020-11-03 Thread GitBox
dengziming commented on a change in pull request #9531: URL: https://github.com/apache/kafka/pull/9531#discussion_r516385415 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -1560,9 +1558,34 @@ private long pollLeader(long currentTimeMs) {

[GitHub] [kafka] olafurpg commented on a change in pull request #7452: KAFKA-8991: Enable scalac optimizer

2020-11-03 Thread GitBox
olafurpg commented on a change in pull request #7452: URL: https://github.com/apache/kafka/pull/7452#discussion_r516618369 ## File path: build.gradle ## @@ -419,6 +419,22 @@ subprojects { "-Xlint:constant", "-Xlint:unused" ] + + // Inline more aggr

[GitHub] [kafka] chia7712 commented on a change in pull request #9536: MINOR: KIP-584: Remove admin client facility to read features from controller

2020-11-03 Thread GitBox
chia7712 commented on a change in pull request #9536: URL: https://github.com/apache/kafka/pull/9536#discussion_r515749438 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java ## @@ -4353,11 +4353,8 @@ void handleFailure(Throwable throwable)

[GitHub] [kafka] bbejeck opened a new pull request #9544: MINOR: Add back section taken out by mistake

2020-11-03 Thread GitBox
bbejeck opened a new pull request #9544: URL: https://github.com/apache/kafka/pull/9544 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing s

[GitHub] [kafka] chia7712 commented on pull request #9527: MINOR: avoid unnecessary conversion and tuple when updating error met…

2020-11-03 Thread GitBox
chia7712 commented on pull request #9527: URL: https://github.com/apache/kafka/pull/9527#issuecomment-721002109 the failed tests are unrelated to this PR. rebase to trigger QA This is an automated message from the Apache Git

[GitHub] [kafka] scanterog commented on pull request #9545: [mm2] Allow Checkpoints for consumers using static partition assignments

2020-11-03 Thread GitBox
scanterog commented on pull request #9545: URL: https://github.com/apache/kafka/pull/9545#issuecomment-720844769 cc: @ryannedolan who added this filter and might have more context on why this was needed. cc: @mimaison for more eyes on this :) ---

[GitHub] [kafka] mjsax commented on a change in pull request #9000: KAFKA-10036 Improve handling and documentation of Suppliers

2020-11-03 Thread GitBox
mjsax commented on a change in pull request #9000: URL: https://github.com/apache/kafka/pull/9000#discussion_r516239935 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/ValueTransformerSupplier.java ## @@ -33,12 +38,17 @@ * @see TransformerSupplier * @s

[GitHub] [kafka] chia7712 commented on pull request #6820: MINOR: move connectorConfig to AbstractHerder

2020-11-03 Thread GitBox
chia7712 commented on pull request #6820: URL: https://github.com/apache/kafka/pull/6820#issuecomment-720874808 @ivanyu nice catching. Could you fix those conflicting files? I'd like to commit it :) This is an automated mess

[GitHub] [kafka] chia7712 merged pull request #6647: MINOR: Fix an example in the Kafka Streams tutorial to be compilable

2020-11-03 Thread GitBox
chia7712 merged pull request #6647: URL: https://github.com/apache/kafka/pull/6647 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] ableegoldman edited a comment on pull request #9515: KAFKA-10651: read offsets directly from checkpoint for uninitialized tasks

2020-11-03 Thread GitBox
ableegoldman edited a comment on pull request #9515: URL: https://github.com/apache/kafka/pull/9515#issuecomment-719848567 Cherry-picked to 2.7 (cc/ @bbejeck) and 2.6 This is an automated message from the Apache Git Service.

[GitHub] [kafka] bbejeck opened a new pull request #9542: MINOR: Update trunk docs to match site docs

2020-11-03 Thread GitBox
bbejeck opened a new pull request #9542: URL: https://github.com/apache/kafka/pull/9542 Migration of changes for AK site upgrade done on site but not migrated to AK trunk *Summary of testing strategy (including rationale) for the feature or bug fix. Unit and/or integration test

[GitHub] [kafka] chia7712 closed pull request #5490: MINOR: Fix typo in TaskManager

2020-11-03 Thread GitBox
chia7712 closed pull request #5490: URL: https://github.com/apache/kafka/pull/5490 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] chia7712 commented on pull request #7266: MINOR: Fix capitalization inconsistency in ConfigCommand.scala

2020-11-03 Thread GitBox
chia7712 commented on pull request #7266: URL: https://github.com/apache/kafka/pull/7266#issuecomment-720873421 @KevinLiLu Thanks for your patch. I'm going to close this PR since this issue was fixed by https://github.com/apache/kafka/commit/7f35a6713434dd7f2ccd3897aef825d34582beab. Feel

[GitHub] [kafka] hachikuji merged pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-11-03 Thread GitBox
hachikuji merged pull request #9482: URL: https://github.com/apache/kafka/pull/9482 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] wcarlson5 opened a new pull request #9543: makes the Stream thread list resizable

2020-11-03 Thread GitBox
wcarlson5 opened a new pull request #9543: URL: https://github.com/apache/kafka/pull/9543 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[GitHub] [kafka] ijuma commented on a change in pull request #7452: KAFKA-8991: Enable scalac optimizer

2020-11-03 Thread GitBox
ijuma commented on a change in pull request #7452: URL: https://github.com/apache/kafka/pull/7452#discussion_r516664863 ## File path: build.gradle ## @@ -419,6 +419,22 @@ subprojects { "-Xlint:constant", "-Xlint:unused" ] + + // Inline more aggress

[GitHub] [kafka] kowshik commented on a change in pull request #9536: MINOR: KIP-584: Remove admin client facility to read features from controller

2020-11-03 Thread GitBox
kowshik commented on a change in pull request #9536: URL: https://github.com/apache/kafka/pull/9536#discussion_r515769357 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java ## @@ -4353,11 +4353,8 @@ void handleFailure(Throwable throwable)

[GitHub] [kafka] bbejeck edited a comment on pull request #9468: KAFKA-10454 / (2.6) Update copartitionSourceGroups when optimization algorithm is triggered

2020-11-03 Thread GitBox
bbejeck edited a comment on pull request #9468: URL: https://github.com/apache/kafka/pull/9468#issuecomment-720473323 > What's the status of this PR? can we get it merged this week? @mimaison yes, I've been waiting on the Jenkins build which I believe is fixed now. Once I can get th

[GitHub] [kafka] chia7712 merged pull request #9180: MINOR: corrected unit tests in AbstractConfigTest.java- fixed invalid assertions

2020-11-03 Thread GitBox
chia7712 merged pull request #9180: URL: https://github.com/apache/kafka/pull/9180 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] bbejeck merged pull request #9468: KAFKA-10454 / (2.6) Update copartitionSourceGroups when optimization algorithm is triggered

2020-11-03 Thread GitBox
bbejeck merged pull request #9468: URL: https://github.com/apache/kafka/pull/9468 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [kafka] scanterog edited a comment on pull request #9545: [mm2] Allow Checkpoints for consumers using static partition assignments

2020-11-03 Thread GitBox
scanterog edited a comment on pull request #9545: URL: https://github.com/apache/kafka/pull/9545#issuecomment-720844769 cc: @ryannedolan who added this filter and might have more context on why this was needed. cc: @mimaison for more eyes on this :) Update: tests are working fine

[GitHub] [kafka] RamanVerma commented on pull request #9364: KAFKA-10471 Mark broker crash during log loading as unclean shutdown

2020-11-03 Thread GitBox
RamanVerma commented on pull request #9364: URL: https://github.com/apache/kafka/pull/9364#issuecomment-720697940 @junrao I don't think the JDK 15 failure is related to my change. - We see that tests pass till JDK 11 - I have run the test in my IDE with AdoptOpenJDK15 and it passed

[GitHub] [kafka] chia7712 closed pull request #7266: MINOR: Fix capitalization inconsistency in ConfigCommand.scala

2020-11-03 Thread GitBox
chia7712 closed pull request #7266: URL: https://github.com/apache/kafka/pull/7266 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] rajinisivaram commented on a change in pull request #7751: KAFKA-7987: Reinitialize ZookeeperClient after auth failures

2020-11-03 Thread GitBox
rajinisivaram commented on a change in pull request #7751: URL: https://github.com/apache/kafka/pull/7751#discussion_r515933228 ## File path: core/src/main/scala/kafka/zookeeper/ZooKeeperClient.scala ## @@ -437,8 +437,14 @@ class ZooKeeperClient(connectString: String,

[GitHub] [kafka] chia7712 closed pull request #4817: MINOR: Fix Findbugs issue about useless object in KafkaAdminClient

2020-11-03 Thread GitBox
chia7712 closed pull request #4817: URL: https://github.com/apache/kafka/pull/4817 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] splett2 commented on a change in pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-03 Thread GitBox
splett2 commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r516301423 ## File path: core/src/test/scala/integration/kafka/network/DynamicConnectionQuotaTest.scala ## @@ -240,6 +256,16 @@ class DynamicConnectionQuotaTest exten

[GitHub] [kafka] ivanyu commented on pull request #6820: MINOR: move connectorConfig to AbstractHerder

2020-11-03 Thread GitBox
ivanyu commented on pull request #6820: URL: https://github.com/apache/kafka/pull/6820#issuecomment-720930041 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [kafka] ivanyu edited a comment on pull request #6820: MINOR: move connectorConfig to AbstractHerder

2020-11-03 Thread GitBox
ivanyu edited a comment on pull request #6820: URL: https://github.com/apache/kafka/pull/6820#issuecomment-720997301 retest this please UPD: ah, I guess it doesn't work anymore... This is an automated message from the

[GitHub] [kafka] chia7712 commented on pull request #9501: MINOR: move the test cases which don't need brokers from TopicCommand…

2020-11-03 Thread GitBox
chia7712 commented on pull request #9501: URL: https://github.com/apache/kafka/pull/9501#issuecomment-720870283 @dajac Could you take a look? This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [kafka] chia7712 commented on pull request #9180: MINOR: corrected unit tests in AbstractConfigTest.java- fixed invalid assertions

2020-11-03 Thread GitBox
chia7712 commented on pull request #9180: URL: https://github.com/apache/kafka/pull/9180#issuecomment-720937214 @sanketfajage Thanks for your patch. Merge it to trunk. This is an automated message from the Apache Git Service.

[GitHub] [kafka] ableegoldman edited a comment on pull request #9534: KAFKA-10664: Delete existing checkpoint when writing empty offsets

2020-11-03 Thread GitBox
ableegoldman edited a comment on pull request #9534: URL: https://github.com/apache/kafka/pull/9534#issuecomment-719848646 Cherry-picked to 2.7 (cc/ @bbejeck) and 2.6 This is an automated message from the Apache Git Service.

[GitHub] [kafka] hachikuji commented on a change in pull request #9103: KAFKA-10181: Use Envelope RPC to do redirection for (Incremental)AlterConfig, AlterClientQuota and CreateTopics

2020-11-03 Thread GitBox
hachikuji commented on a change in pull request #9103: URL: https://github.com/apache/kafka/pull/9103#discussion_r516363883 ## File path: core/src/test/scala/integration/kafka/server/DynamicBrokerReconfigurationTest.scala ## @@ -70,12 +74,8 @@ import scala.collection.mutable.A

[GitHub] [kafka] chia7712 commented on pull request #9546: MONOR: optimeze PartitionStateMachine handleStateChanges

2020-11-03 Thread GitBox
chia7712 commented on pull request #9546: URL: https://github.com/apache/kafka/pull/9546#issuecomment-720937420 @dengziming Could you rebase PR to trigger QA again? This is an automated message from the Apache Git Service. To

[GitHub] [kafka] dajac commented on a change in pull request #9540: KAFKA-10669: Make CurrentLeaderEpoch field ignorable and set MaxNumOffsets field default to 1

2020-11-03 Thread GitBox
dajac commented on a change in pull request #9540: URL: https://github.com/apache/kafka/pull/9540#discussion_r515806990 ## File path: clients/src/test/java/org/apache/kafka/common/requests/RequestResponseTest.java ## @@ -1261,10 +1261,9 @@ private ListOffsetRequest createListO

[GitHub] [kafka] ivanyu commented on a change in pull request #6820: MINOR: move connectorConfig to AbstractHerder

2020-11-03 Thread GitBox
ivanyu commented on a change in pull request #6820: URL: https://github.com/apache/kafka/pull/6820#discussion_r516447945 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java ## @@ -751,14 +751,8 @@ public Void call() t

[GitHub] [kafka] omkreddy closed pull request #9540: KAFKA-10669: Make CurrentLeaderEpoch field ignorable and set MaxNumOffsets field default to 1

2020-11-03 Thread GitBox
omkreddy closed pull request #9540: URL: https://github.com/apache/kafka/pull/9540 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] hachikuji commented on a change in pull request #9531: KAFKA-10661; Add new resigned state for graceful shutdown/initialization

2020-11-03 Thread GitBox
hachikuji commented on a change in pull request #9531: URL: https://github.com/apache/kafka/pull/9531#discussion_r516198568 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -1560,9 +1558,34 @@ private long pollLeader(long currentTimeMs) {

[GitHub] [kafka] mimaison commented on pull request #9468: KAFKA-10454 / (2.6) Update copartitionSourceGroups when optimization algorithm is triggered

2020-11-03 Thread GitBox
mimaison commented on pull request #9468: URL: https://github.com/apache/kafka/pull/9468#issuecomment-720400557 @bbejeck What's the status of this PR? can we get it merged this week? This is an automated message from the Apac

[GitHub] [kafka] bbejeck commented on pull request #9468: KAFKA-10454 / (2.6) Update copartitionSourceGroups when optimization algorithm is triggered

2020-11-03 Thread GitBox
bbejeck commented on pull request #9468: URL: https://github.com/apache/kafka/pull/9468#issuecomment-720473323 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [kafka] junrao merged pull request #9364: KAFKA-10471 Mark broker crash during log loading as unclean shutdown

2020-11-03 Thread GitBox
junrao merged pull request #9364: URL: https://github.com/apache/kafka/pull/9364 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] scanterog opened a new pull request #9545: [mm2] Allow Checkpoints for consumers using static partition assignments

2020-11-03 Thread GitBox
scanterog opened a new pull request #9545: URL: https://github.com/apache/kafka/pull/9545 Currently mm2 is not generating checkpoints for upstream consumers using static partition assignments. The reason is that these consumers are being explicitly filtered out. I couldn't find why this fi

[GitHub] [kafka] junrao commented on pull request #9364: KAFKA-10471 Mark broker crash during log loading as unclean shutdown

2020-11-03 Thread GitBox
junrao commented on pull request #9364: URL: https://github.com/apache/kafka/pull/9364#issuecomment-720623777 @RamanVerma Was the test failure kafka.admin.ReassignPartitionsIntegrationTest.testLogDirReassignment in JDK 15 related to this PR? -

[GitHub] [kafka] chia7712 commented on pull request #4817: MINOR: Fix Findbugs issue about useless object in KafkaAdminClient

2020-11-03 Thread GitBox
chia7712 commented on pull request #4817: URL: https://github.com/apache/kafka/pull/4817#issuecomment-720878512 @attila-s Thanks for your patch. This issue was fixed by https://github.com/apache/kafka/commit/49db5a63c043b50c10c2dfd0648f8d74ee917b6a. please feel free to ping me if you have

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-11-03 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r516151645 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -1017,12 +1028,9 @@ private boolean handleFetchResponse(

[GitHub] [kafka] chia7712 commented on pull request #5490: MINOR: Fix typo in TaskManager

2020-11-03 Thread GitBox
chia7712 commented on pull request #5490: URL: https://github.com/apache/kafka/pull/5490#issuecomment-720877498 @lucapette Thanks for your patch. I'm going to close this PR since that issue was fixed by https://github.com/apache/kafka/commit/4090f9a2b0a95e4da127e4786007542276d97520. Feel

[GitHub] [kafka] junrao merged pull request #9536: MINOR: KIP-584: Remove admin client facility to read features from controller

2020-11-03 Thread GitBox
junrao merged pull request #9536: URL: https://github.com/apache/kafka/pull/9536 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] dajac opened a new pull request #9547: KAFKA-9630; Migrate OffsetsForLeaderEpochRequest/Response to the auto-generated protocol

2020-11-03 Thread GitBox
dajac opened a new pull request #9547: URL: https://github.com/apache/kafka/pull/9547 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade n

[GitHub] [kafka] omkreddy commented on pull request #9540: KAFKA-10669: Make CurrentLeaderEpoch field ignorable and set MaxNumOffsets field default to 1

2020-11-03 Thread GitBox
omkreddy commented on pull request #9540: URL: https://github.com/apache/kafka/pull/9540#issuecomment-720433673 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [kafka] dengziming commented on a change in pull request #9473: KAFKA-10545: Create topic IDs in ZooKeeper and Controller

2020-11-03 Thread GitBox
dengziming commented on a change in pull request #9473: URL: https://github.com/apache/kafka/pull/9473#discussion_r516549058 ## File path: core/src/main/scala/kafka/controller/ControllerContext.scala ## @@ -83,6 +83,8 @@ class ControllerContext { var epochZkVersion: Int = Ka

[GitHub] [kafka] kowshik commented on pull request #9536: MINOR: KIP-584: Remove admin client facility to read features from controller

2020-11-03 Thread GitBox
kowshik commented on pull request #9536: URL: https://github.com/apache/kafka/pull/9536#issuecomment-720276930 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [kafka] dongjinleekr commented on pull request #9414: KAFKA-10585: Kafka Streams should clean up the state store directory from cleanup

2020-11-03 Thread GitBox
dongjinleekr commented on pull request #9414: URL: https://github.com/apache/kafka/pull/9414#issuecomment-721135460 Hi @vvcephei, Sorry for being late. Defending myself, I have been too busy to finalize my project last week, and it ended yesterday. Sure, I am now resolving the broken

[GitHub] [kafka] soarez commented on a change in pull request #9000: KAFKA-10036 Improve handling and documentation of Suppliers

2020-11-03 Thread GitBox
soarez commented on a change in pull request #9000: URL: https://github.com/apache/kafka/pull/9000#discussion_r516680210 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/ValueTransformerSupplier.java ## @@ -33,12 +38,17 @@ * @see TransformerSupplier * @

[GitHub] [kafka] soarez commented on pull request #9000: KAFKA-10036 Improve handling and documentation of Suppliers

2020-11-03 Thread GitBox
soarez commented on pull request #9000: URL: https://github.com/apache/kafka/pull/9000#issuecomment-721129774 Thanks for giving this another look @mjsax This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] olafurpg opened a new pull request #9548: Disable inlining of Scala library methods.

2020-11-03 Thread GitBox
olafurpg opened a new pull request #9548: URL: https://github.com/apache/kafka/pull/9548 Previously, the Scala compiler was configured to inline all usages of symbols in under the `scala` package. This is problematic because it means that published Kafka jars must run with the exact sa

[GitHub] [kafka] olafurpg commented on a change in pull request #7452: KAFKA-8991: Enable scalac optimizer

2020-11-03 Thread GitBox
olafurpg commented on a change in pull request #7452: URL: https://github.com/apache/kafka/pull/7452#discussion_r516703080 ## File path: build.gradle ## @@ -419,6 +419,22 @@ subprojects { "-Xlint:constant", "-Xlint:unused" ] + + // Inline more aggr

[GitHub] [kafka] ijuma commented on pull request #9548: Disable inlining of Scala library methods.

2020-11-03 Thread GitBox
ijuma commented on pull request #9548: URL: https://github.com/apache/kafka/pull/9548#issuecomment-721150214 @olafurpg Thanks for the PR. Can you please elaborate why there are conflicting versions of the Scala library when the core jar is used? The core jar doesn't export a public API and

[GitHub] [kafka] ijuma commented on a change in pull request #7452: KAFKA-8991: Enable scalac optimizer

2020-11-03 Thread GitBox
ijuma commented on a change in pull request #7452: URL: https://github.com/apache/kafka/pull/7452#discussion_r516710457 ## File path: build.gradle ## @@ -419,6 +419,22 @@ subprojects { "-Xlint:constant", "-Xlint:unused" ] + + // Inline more aggress

[GitHub] [kafka] ijuma commented on a change in pull request #7452: KAFKA-8991: Enable scalac optimizer

2020-11-03 Thread GitBox
ijuma commented on a change in pull request #7452: URL: https://github.com/apache/kafka/pull/7452#discussion_r516710457 ## File path: build.gradle ## @@ -419,6 +419,22 @@ subprojects { "-Xlint:constant", "-Xlint:unused" ] + + // Inline more aggress

[GitHub] [kafka] wcarlson5 commented on pull request #9543: makes the Stream thread list resizable

2020-11-03 Thread GitBox
wcarlson5 commented on pull request #9543: URL: https://github.com/apache/kafka/pull/9543#issuecomment-721154784 retest this please This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [kafka] olafurpg commented on pull request #9548: Disable inlining of Scala library methods.

2020-11-03 Thread GitBox
olafurpg commented on pull request #9548: URL: https://github.com/apache/kafka/pull/9548#issuecomment-721182321 Even if users can't directly reference APIs of the core Kafka jar, the bytecode of the core jar is still evaluated at runtime. With the inliner enabled, the core Kafka jar direct

[GitHub] [kafka] olafurpg commented on pull request #9548: Disable inlining of Scala library methods.

2020-11-03 Thread GitBox
olafurpg commented on pull request #9548: URL: https://github.com/apache/kafka/pull/9548#issuecomment-721184420 To put it differently, enabling the inliner allows Kafka to access private symbols from `scala-library.jar`. The upstream commit https://github.com/scala/scala/commit/44318c8959b

[GitHub] [kafka] ijuma commented on pull request #9548: Disable inlining of Scala library methods.

2020-11-03 Thread GitBox
ijuma commented on pull request #9548: URL: https://github.com/apache/kafka/pull/9548#issuecomment-721197381 I understand that, the point is that the core jar is not meant to be a library. It has no public APIs. The clients jar is meant to be a library. ---

[GitHub] [kafka] lbradstreet commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-03 Thread GitBox
lbradstreet commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-721201536 @chia7712 @hachikuji for the ProduceResponse handling, is this the overall broker side regression since you need both the construction and toStruct? ``` construction regr

[GitHub] [kafka] ijuma commented on pull request #9548: Disable inlining of Scala library methods.

2020-11-03 Thread GitBox
ijuma commented on pull request #9548: URL: https://github.com/apache/kafka/pull/9548#issuecomment-721201268 To elaborate a bit more, there are two cases where two cases where this can cause problems that I can think of: 1. If `core` is used as a library and there is a mismatch betwe

[GitHub] [kafka] tombentley opened a new pull request #9549: KIP-145: Add SMTs, HeaderFrom, DropHeaders and InsertHeader

2020-11-03 Thread GitBox
tombentley opened a new pull request #9549: URL: https://github.com/apache/kafka/pull/9549 These SMTs were originally specified in KIP-145 but never implemented at the time. HeaderTo is not included since its original specification doesn't deal with the fact that there can be >1 hea

[GitHub] [kafka] tombentley commented on pull request #9549: KIP-145: Add SMTs, HeaderFrom, DropHeaders and InsertHeader

2020-11-03 Thread GitBox
tombentley commented on pull request #9549: URL: https://github.com/apache/kafka/pull/9549#issuecomment-721226685 @kkonstantine would you be able to review this? These SMTs were originally specified in [KIP145](https://cwiki.apache.org/confluence/display/KAFKA/KIP-145+-+Expose+Record+Heade

[GitHub] [kafka] tombentley commented on pull request #9266: KAFKA-10469: Resolve logger levels hierarchically

2020-11-03 Thread GitBox
tombentley commented on pull request #9266: URL: https://github.com/apache/kafka/pull/9266#issuecomment-721228240 Maybe @vvcephei, @mimaison or @gwenshap could take a look at this since you voted on the KIP? Thanks! This is

[GitHub] [kafka] tombentley commented on pull request #9266: KAFKA-10469: Resolve logger levels hierarchically

2020-11-03 Thread GitBox
tombentley commented on pull request #9266: URL: https://github.com/apache/kafka/pull/9266#issuecomment-721229160 We just need to revert 75e5358 (which reverted the original merged). Do you need an new PR for that? This is a

[GitHub] [kafka] ijuma commented on pull request #9266: KAFKA-10469: Resolve logger levels hierarchically

2020-11-03 Thread GitBox
ijuma commented on pull request #9266: URL: https://github.com/apache/kafka/pull/9266#issuecomment-721229952 I can reapply the change soon. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] tombentley commented on pull request #9365: KAFKA-10566: Fix erroneous config usage warnings

2020-11-03 Thread GitBox
tombentley commented on pull request #9365: URL: https://github.com/apache/kafka/pull/9365#issuecomment-721230361 Maybe @chia7712, @dajac or @mimaison could take a look at this? This is an automated message from the Apache Gi

[GitHub] [kafka] tombentley commented on pull request #9266: KAFKA-10469: Resolve logger levels hierarchically

2020-11-03 Thread GitBox
tombentley commented on pull request #9266: URL: https://github.com/apache/kafka/pull/9266#issuecomment-721231570 Great, thanks Ismael. This is an automated message from the Apache Git Service. To respond to the message, plea

[jira] [Created] (KAFKA-10678) Re-deploying Streams app causes rebalance and task migration

2020-11-03 Thread Bradley Peterson (Jira)
Bradley Peterson created KAFKA-10678: Summary: Re-deploying Streams app causes rebalance and task migration Key: KAFKA-10678 URL: https://issues.apache.org/jira/browse/KAFKA-10678 Project: Kafka

[jira] [Updated] (KAFKA-10678) Re-deploying Streams app causes rebalance and task migration

2020-11-03 Thread Bradley Peterson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bradley Peterson updated KAFKA-10678: - Attachment: broker before after > Re-deploying Streams a

[jira] [Resolved] (KAFKA-10633) Constant probing rebalances in Streams 2.6

2020-11-03 Thread Bradley Peterson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bradley Peterson resolved KAFKA-10633. -- Fix Version/s: 2.6.1 Resolution: Fixed Closing this ticket as the issue is fixe

[jira] [Commented] (KAFKA-10515) NPE: Foreign key join serde may not be initialized with default serde if application is distributed

2020-11-03 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225532#comment-17225532 ] John Roesler commented on KAFKA-10515: -- [https://github.com/apache/kafka/pull/9467]

[GitHub] [kafka] vvcephei commented on pull request #9414: KAFKA-10585: Kafka Streams should clean up the state store directory from cleanup

2020-11-03 Thread GitBox
vvcephei commented on pull request #9414: URL: https://github.com/apache/kafka/pull/9414#issuecomment-721239457 Perfect. Thanks, @dongjinleekr ! No defense is necessary :) I was just wondering if it was still on your radar. Thanks again for the contribution!

[GitHub] [kafka] brbrown25 edited a comment on pull request #9057: KAFKA-10299: Implementing Kafka Connect Hash SMT to allow for hashing…

2020-11-03 Thread GitBox
brbrown25 edited a comment on pull request #9057: URL: https://github.com/apache/kafka/pull/9057#issuecomment-717322569 - [x] update with latest master - [ ] implement nested field support - [ ] implement salt support - [ ] update rat configuration

[GitHub] [kafka] mingaliu commented on pull request #9548: Disable inlining of Scala library methods.

2020-11-03 Thread GitBox
mingaliu commented on pull request #9548: URL: https://github.com/apache/kafka/pull/9548#issuecomment-721242229 @ijuma , here is the callstack, and you can see why Kafka core is pulled in from the client side: java.lang.NoClassDefFoundError: scala/math/Ordering$$anon$7 at kafka.

[GitHub] [kafka] brbrown25 edited a comment on pull request #9057: KAFKA-10299: Implementing Kafka Connect Hash SMT to allow for hashing…

2020-11-03 Thread GitBox
brbrown25 edited a comment on pull request #9057: URL: https://github.com/apache/kafka/pull/9057#issuecomment-717322569 - [x] update with latest master - [ ] implement nested field support - [ ] implement salt support - [x] update rat configuration

[GitHub] [kafka] olafurpg commented on pull request #9548: Disable inlining of Scala library methods.

2020-11-03 Thread GitBox
olafurpg commented on pull request #9548: URL: https://github.com/apache/kafka/pull/9548#issuecomment-721247846 Another related case (https://github.com/akka/alpakka-kafka/pull/1212) is also from using `EmbeddedKafka` as a library for testing purposes https://github.com/embeddedkafka/embed

[GitHub] [kafka] mingaliu edited a comment on pull request #9548: Disable inlining of Scala library methods.

2020-11-03 Thread GitBox
mingaliu edited a comment on pull request #9548: URL: https://github.com/apache/kafka/pull/9548#issuecomment-721242229 @ijuma , here is the callstack, and you can see why Kafka core is pulled in from the client side (due to using KafakEmbedded in testing): ``` java.lang.NoClassDe

  1   2   >