[GitHub] [kafka] huxihx opened a new pull request #8984: KAFKA-10227: Enforce cleanup policy to only contain compact or delete once

2020-07-06 Thread GitBox
huxihx opened a new pull request #8984: URL: https://github.com/apache/kafka/pull/8984 https://issues.apache.org/jira/browse/KAFKA-10227 This patch fixes both TopicCommand and ConfigCommand. When duplicate items are specified in `cleanup.policy`, they will be removed automatically an

[GitHub] [kafka] huxihx commented on pull request #8966: KAFKA-10220: add null check for configurationKey

2020-07-06 Thread GitBox
huxihx commented on pull request #8966: URL: https://github.com/apache/kafka/pull/8966#issuecomment-654076753 retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] huxihx commented on pull request #8957: KAFKA-5235: GetOffsetShell: support for multiple topics and consumer configuration override

2020-07-06 Thread GitBox
huxihx commented on pull request #8957: URL: https://github.com/apache/kafka/pull/8957#issuecomment-654083327 I do not see the counterpart for Windows. We should add `kafka-get-offsets.bat` as well. This is an automated mess

[GitHub] [kafka] gunnarmorling closed pull request #8895: KAFKA-8398 Avoiding NPE in ByteBufferUnmapper#unmap()

2020-07-06 Thread GitBox
gunnarmorling closed pull request #8895: URL: https://github.com/apache/kafka/pull/8895 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] gunnarmorling commented on pull request #8895: KAFKA-8398 Avoiding NPE in ByteBufferUnmapper#unmap()

2020-07-06 Thread GitBox
gunnarmorling commented on pull request #8895: URL: https://github.com/apache/kafka/pull/8895#issuecomment-654089873 Makes sense. So I'm going to close this one here, looking forward to the other fix being released soon. Thanks!

[GitHub] [kafka] dajac opened a new pull request #8985: MINOR; alterReplicaLogDirs should not fail all the futures when only one call fails

2020-07-06 Thread GitBox
dajac opened a new pull request #8985: URL: https://github.com/apache/kafka/pull/8985 I have noticed that `alterReplicaLogDirs` fails all the unrealised futures when one of the call sent out fails (e.g. timeout). So if two calls A and B are sent out, if A fails before B comes back, all the

[jira] [Created] (KAFKA-10236) Kafka Streams | onCommit interceptor with EOS enabled

2020-07-06 Thread Vinodhini (Jira)
Vinodhini created KAFKA-10236: - Summary: Kafka Streams | onCommit interceptor with EOS enabled Key: KAFKA-10236 URL: https://issues.apache.org/jira/browse/KAFKA-10236 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-10236) Kafka Streams | onCommit interceptor with EOS enabled

2020-07-06 Thread Vinodhini (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinodhini updated KAFKA-10236: -- Description: Coming from  [https://stackoverflow.com/questions/62700075/is-there-a-way-to-get-committe

[GitHub] [kafka] rajinisivaram opened a new pull request #8986: KAFKA-10233; Add backoff after AuthorizationExceptions in consumer

2020-07-06 Thread GitBox
rajinisivaram opened a new pull request #8986: URL: https://github.com/apache/kafka/pull/8986 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgra

[GitHub] [kafka] rajinisivaram commented on a change in pull request #8986: KAFKA-10233; Add backoff after AuthorizationExceptions in consumer

2020-07-06 Thread GitBox
rajinisivaram commented on a change in pull request #8986: URL: https://github.com/apache/kafka/pull/8986#discussion_r450129557 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java ## @@ -460,15 +463,21 @@ boolean joinGroupIf

[jira] [Comment Edited] (KAFKA-10017) Flaky Test EosBetaUpgradeIntegrationTest.shouldUpgradeFromEosAlphaToEosBeta

2020-07-06 Thread huxihx (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151960#comment-17151960 ] huxihx edited comment on KAFKA-10017 at 7/6/20, 11:19 AM: -- [htt

[jira] [Commented] (KAFKA-10017) Flaky Test EosBetaUpgradeIntegrationTest.shouldUpgradeFromEosAlphaToEosBeta

2020-07-06 Thread huxihx (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151960#comment-17151960 ] huxihx commented on KAFKA-10017: [https://builds.apache.org/job/kafka-pr-jdk14-scala2.13

[GitHub] [kafka] huxihx commented on pull request #8984: KAFKA-10227: Enforce cleanup policy to only contain compact or delete once

2020-07-06 Thread GitBox
huxihx commented on pull request #8984: URL: https://github.com/apache/kafka/pull/8984#issuecomment-654175158 Seems it failed by an unrelated issue tracked in [KAFKA-10017 ](https://issues.apache.org/jira/browse/KAFKA-10017#). --

[jira] [Assigned] (KAFKA-8666) Improve Documentation on usage of Materialized config object

2020-07-06 Thread highluck (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] highluck reassigned KAFKA-8666: --- Assignee: highluck > Improve Documentation on usage of Materialized config object >

[jira] [Assigned] (KAFKA-8666) Improve Documentation on usage of Materialized config object

2020-07-06 Thread highluck (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] highluck reassigned KAFKA-8666: --- Assignee: (was: highluck) > Improve Documentation on usage of Materialized config object > -

[GitHub] [kafka] dajac commented on pull request #8984: KAFKA-10227: Enforce cleanup policy to only contain compact or delete once

2020-07-06 Thread GitBox
dajac commented on pull request #8984: URL: https://github.com/apache/kafka/pull/8984#issuecomment-654251862 @huxihx Thanks for the PR. Have you considered doing this in the broker? Doing it in the broker has the advantage that it would work for all the clients in the ecosystem. ---

[jira] [Commented] (KAFKA-10217) Move nodeLevelSensor and storeLevelSensor methods from StreamsMetricsImpl to StreamsMetrics

2020-07-06 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152030#comment-17152030 ] Bruno Cadonna commented on KAFKA-10217: --- If the goal of this ticket is to get rid

[jira] [Commented] (KAFKA-10217) Move nodeLevelSensor and storeLevelSensor methods from StreamsMetricsImpl to StreamsMetrics

2020-07-06 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152034#comment-17152034 ] Bruno Cadonna commented on KAFKA-10217: --- To use {{nodeLevelSensor()}} in {{Streams

[GitHub] [kafka] dajac commented on pull request #8916: MINOR; Provide lowest and highest supported versions alongside with the schema in the auto-generated protocol

2020-07-06 Thread GitBox
dajac commented on pull request #8916: URL: https://github.com/apache/kafka/pull/8916#issuecomment-654262576 @ijuma Indeed, it looks similar except that that PR works only for `REQUEST` or `RESPONSE` types. In my case, I wanted to have a way to get versions for a `DATA` type to avoid havin

[GitHub] [kafka] huxihx commented on pull request #8984: KAFKA-10227: Enforce cleanup policy to only contain compact or delete once

2020-07-06 Thread GitBox
huxihx commented on pull request #8984: URL: https://github.com/apache/kafka/pull/8984#issuecomment-654286184 @dajac Thank you for the comment. Well, it is quite tempting to put all the check things on the broker side. However, we should still have the current tools refined to be robust en

[GitHub] [kafka] ijuma commented on a change in pull request #8983: KAFKA-8398: Prevent NPE in `forceUnmap`

2020-07-06 Thread GitBox
ijuma commented on a change in pull request #8983: URL: https://github.com/apache/kafka/pull/8983#discussion_r450279301 ## File path: core/src/main/scala/kafka/log/AbstractIndex.scala ## @@ -311,9 +311,11 @@ abstract class AbstractIndex(@volatile private var _file: File, val b

[GitHub] [kafka] kkonstantine commented on pull request #8973: KAFKA-10218: Stop reading config topic in every subsequent tick if catchup fails once

2020-07-06 Thread GitBox
kkonstantine commented on pull request #8973: URL: https://github.com/apache/kafka/pull/8973#issuecomment-654289206 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] kkonstantine commented on a change in pull request #8928: KAFKA-10192: Wait for REST API to become available before testing blocked connectors

2020-07-06 Thread GitBox
kkonstantine commented on a change in pull request #8928: URL: https://github.com/apache/kafka/pull/8928#discussion_r450292688 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/integration/BlockingConnectorTest.java ## @@ -76,6 +78,15 @@ public void setup()

[jira] [Assigned] (KAFKA-10221) Backport fix for KAFKA-9603 to 2.5

2020-07-06 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna reassigned KAFKA-10221: - Assignee: Bruno Cadonna > Backport fix for KAFKA-9603 to 2.5 > ---

[GitHub] [kafka] dajac commented on a change in pull request #8984: KAFKA-10227: Enforce cleanup policy to only contain compact or delete once

2020-07-06 Thread GitBox
dajac commented on a change in pull request #8984: URL: https://github.com/apache/kafka/pull/8984#discussion_r450315712 ## File path: core/src/main/scala/kafka/admin/ConfigCommand.scala ## @@ -268,6 +268,17 @@ object ConfigCommand extends Config { println(s"WARNING: The

[GitHub] [kafka] dajac commented on pull request #8984: KAFKA-10227: Enforce cleanup policy to only contain compact or delete once

2020-07-06 Thread GitBox
dajac commented on pull request #8984: URL: https://github.com/apache/kafka/pull/8984#issuecomment-654319808 @huxihx I agree that improving `TopicCommand` and `ConfigCommand` is a good thing. I think that we have to be cautious with having generic validations in the clients though as it co

[jira] [Commented] (KAFKA-10218) DistributedHerder's canReadConfigs field is never reset to true

2020-07-06 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152108#comment-17152108 ] Randall Hauch commented on KAFKA-10218: --- Thanks, [~ChrisEgerton]. You didn't reall

[jira] [Comment Edited] (KAFKA-10218) DistributedHerder's canReadConfigs field is never reset to true

2020-07-06 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152108#comment-17152108 ] Randall Hauch edited comment on KAFKA-10218 at 7/6/20, 3:44 PM: --

[GitHub] [kafka] ableegoldman commented on pull request #8962: KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory

2020-07-06 Thread GitBox
ableegoldman commented on pull request #8962: URL: https://github.com/apache/kafka/pull/8962#issuecomment-654344297 @guozhangwang I don't think this PR has anything to do with the flaky EosBetaUpgradeIntegrationTest. It might make it worse, not sure, but it definitely won't fix it without

[GitHub] [kafka] abbccdda commented on pull request #8939: MINOR: rename class `RecordTimeDefintion` to `RecordTimeDefinition`

2020-07-06 Thread GitBox
abbccdda commented on pull request #8939: URL: https://github.com/apache/kafka/pull/8939#issuecomment-654346340 Confirmed only known flaky tests are failing. Merging This is an automated message from the Apache Git Service. T

[GitHub] [kafka] ableegoldman commented on pull request #8963: KAFKA-10017: fix flaky EosBetaUpgradeIntegrationTest

2020-07-06 Thread GitBox
ableegoldman commented on pull request #8963: URL: https://github.com/apache/kafka/pull/8963#issuecomment-654347195 @guozhangwang I don't _think_ we need [8962](https://github.com/apache/kafka/pull/8962) to be merged first T

[GitHub] [kafka] abbccdda merged pull request #8939: MINOR: rename class `RecordTimeDefintion` to `RecordTimeDefinition`

2020-07-06 Thread GitBox
abbccdda merged pull request #8939: URL: https://github.com/apache/kafka/pull/8939 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] abbccdda commented on pull request #8274: KAFKA-9699: remove -E option from sed in kafka-run-class.sh

2020-07-06 Thread GitBox
abbccdda commented on pull request #8274: URL: https://github.com/apache/kafka/pull/8274#issuecomment-654347923 @mumrah Could you also take a look? This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [kafka] ableegoldman edited a comment on pull request #8963: KAFKA-10017: fix flaky EosBetaUpgradeIntegrationTest

2020-07-06 Thread GitBox
ableegoldman edited a comment on pull request #8963: URL: https://github.com/apache/kafka/pull/8963#issuecomment-653176567 > Seems still got 3 failures, while 1/3 green @abbccdda Did the EosBetaUpgradeIntegrationTest fail 3 times? Did you get the link or stacktrace before kicking off

[GitHub] [kafka] hachikuji commented on pull request #8979: KAFKA-10223; Make ReplicaNotAvailableException retriable metadata exception

2020-07-06 Thread GitBox
hachikuji commented on pull request #8979: URL: https://github.com/apache/kafka/pull/8979#issuecomment-654360639 I'm not sure the REPLICA_NOT_AVAILABLE error is super helpful as distinguished from NOT_LEADER_FOR_PARTITION. One option here is just to use the latter in all cases.

[GitHub] [kafka] vvcephei commented on a change in pull request #8971: MINOR: prune the metadata upgrade test matrix

2020-07-06 Thread GitBox
vvcephei commented on a change in pull request #8971: URL: https://github.com/apache/kafka/pull/8971#discussion_r450367500 ## File path: tests/kafkatest/tests/streams/streams_upgrade_test.py ## @@ -189,9 +185,8 @@ def test_upgrade_downgrade_brokers(self, from_version, to_versi

[jira] [Updated] (KAFKA-10230) Include min.insync.replicas in MetadataResponse to make Producer smarter in determining unavailable partitions

2020-07-06 Thread Arvin Zheng (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arvin Zheng updated KAFKA-10230: Description: check  [https://cwiki.apache.org/confluence/display/KAFKA/KIP-637%3A+Include+min.insyn

[GitHub] [kafka] hachikuji merged pull request #8970: MINOR: Improve logging around initial log loading

2020-07-06 Thread GitBox
hachikuji merged pull request #8970: URL: https://github.com/apache/kafka/pull/8970 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] vvcephei commented on a change in pull request #8962: KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory

2020-07-06 Thread GitBox
vvcephei commented on a change in pull request #8962: URL: https://github.com/apache/kafka/pull/8962#discussion_r450376026 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -270,8 +270,11 @@ public void handleAssignment(fin

[jira] [Commented] (KAFKA-10166) Excessive TaskCorruptedException seen in testing

2020-07-06 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152198#comment-17152198 ] Randall Hauch commented on KAFKA-10166: --- [~ableegoldman] do you have an updated ET

[jira] [Commented] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

2020-07-06 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152199#comment-17152199 ] Randall Hauch commented on KAFKA-10134: --- [~guozhang] do you have an updated ETA to

[jira] [Commented] (KAFKA-10143) Can no longer change replication throttle with reassignment tool

2020-07-06 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152200#comment-17152200 ] Randall Hauch commented on KAFKA-10143: --- [~hachikuji] should this remain a 2.6 blo

[jira] [Commented] (KAFKA-10166) Excessive TaskCorruptedException seen in testing

2020-07-06 Thread Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152208#comment-17152208 ] Sophie Blee-Goldman commented on KAFKA-10166: - [~rhauch] I think we can get

[jira] [Updated] (KAFKA-10236) Kafka Streams | onCommit interceptor with EOS enabled

2020-07-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-10236: Labels: needs-kip (was: ) > Kafka Streams | onCommit interceptor with EOS enabled >

[jira] [Commented] (KAFKA-10236) Kafka Streams | onCommit interceptor with EOS enabled

2020-07-06 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152215#comment-17152215 ] Matthias J. Sax commented on KAFKA-10236: - Thanks for creating the ticket. This

[GitHub] [kafka] ijuma commented on pull request #8979: KAFKA-10223; Make ReplicaNotAvailableException retriable metadata exception

2020-07-06 Thread GitBox
ijuma commented on pull request #8979: URL: https://github.com/apache/kafka/pull/8979#issuecomment-654382837 I think the confusing part is returning NOT_LEADER_FOR_PARTITION if you are trying to fetch from a follower. This i

[GitHub] [kafka] ijuma commented on pull request #8979: KAFKA-10223; Make ReplicaNotAvailableException retriable metadata exception

2020-07-06 Thread GitBox
ijuma commented on pull request #8979: URL: https://github.com/apache/kafka/pull/8979#issuecomment-654383081 Maybe we could rename `NOT_LEADER_FOR_PARTITION`. This is an automated message from the Apache Git Service. To respo

[GitHub] [kafka] junrao merged pull request #8479: KAFKA-9769: Finish operations for leaderEpoch-updated partitions up to point ZK Exception

2020-07-06 Thread GitBox
junrao merged pull request #8479: URL: https://github.com/apache/kafka/pull/8479 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Resolved] (KAFKA-9769) ReplicaManager Partition.makeFollower Increases LeaderEpoch when ZooKeeper disconnect occurs

2020-07-06 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-9769. Fix Version/s: 2.7.0 Assignee: Andrew Choi Resolution: Fixed merged the PR to trunk > Repl

[GitHub] [kafka] ijuma merged pull request #8983: KAFKA-8398: Prevent NPE in `forceUnmap`

2020-07-06 Thread GitBox
ijuma merged pull request #8983: URL: https://github.com/apache/kafka/pull/8983 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] abbccdda commented on a change in pull request #8913: KAFKA-10191 fix flaky StreamsOptimizedTest

2020-07-06 Thread GitBox
abbccdda commented on a change in pull request #8913: URL: https://github.com/apache/kafka/pull/8913#discussion_r450408632 ## File path: tests/kafkatest/tests/streams/streams_optimized_test.py ## @@ -90,6 +93,12 @@ def test_upgrade_optimized_topology(self): self.kafka.

[GitHub] [kafka] abbccdda commented on a change in pull request #8913: KAFKA-10191 fix flaky StreamsOptimizedTest

2020-07-06 Thread GitBox
abbccdda commented on a change in pull request #8913: URL: https://github.com/apache/kafka/pull/8913#discussion_r450409097 ## File path: tests/kafkatest/services/streams.py ## @@ -448,6 +452,41 @@ def __init__(self, test_context, kafka, configs):

[jira] [Commented] (KAFKA-10017) Flaky Test EosBetaUpgradeIntegrationTest.shouldUpgradeFromEosAlphaToEosBeta

2020-07-06 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152245#comment-17152245 ] Randall Hauch commented on KAFKA-10017: --- These seem to fail pretty regularly in AK

[jira] [Commented] (KAFKA-10017) Flaky Test EosBetaUpgradeIntegrationTest.shouldUpgradeFromEosAlphaToEosBeta

2020-07-06 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152247#comment-17152247 ] Randall Hauch commented on KAFKA-10017: --- [~guozhang], do you have an ETA on this?

[jira] [Assigned] (KAFKA-10017) Flaky Test EosBetaUpgradeIntegrationTest.shouldUpgradeFromEosAlphaToEosBeta

2020-07-06 Thread Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sophie Blee-Goldman reassigned KAFKA-10017: --- Assignee: Sophie Blee-Goldman (was: Matthias J. Sax) > Flaky Test EosBetaU

[GitHub] [kafka] junrao commented on a change in pull request #8974: KAFKA-10225 Increase default zk session timeout for system tests

2020-07-06 Thread GitBox
junrao commented on a change in pull request #8974: URL: https://github.com/apache/kafka/pull/8974#discussion_r450419922 ## File path: tests/kafkatest/services/kafka/kafka.py ## @@ -185,7 +185,11 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PL

[GitHub] [kafka] hachikuji commented on pull request #8979: KAFKA-10223; Make ReplicaNotAvailableException retriable metadata exception

2020-07-06 Thread GitBox
hachikuji commented on pull request #8979: URL: https://github.com/apache/kafka/pull/8979#issuecomment-654410668 I agree it's a little weird. Renaming NOT_LEADER is not a bad idea. I think what we want the error to convey is that the recipient is not a valid request target for the given pa

[jira] [Created] (KAFKA-10237) Properly handle in-memory stores OOM

2020-07-06 Thread Boyang Chen (Jira)
Boyang Chen created KAFKA-10237: --- Summary: Properly handle in-memory stores OOM Key: KAFKA-10237 URL: https://issues.apache.org/jira/browse/KAFKA-10237 Project: Kafka Issue Type: Improvement

[jira] [Resolved] (KAFKA-10173) BufferUnderflowException during Kafka Streams Upgrade

2020-07-06 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-10173. -- Fix Version/s: (was: 2.5.1) (was: 2.4.2) Resolution: Fixed >

[jira] [Commented] (KAFKA-10173) BufferUnderflowException during Kafka Streams Upgrade

2020-07-06 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152259#comment-17152259 ] John Roesler commented on KAFKA-10173: -- Hey all, I'm resolving this ticket to unblo

[jira] [Updated] (KAFKA-10173) BufferUnderflowException during Kafka Streams Upgrade

2020-07-06 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-10173: - Fix Version/s: 2.5.1 2.4.2 > BufferUnderflowException during Kafka Streams Up

[GitHub] [kafka] ableegoldman commented on a change in pull request #8962: KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory

2020-07-06 Thread GitBox
ableegoldman commented on a change in pull request #8962: URL: https://github.com/apache/kafka/pull/8962#discussion_r450429655 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -270,8 +270,11 @@ public void handleAssignment

[GitHub] [kafka] junrao commented on a change in pull request #8981: KAFKA-10235 Fix flaky transactions_test.py

2020-07-06 Thread GitBox
junrao commented on a change in pull request #8981: URL: https://github.com/apache/kafka/pull/8981#discussion_r450430604 ## File path: tests/kafkatest/tests/core/transactions_test.py ## @@ -53,7 +53,11 @@ def __init__(self, test_context): self.zk = ZookeeperService(tes

[GitHub] [kafka] ableegoldman commented on a change in pull request #8962: KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory

2020-07-06 Thread GitBox
ableegoldman commented on a change in pull request #8962: URL: https://github.com/apache/kafka/pull/8962#discussion_r450429655 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -270,8 +270,11 @@ public void handleAssignment

[GitHub] [kafka] ableegoldman commented on a change in pull request #8962: KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory

2020-07-06 Thread GitBox
ableegoldman commented on a change in pull request #8962: URL: https://github.com/apache/kafka/pull/8962#discussion_r450429655 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -270,8 +270,11 @@ public void handleAssignment

[jira] [Created] (KAFKA-10238) kafka-console-producer null value

2020-07-06 Thread Ryan (Jira)
Ryan created KAFKA-10238: Summary: kafka-console-producer null value Key: KAFKA-10238 URL: https://issues.apache.org/jira/browse/KAFKA-10238 Project: Kafka Issue Type: Improvement Component

[GitHub] [kafka] cadonna opened a new pull request #8987: KAFKA-10221: Backport fix for KAFKA-9603 to 2.5

2020-07-06 Thread GitBox
cadonna opened a new pull request #8987: URL: https://github.com/apache/kafka/pull/8987 KAFKA-9603 reports that the number of open files keeps increasing in RocksDB. The reason is that bulk loading is turned on but never turned off in segmented state stores for standby tasks. Th

[GitHub] [kafka] cadonna commented on pull request #8987: KAFKA-10221: Backport fix for KAFKA-9603 to 2.5

2020-07-06 Thread GitBox
cadonna commented on pull request #8987: URL: https://github.com/apache/kafka/pull/8987#issuecomment-654433640 Call for review: @vvcephei @ableegoldman @guozhangwang @mjsax @abbccdda This is an automated message from the Apa

[jira] [Commented] (KAFKA-4169) Calculation of message size is too conservative for compressed messages

2020-07-06 Thread Jorge Esteban Quilcate Otoya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152281#comment-17152281 ] Jorge Esteban Quilcate Otoya commented on KAFKA-4169: - I'm wondering

[jira] [Created] (KAFKA-10239) The groupInstanceId field in DescribeGroup response should be ignorable

2020-07-06 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-10239: --- Summary: The groupInstanceId field in DescribeGroup response should be ignorable Key: KAFKA-10239 URL: https://issues.apache.org/jira/browse/KAFKA-10239 Project

[jira] [Updated] (KAFKA-10239) The groupInstanceId field in DescribeGroup response should be ignorable

2020-07-06 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-10239: Fix Version/s: 2.5.1 2.4.2 > The groupInstanceId field in DescribeGroup

[jira] [Updated] (KAFKA-10239) The groupInstanceId field in DescribeGroup response should be ignorable

2020-07-06 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-10239: Affects Version/s: 2.5.0 2.4.1 > The groupInstanceId field in Descr

[GitHub] [kafka] ijuma commented on pull request #8742: KAFKA-10057 optimize class ConfigCommand method alterConfig parameters

2020-07-06 Thread GitBox
ijuma commented on pull request #8742: URL: https://github.com/apache/kafka/pull/8742#issuecomment-654436295 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [kafka] ijuma commented on pull request #8579: KAFKA-9930: Prevent ReplicaFetcherThread from throwing UnknownTopicOrPartitionException upon topic creation and deletion.

2020-07-06 Thread GitBox
ijuma commented on pull request #8579: URL: https://github.com/apache/kafka/pull/8579#issuecomment-654436916 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log o

[jira] [Assigned] (KAFKA-10239) The groupInstanceId field in DescribeGroup response should be ignorable

2020-07-06 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen reassigned KAFKA-10239: --- Assignee: Boyang Chen > The groupInstanceId field in DescribeGroup response should be ignor

[jira] [Created] (KAFKA-10240) Sink tasks should not throw WakeupException on shutdown

2020-07-06 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-10240: - Summary: Sink tasks should not throw WakeupException on shutdown Key: KAFKA-10240 URL: https://issues.apache.org/jira/browse/KAFKA-10240 Project: Kafka Iss

[GitHub] [kafka] guozhangwang commented on a change in pull request #8962: KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory

2020-07-06 Thread GitBox
guozhangwang commented on a change in pull request #8962: URL: https://github.com/apache/kafka/pull/8962#discussion_r450471701 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -270,8 +270,11 @@ public void handleAssignment

[GitHub] [kafka] guozhangwang commented on a change in pull request #8962: KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory

2020-07-06 Thread GitBox
guozhangwang commented on a change in pull request #8962: URL: https://github.com/apache/kafka/pull/8962#discussion_r450471701 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -270,8 +270,11 @@ public void handleAssignment

[GitHub] [kafka] guozhangwang opened a new pull request #8988: KAFKA-10199: Separate restore threads [WIP]

2020-07-06 Thread GitBox
guozhangwang opened a new pull request #8988: URL: https://github.com/apache/kafka/pull/8988 Main Ideas: 1) keep the StoreChangelogReader to be thread-safe, and restrict the shared access only on a few methods. 2) having N restore threads with N stream threads, with a 1-1 mapping

[GitHub] [kafka] guozhangwang commented on pull request #8988: KAFKA-10199: Separate restore threads [WIP]

2020-07-06 Thread GitBox
guozhangwang commented on pull request #8988: URL: https://github.com/apache/kafka/pull/8988#issuecomment-654459797 @ableegoldman @cadonna for taking a look at the synchronization mechanism. This is an automated message from

[GitHub] [kafka] guozhangwang merged pull request #8963: KAFKA-10017: fix flaky EosBetaUpgradeIntegrationTest

2020-07-06 Thread GitBox
guozhangwang merged pull request #8963: URL: https://github.com/apache/kafka/pull/8963 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (KAFKA-10017) Flaky Test EosBetaUpgradeIntegrationTest.shouldUpgradeFromEosAlphaToEosBeta

2020-07-06 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152329#comment-17152329 ] Guozhang Wang commented on KAFKA-10017: --- Hi [~rhauch] I've just pushed a PR from

[GitHub] [kafka] abbccdda opened a new pull request #8989: KAFKA-10239: Make GroupInstanceId ignorable in DescribeGroups

2020-07-06 Thread GitBox
abbccdda opened a new pull request #8989: URL: https://github.com/apache/kafka/pull/8989 This is a bug fix for older admin clients using static membership and call DescribeGroups. By making `groupInstanceId` ignorable, it would not crash upon handling the response. Added test covera

[GitHub] [kafka] guozhangwang commented on pull request #8963: KAFKA-10017: fix flaky EosBetaUpgradeIntegrationTest

2020-07-06 Thread GitBox
guozhangwang commented on pull request #8963: URL: https://github.com/apache/kafka/pull/8963#issuecomment-654472201 Merged to 2.6 as well and re-enabled the integration test. This is an automated message from the Apache Git S

[GitHub] [kafka] hachikuji commented on pull request #8989: KAFKA-10239: Make GroupInstanceId ignorable in DescribeGroups

2020-07-06 Thread GitBox
hachikuji commented on pull request #8989: URL: https://github.com/apache/kafka/pull/8989#issuecomment-654473290 @abbccdda Thanks for the quick fix. I am wondering if we have a similar problem with authorized operations: ``` { "name": "AuthorizedOperations", "type": "int32", "ve

[GitHub] [kafka] hachikuji commented on pull request #8989: KAFKA-10239: Make GroupInstanceId ignorable in DescribeGroups

2020-07-06 Thread GitBox
hachikuji commented on pull request #8989: URL: https://github.com/apache/kafka/pull/8989#issuecomment-654473843 Oh, sorry, I missed the version check. So above looks fine. This is an automated message from the Apache Git Ser

[GitHub] [kafka] guozhangwang commented on pull request #8971: MINOR: prune the metadata upgrade test matrix

2020-07-06 Thread GitBox
guozhangwang commented on pull request #8971: URL: https://github.com/apache/kafka/pull/8971#issuecomment-654474451 LGTM! This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [kafka] guozhangwang commented on a change in pull request #8971: MINOR: prune the metadata upgrade test matrix

2020-07-06 Thread GitBox
guozhangwang commented on a change in pull request #8971: URL: https://github.com/apache/kafka/pull/8971#discussion_r450488992 ## File path: tests/kafkatest/tests/streams/streams_upgrade_test.py ## @@ -189,9 +185,8 @@ def test_upgrade_downgrade_brokers(self, from_version, to_v

[GitHub] [kafka] guozhangwang commented on a change in pull request #8971: MINOR: prune the metadata upgrade test matrix

2020-07-06 Thread GitBox
guozhangwang commented on a change in pull request #8971: URL: https://github.com/apache/kafka/pull/8971#discussion_r450488992 ## File path: tests/kafkatest/tests/streams/streams_upgrade_test.py ## @@ -189,9 +185,8 @@ def test_upgrade_downgrade_brokers(self, from_version, to_v

[GitHub] [kafka] hachikuji commented on a change in pull request #8989: KAFKA-10239: Make GroupInstanceId ignorable in DescribeGroups

2020-07-06 Thread GitBox
hachikuji commented on a change in pull request #8989: URL: https://github.com/apache/kafka/pull/8989#discussion_r450490649 ## File path: clients/src/test/java/org/apache/kafka/common/message/MessageTest.java ## @@ -218,6 +220,59 @@ public void testOffsetCommitDefaultGroupInst

[GitHub] [kafka] ijuma commented on pull request #8983: KAFKA-8398: Prevent NPE in `forceUnmap`

2020-07-06 Thread GitBox
ijuma commented on pull request #8983: URL: https://github.com/apache/kafka/pull/8983#issuecomment-654483830 Merged to trunk and cherry-picked to 2.6 since it's extremely low risk (checked with @rhauch that this was OK). Thi

[GitHub] [kafka] ijuma commented on pull request #8985: MINOR; KafkaAdminClient#alterReplicaLogDirs should not fail all the futures when only one call fails

2020-07-06 Thread GitBox
ijuma commented on pull request #8985: URL: https://github.com/apache/kafka/pull/8985#issuecomment-654484579 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [kafka] hachikuji commented on a change in pull request #8864: KAFKA-9274: Mark `retries` config as deprecated and add new `task.timeout.ms` config

2020-07-06 Thread GitBox
hachikuji commented on a change in pull request #8864: URL: https://github.com/apache/kafka/pull/8864#discussion_r450493796 ## File path: clients/src/test/java/org/apache/kafka/clients/admin/KafkaAdminClientTest.java ## @@ -1447,6 +1451,7 @@ public void testListConsumerGroups

[GitHub] [kafka] ijuma commented on pull request #8742: KAFKA-10057 optimize class ConfigCommand method alterConfig parameters

2020-07-06 Thread GitBox
ijuma commented on pull request #8742: URL: https://github.com/apache/kafka/pull/8742#issuecomment-654486027 @sasukerui Can you please change the PR to target `master`? This is an automated message from the Apache Git Service

[GitHub] [kafka] cmccabe commented on a change in pull request #8863: KAFKA-8528: Expose Trogdor-specific JMX metrics for Tasks and Agents

2020-07-06 Thread GitBox
cmccabe commented on a change in pull request #8863: URL: https://github.com/apache/kafka/pull/8863#discussion_r450503568 ## File path: tools/src/main/java/org/apache/kafka/trogdor/agent/Agent.java ## @@ -92,6 +93,8 @@ private final Time time; +final TrogdorMetrics

[GitHub] [kafka] cmccabe commented on a change in pull request #8863: KAFKA-8528: Expose Trogdor-specific JMX metrics for Tasks and Agents

2020-07-06 Thread GitBox
cmccabe commented on a change in pull request #8863: URL: https://github.com/apache/kafka/pull/8863#discussion_r450504170 ## File path: tools/src/main/java/org/apache/kafka/trogdor/coordinator/TaskManager.java ## @@ -127,7 +127,12 @@ */ private long nextWorkerId;

[GitHub] [kafka] cmccabe commented on a change in pull request #8863: KAFKA-8528: Expose Trogdor-specific JMX metrics for Tasks and Agents

2020-07-06 Thread GitBox
cmccabe commented on a change in pull request #8863: URL: https://github.com/apache/kafka/pull/8863#discussion_r450504757 ## File path: tools/src/main/java/org/apache/kafka/trogdor/coordinator/TaskManager.java ## @@ -395,11 +401,13 @@ public Void call() throws Exception {

[jira] [Updated] (KAFKA-10220) NPE when describing resources

2020-07-06 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-10220: Fix Version/s: 2.7.0 > NPE when describing resources > - > >

[GitHub] [kafka] vvcephei commented on pull request #8962: KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory

2020-07-06 Thread GitBox
vvcephei commented on pull request #8962: URL: https://github.com/apache/kafka/pull/8962#issuecomment-654489992 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] vvcephei commented on pull request #8962: KAFKA-10166: checkpoint recycled standbys and ignore empty rocksdb base directory

2020-07-06 Thread GitBox
vvcephei commented on pull request #8962: URL: https://github.com/apache/kafka/pull/8962#issuecomment-654490080 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

  1   2   >