[GitHub] [kafka] dajac commented on a change in pull request #10438: KAFKA-12579: Remove various deprecated clients classes/methods for 3.0

2021-04-05 Thread GitBox
dajac commented on a change in pull request #10438: URL: https://github.com/apache/kafka/pull/10438#discussion_r607567743 ## File path: docs/upgrade.html ## @@ -27,18 +27,30 @@ Notable changes in 3 or updating the application not to use internal classes. The Strea

[jira] [Commented] (KAFKA-12492) Formatting of example RocksDBConfigSetter is messed up

2021-04-05 Thread Ben Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315284#comment-17315284 ] Ben Chen commented on KAFKA-12492: -- Do we need to run [local tests|https://github.com/

[GitHub] [kafka] satishd commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-05 Thread GitBox
satishd commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r607562925 ## File path: remote-storage/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogMetadataCache.java ## @@ -0,0 +1,331 @@ +/* + * Licensed t

[GitHub] [kafka] satishd commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-05 Thread GitBox
satishd commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r607561788 ## File path: clients/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteStorageException.java ## @@ -28,9 +27,14 @@ public RemoteStorageExce

[GitHub] [kafka] satishd commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-05 Thread GitBox
satishd commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r607559905 ## File path: remote-storage/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogMetadataCache.java ## @@ -0,0 +1,331 @@ +/* + * Licensed t

[GitHub] [kafka] satishd commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-05 Thread GitBox
satishd commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r607556778 ## File path: clients/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogSegmentState.java ## @@ -21,14 +21,16 @@ import java.util.Arrays

[GitHub] [kafka] cc13ny opened a new pull request #10486: KAFKA-12492: Fix the formatting of example RocksDBConfigSetter

2021-04-05 Thread GitBox
cc13ny opened a new pull request #10486: URL: https://github.com/apache/kafka/pull/10486 Fix the formatting of example RocksDBConfigSetter due to the un-arranged spaces within tag. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [kafka] showuon commented on pull request #10338: KAFKA-10251: wait for consumer rebalance completed before consuming records

2021-04-05 Thread GitBox
showuon commented on pull request #10338: URL: https://github.com/apache/kafka/pull/10338#issuecomment-813859513 @apurvam @hachikuji , call for review. Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [kafka] satishd commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-05 Thread GitBox
satishd commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r607551710 ## File path: remote-storage/src/test/java/org/apache/kafka/server/log/remote/storage/InmemoryRemoteLogMetadataManager.java ## @@ -0,0 +1,157 @@ +/* + *

[jira] [Commented] (KAFKA-12598) Remove deprecated --zookeeper in ConfigCommand

2021-04-05 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315268#comment-17315268 ] Luke Chen commented on KAFKA-12598: --- [~rajinisiva...@gmail.com]  [~rsivaram] , could y

[GitHub] [kafka] satishd commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-05 Thread GitBox
satishd commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r607546803 ## File path: remote-storage/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogMetadataCache.java ## @@ -0,0 +1,331 @@ +/* + * Licensed t

[GitHub] [kafka] ijuma merged pull request #10479: MINOR: Jenkinsfile's `post` needs `agent` to be set

2021-04-05 Thread GitBox
ijuma merged pull request #10479: URL: https://github.com/apache/kafka/pull/10479 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please c

[GitHub] [kafka] highluck commented on pull request #10302: KAFKA-7785: move internal DefaultPartitionGrouper

2021-04-05 Thread GitBox
highluck commented on pull request #10302: URL: https://github.com/apache/kafka/pull/10302#issuecomment-813824311 @guozhangwang May I ask for a merge? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [kafka] highluck commented on pull request #9861: MINOR: Modify unnecessary access specifiers

2021-04-05 Thread GitBox
highluck commented on pull request #9861: URL: https://github.com/apache/kafka/pull/9861#issuecomment-813823981 @chia7712 May I ask for a review of this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [kafka] highluck commented on a change in pull request #9851: KAFKA-10769 Remove JoinGroupRequest#containsValidPattern as it is dup…

2021-04-05 Thread GitBox
highluck commented on a change in pull request #9851: URL: https://github.com/apache/kafka/pull/9851#discussion_r607505049 ## File path: clients/src/test/java/org/apache/kafka/common/requests/JoinGroupRequestTest.java ## @@ -58,17 +57,7 @@ public void shouldThrowOnInvalidGroup

[GitHub] [kafka] ijuma opened a new pull request #10485: MINOR: Enable scala/java joint compilation consistently for `core` module

2021-04-05 Thread GitBox
ijuma opened a new pull request #10485: URL: https://github.com/apache/kafka/pull/10485 We were doing it only for test files previously. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status

[GitHub] [kafka] chia7712 commented on a change in pull request #9851: KAFKA-10769 Remove JoinGroupRequest#containsValidPattern as it is dup…

2021-04-05 Thread GitBox
chia7712 commented on a change in pull request #9851: URL: https://github.com/apache/kafka/pull/9851#discussion_r607490193 ## File path: clients/src/test/java/org/apache/kafka/common/requests/JoinGroupRequestTest.java ## @@ -58,17 +57,7 @@ public void shouldThrowOnInvalidGroup

[GitHub] [kafka] chia7712 commented on a change in pull request #10446: MINOR: [ConfigEntry.class] add 'type' to 'toString' and 'hashCode'

2021-04-05 Thread GitBox
chia7712 commented on a change in pull request #10446: URL: https://github.com/apache/kafka/pull/10446#discussion_r607486103 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/ConfigEntry.java ## @@ -167,6 +169,8 @@ public int hashCode() { result = prim

[jira] [Resolved] (KAFKA-8924) Default grace period (-1) of TimeWindows causes suppress to emit events after 24h

2021-04-05 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-8924. --- Resolution: Duplicate > Default grace period (-1) of TimeWindows causes suppr

[jira] [Commented] (KAFKA-8924) Default grace period (-1) of TimeWindows causes suppress to emit events after 24h

2021-04-05 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315217#comment-17315217 ] A. Sophie Blee-Goldman commented on KAFKA-8924: --- Can we close this as a dup

[jira] [Assigned] (KAFKA-8613) Set default grace period to 0

2021-04-05 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reassigned KAFKA-8613: - Assignee: A. Sophie Blee-Goldman > Set default grace period to 0 > -

[jira] [Resolved] (KAFKA-6603) Kafka streams off heap memory usage does not match expected values from configuration

2021-04-05 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-6603. --- Resolution: Fixed > Kafka streams off heap memory usage does not match expect

[jira] [Resolved] (KAFKA-8165) Streams task causes Out Of Memory after connection issues and store restoration

2021-04-05 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-8165. --- Resolution: Fixed > Streams task causes Out Of Memory after connection issues

[jira] [Commented] (KAFKA-12419) Remove Deprecated APIs of Kafka Streams in 3.0

2021-04-05 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315209#comment-17315209 ] A. Sophie Blee-Goldman commented on KAFKA-12419: Thanks -- wanted to mak

[GitHub] [kafka] ableegoldman opened a new pull request #10484: MINOR: un-deprecate StreamsConfig overloads to support dependency injection

2021-04-05 Thread GitBox
ableegoldman opened a new pull request #10484: URL: https://github.com/apache/kafka/pull/10484 In [#5344](https://github.com/apache/kafka/pull/5344#issuecomment-413350338) it came to our attention that the StreamsConfig overloads of the KafkaStreams constructors are actually quite useful f

[GitHub] [kafka] hachikuji commented on pull request #10483: KAFKA-12586; Add `DescribeTransactions` Admin API

2021-04-05 Thread GitBox
hachikuji commented on pull request #10483: URL: https://github.com/apache/kafka/pull/10483#issuecomment-813766742 @chia7712 @dajac No rush, but when you have time, this is a continuation of the previous work which added `AdminApiDriver`. This patch contains `CoordinatorStrategy`, which is

[GitHub] [kafka] hachikuji opened a new pull request #10483: KAFKA-12586; Add `DescribeTransactions` Admin API

2021-04-05 Thread GitBox
hachikuji opened a new pull request #10483: URL: https://github.com/apache/kafka/pull/10483 This patch contains the `Admin` implementation of the `DescribeTransactions` APIs described in KIP-664: KIP-664: https://cwiki.apache.org/confluence/display/KAFKA/KIP-664%3A+Provide+tooling+to+detec

[GitHub] [kafka] dengziming commented on a change in pull request #10393: KAFKA-12539: Refactor KafkaRaftCllient handleVoteRequest to reduce cyclomatic complexity

2021-04-05 Thread GitBox
dengziming commented on a change in pull request #10393: URL: https://github.com/apache/kafka/pull/10393#discussion_r607430517 ## File path: raft/src/main/java/org/apache/kafka/raft/CandidateState.java ## @@ -235,6 +240,15 @@ public int epoch() { return highWatermark;

[GitHub] [kafka] ableegoldman commented on a change in pull request #10482: KAFKA-12499: add transaction timeout verification

2021-04-05 Thread GitBox
ableegoldman commented on a change in pull request #10482: URL: https://github.com/apache/kafka/pull/10482#discussion_r607423468 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsProducer.java ## @@ -106,7 +107,7 @@ public StreamsProducer(

[GitHub] [kafka] lamberken commented on pull request #10469: KAFKA-12611: Fix using random payload in ProducerPerformance incorrectly

2021-04-05 Thread GitBox
lamberken commented on pull request #10469: URL: https://github.com/apache/kafka/pull/10469#issuecomment-813750487 hi @mageshn @C0urante @gharris1727 , please review when you get a chance. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[jira] [Commented] (KAFKA-12492) Formatting of example RocksDBConfigSetter is messed up

2021-04-05 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315186#comment-17315186 ] A. Sophie Blee-Goldman commented on KAFKA-12492: Yeah, you need to check

[jira] [Commented] (KAFKA-12492) Formatting of example RocksDBConfigSetter is messed up

2021-04-05 Thread Ben Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315185#comment-17315185 ] Ben Chen commented on KAFKA-12492: -- Got it. Thanks. I guess I missed something mentione

[GitHub] [kafka] mjsax commented on a change in pull request #10482: KAFKA-12499: add transaction timeout verification

2021-04-05 Thread GitBox
mjsax commented on a change in pull request #10482: URL: https://github.com/apache/kafka/pull/10482#discussion_r607414956 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsProducer.java ## @@ -106,7 +107,7 @@ public StreamsProducer(final S

[jira] [Commented] (KAFKA-12492) Formatting of example RocksDBConfigSetter is messed up

2021-04-05 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315183#comment-17315183 ] A. Sophie Blee-Goldman commented on KAFKA-12492: Thanks, I'll take a loo

[GitHub] [kafka] abbccdda closed pull request #9560: KAFKA-10345: Add ZK-notification based update for trust/key store paths

2021-04-05 Thread GitBox
abbccdda closed pull request #9560: URL: https://github.com/apache/kafka/pull/9560 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [kafka] abbccdda opened a new pull request #10482: KAFKA-12499: add transaction timeout verification

2021-04-05 Thread GitBox
abbccdda opened a new pull request #10482: URL: https://github.com/apache/kafka/pull/10482 This PR tries to add the check for transaction timeout for a comparison against commit interval of streams. If transaction timeout is smaller than commit interval, stream should crash and inform user

[jira] [Issue Comment Deleted] (KAFKA-12492) Formatting of example RocksDBConfigSetter is messed up

2021-04-05 Thread Ben Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Chen updated KAFKA-12492: - Comment: was deleted (was: https://github.com/apache/kafka-site/pull/345/) > Formatting of example Rock

[jira] [Commented] (KAFKA-12492) Formatting of example RocksDBConfigSetter is messed up

2021-04-05 Thread Ben Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315173#comment-17315173 ] Ben Chen commented on KAFKA-12492: -- https://github.com/apache/kafka-site/pull/345/ > F

[jira] [Commented] (KAFKA-12492) Formatting of example RocksDBConfigSetter is messed up

2021-04-05 Thread Ben Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315172#comment-17315172 ] Ben Chen commented on KAFKA-12492: -- [~ableegoldman] appreciate for the detailed explana

[jira] [Commented] (KAFKA-12574) Deprecate eos-alpha

2021-04-05 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315156#comment-17315156 ] Ismael Juma commented on KAFKA-12574: - For 4.0, we can make eos-v2 the default and m

[jira] [Commented] (KAFKA-12574) Deprecate eos-alpha

2021-04-05 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315153#comment-17315153 ] Guozhang Wang commented on KAFKA-12574: --- I'm fine with option 2) as well, to stay

[GitHub] [kafka] dielhennr commented on a change in pull request #10480: KAFKA-12265: Move the BatchAccumulator in KafkaRaftClient to LeaderState

2021-04-05 Thread GitBox
dielhennr commented on a change in pull request #10480: URL: https://github.com/apache/kafka/pull/10480#discussion_r607386832 ## File path: raft/src/main/java/org/apache/kafka/raft/LeaderState.java ## @@ -319,6 +328,10 @@ public String name() { } @Override -publ

[GitHub] [kafka] dielhennr commented on a change in pull request #10480: KAFKA-12265: Move the BatchAccumulator in KafkaRaftClient to LeaderState

2021-04-05 Thread GitBox
dielhennr commented on a change in pull request #10480: URL: https://github.com/apache/kafka/pull/10480#discussion_r607383662 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -2252,9 +2247,12 @@ public Long scheduleAtomicAppend(int epoch, List

[GitHub] [kafka] hachikuji commented on a change in pull request #10480: KAFKA-12265: Move the BatchAccumulator in KafkaRaftClient to LeaderState

2021-04-05 Thread GitBox
hachikuji commented on a change in pull request #10480: URL: https://github.com/apache/kafka/pull/10480#discussion_r607381875 ## File path: raft/src/test/java/org/apache/kafka/raft/LeaderStateTest.java ## @@ -36,30 +36,31 @@ import static org.junit.jupiter.api.Assertions.asser

[GitHub] [kafka] hachikuji commented on a change in pull request #10480: KAFKA-12265: Move the BatchAccumulator in KafkaRaftClient to LeaderState

2021-04-05 Thread GitBox
hachikuji commented on a change in pull request #10480: URL: https://github.com/apache/kafka/pull/10480#discussion_r607380961 ## File path: raft/src/test/java/org/apache/kafka/raft/LeaderStateTest.java ## @@ -36,30 +36,31 @@ import static org.junit.jupiter.api.Assertions.asser

[jira] [Resolved] (KAFKA-12294) Consider using the forwarding mechanism for metadata auto topic creation

2021-04-05 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-12294. - Fix Version/s: 3.0.0 Resolution: Fixed > Consider using the forwarding mechanism

[GitHub] [kafka] hachikuji merged pull request #10142: KAFKA-12294: forward auto topic request within envelope on behalf of clients

2021-04-05 Thread GitBox
hachikuji merged pull request #10142: URL: https://github.com/apache/kafka/pull/10142 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, plea

[jira] [Commented] (KAFKA-12574) Deprecate eos-alpha

2021-04-05 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315144#comment-17315144 ] Ismael Juma commented on KAFKA-12574: - {quote}Regarding the proposal, I thin users w

[jira] [Commented] (KAFKA-12574) Deprecate eos-alpha

2021-04-05 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315142#comment-17315142 ] A. Sophie Blee-Goldman commented on KAFKA-12574: > This is well document

[GitHub] [kafka] hachikuji commented on a change in pull request #9441: KAFKA-10614: Ensure group state (un)load is executed in the submitted order

2021-04-05 Thread GitBox
hachikuji commented on a change in pull request #9441: URL: https://github.com/apache/kafka/pull/9441#discussion_r607374397 ## File path: core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala ## @@ -905,19 +907,33 @@ class GroupCoordinator(val brokerId: Int, *

[jira] [Updated] (KAFKA-12607) Allow votes to be granted in resigned state

2021-04-05 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12607: -- Labels: newbie++ (was: ) > Allow votes to be granted in resigned state >

[jira] [Commented] (KAFKA-12492) Formatting of example RocksDBConfigSetter is messed up

2021-04-05 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315140#comment-17315140 ] A. Sophie Blee-Goldman commented on KAFKA-12492: The kafka-site repo is

[GitHub] [kafka] hachikuji commented on pull request #10481: KAFKA-12619; Raft leader should expose hw only after committing LeaderChange

2021-04-05 Thread GitBox
hachikuji commented on pull request #10481: URL: https://github.com/apache/kafka/pull/10481#issuecomment-813686724 @guozhangwang Thanks for the quick comment. I did consider that. I can't say I had a particularly strong reason to reject it, but ultimately I convinced myself that modifying

[GitHub] [kafka] ewencp commented on a change in pull request #10474: KAFKA-12602: Fix LICENSE file

2021-04-05 Thread GitBox
ewencp commented on a change in pull request #10474: URL: https://github.com/apache/kafka/pull/10474#discussion_r607362884 ## File path: LICENSE-binary ## @@ -0,0 +1,602 @@ + + Apache License + Version 2.0, January 2004

[GitHub] [kafka] guozhangwang commented on pull request #10481: KAFKA-12619; Raft leader should expose hw only after committing LeaderChange

2021-04-05 Thread GitBox
guozhangwang commented on pull request #10481: URL: https://github.com/apache/kafka/pull/10481#issuecomment-813683249 Nice catch! Regarding the fix, WDYT to just remember the `LeaderChange` record's offset and compare against it instead of the epoch start offset? I'm thinking if in the fut

[GitHub] [kafka] hachikuji opened a new pull request #10481: KAFKA-12619; Raft leader should expose hw only after committing LeaderChange

2021-04-05 Thread GitBox
hachikuji opened a new pull request #10481: URL: https://github.com/apache/kafka/pull/10481 KIP-595 describes an extra condition on commitment here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum#KIP595:ARaftProtocolfortheMetadataQuorum-

[GitHub] [kafka] jsancio commented on a change in pull request #10480: KAFKA-12265: Move the BatchAccumulator in KafkaRaftClient to LeaderState

2021-04-05 Thread GitBox
jsancio commented on a change in pull request #10480: URL: https://github.com/apache/kafka/pull/10480#discussion_r607353807 ## File path: raft/src/main/java/org/apache/kafka/raft/LeaderState.java ## @@ -48,13 +49,16 @@ private final Set grantingVoters = new HashSet<>();

[GitHub] [kafka] ableegoldman commented on a change in pull request #10474: KAFKA-12602: Fix LICENSE file

2021-04-05 Thread GitBox
ableegoldman commented on a change in pull request #10474: URL: https://github.com/apache/kafka/pull/10474#discussion_r607355497 ## File path: licenses/DWTFYWTPL ## @@ -0,0 +1,14 @@ +DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE Review comment: 😂 -- This

[GitHub] [kafka] rsomu commented on pull request #4040: KAFKA-6324: Change LogSegment.delete to deleteIfExists and harden log recovery

2021-04-05 Thread GitBox
rsomu commented on pull request #4040: URL: https://github.com/apache/kafka/pull/4040#issuecomment-813671433 Is this fix supposed to avoid the[ NFS silly renames](https://sbg.technology/2018/07/10/kafka-nfs/) issue? I am currently testing Kafka 2.7 on NFS filesystem and encountered the sa

[GitHub] [kafka] dielhennr opened a new pull request #10480: KAFKA-12265: Move the BatchAccumulator in KafkaRaftClient to LeaderState

2021-04-05 Thread GitBox
dielhennr opened a new pull request #10480: URL: https://github.com/apache/kafka/pull/10480 The KafkaRaftClient has a field for the BatchAccumulator that is only used and set when it is the leader. In other cases, leader specific information was stored in LeaderState. In a recent change Ep

[jira] [Updated] (KAFKA-6985) Error connection between cluster node

2021-04-05 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-6985: - Component/s: (was: KafkaConnect) core > Error connection between cluster nod

[jira] [Resolved] (KAFKA-8551) Comments for connectors() in Herder interface

2021-04-05 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-8551. -- Resolution: Won't Fix Marking as won't fix, since the details are insufficient to try to addres

[jira] [Resolved] (KAFKA-8664) non-JSON format messages when streaming data from Kafka to Mongo

2021-04-05 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-8664. -- Resolution: Won't Fix The reported problem is for a connector implementation that is not owned

[jira] [Resolved] (KAFKA-8867) Kafka Connect JDBC fails to create PostgreSQL table with default boolean value in schema

2021-04-05 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-8867. -- Resolution: Won't Fix The reported problem is for the Confluent JDBC source/sink connector, and

[GitHub] [kafka] ijuma opened a new pull request #10479: MINOR: Jenkinsfile's `post` needs `agent` to be set

2021-04-05 Thread GitBox
ijuma opened a new pull request #10479: URL: https://github.com/apache/kafka/pull/10479 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes)

[jira] [Resolved] (KAFKA-8961) Unable to create secure JDBC connection through Kafka Connect

2021-04-05 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-8961. -- Resolution: Won't Fix This is not a problem of the Connect framework, and is instead an issue w

[jira] [Updated] (KAFKA-9017) We see timeout in kafka in production cluster

2021-04-05 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-9017: - Component/s: (was: KafkaConnect) core > We see timeout in kafka in productio

[jira] [Commented] (KAFKA-10715) Support Kafka connect converter for AVRO

2021-04-05 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315110#comment-17315110 ] Randall Hauch commented on KAFKA-10715: --- There are multiple Converter implementati

[jira] [Resolved] (KAFKA-10715) Support Kafka connect converter for AVRO

2021-04-05 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch resolved KAFKA-10715. --- Resolution: Won't Do > Support Kafka connect converter for AVRO > --

[jira] [Updated] (KAFKA-9988) Connect incorrectly logs that task has failed when one takes too long to shutdown

2021-04-05 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-9988: - Labels: newbie (was: ) > Connect incorrectly logs that task has failed when one takes too long t

[GitHub] [kafka] cmccabe commented on a change in pull request #10455: MINOR: Support ExponentialBackoff without jitter

2021-04-05 Thread GitBox
cmccabe commented on a change in pull request #10455: URL: https://github.com/apache/kafka/pull/10455#discussion_r607330220 ## File path: clients/src/main/java/org/apache/kafka/common/utils/ExponentialBackoff.java ## @@ -47,7 +47,8 @@ public long backoff(long attempts) {

[jira] [Updated] (KAFKA-12619) Ensure LeaderChange message is committed before initializing high watermark

2021-04-05 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-12619: Description: KIP-595 describes an extra condition on commitment here: https://cwiki.apach

[jira] [Created] (KAFKA-12619) Ensure LeaderChange message is committed before initializing high watermark

2021-04-05 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-12619: --- Summary: Ensure LeaderChange message is committed before initializing high watermark Key: KAFKA-12619 URL: https://issues.apache.org/jira/browse/KAFKA-12619 Pro

[jira] [Updated] (KAFKA-7438) Replace EasyMock and PowerMock with Mockito

2021-04-05 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-7438: --- Summary: Replace EasyMock and PowerMock with Mockito (was: Replace EasyMock and PowerMock with Mocki

[jira] [Commented] (KAFKA-12618) Convert LogManager (and other EasyMocks) in ReplicaManagerTest to Mockito

2021-04-05 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315087#comment-17315087 ] Ismael Juma commented on KAFKA-12618: - Also see https://issues.apache.org/jira/brows

[jira] [Updated] (KAFKA-7438) Replace EasyMock and PowerMock with Mockito in the clients module

2021-04-05 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-7438: --- Summary: Replace EasyMock and PowerMock with Mockito in the clients module (was: Replace EasyMock an

[jira] [Created] (KAFKA-12618) Convert LogManager (and other EasyMocks) in ReplicaManagerTest to Mockito

2021-04-05 Thread Justine Olshan (Jira)
Justine Olshan created KAFKA-12618: -- Summary: Convert LogManager (and other EasyMocks) in ReplicaManagerTest to Mockito Key: KAFKA-12618 URL: https://issues.apache.org/jira/browse/KAFKA-12618 Project

[GitHub] [kafka] mumrah commented on a change in pull request #10455: MINOR: Support ExponentialBackoff without jitter

2021-04-05 Thread GitBox
mumrah commented on a change in pull request #10455: URL: https://github.com/apache/kafka/pull/10455#discussion_r607313601 ## File path: clients/src/main/java/org/apache/kafka/common/utils/ExponentialBackoff.java ## @@ -47,7 +47,8 @@ public long backoff(long attempts) {

[GitHub] [kafka] mumrah commented on a change in pull request #10254: KAFKA-12406 Integrate client quotas with raft broker

2021-04-05 Thread GitBox
mumrah commented on a change in pull request #10254: URL: https://github.com/apache/kafka/pull/10254#discussion_r607290168 ## File path: core/src/test/scala/integration/kafka/server/RaftClusterTest.scala ## @@ -212,4 +214,103 @@ class RaftClusterTest { cluster.close()

[GitHub] [kafka] mageshn commented on a change in pull request #10475: KAFKA-12610: Implement PluginClassLoader::getResource

2021-04-05 Thread GitBox
mageshn commented on a change in pull request #10475: URL: https://github.com/apache/kafka/pull/10475#discussion_r607285210 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/Plugins.java ## @@ -61,12 +61,24 @@ public Plugins(Map props) {

[GitHub] [kafka] mumrah commented on a change in pull request #10254: KAFKA-12406 Integrate client quotas with raft broker

2021-04-05 Thread GitBox
mumrah commented on a change in pull request #10254: URL: https://github.com/apache/kafka/pull/10254#discussion_r607281769 ## File path: core/src/main/scala/kafka/server/metadata/ClientQuotaMetadataManager.scala ## @@ -121,16 +121,16 @@ class ClientQuotaMetadataManager(private

[GitHub] [kafka] mumrah commented on a change in pull request #10254: KAFKA-12406 Integrate client quotas with raft broker

2021-04-05 Thread GitBox
mumrah commented on a change in pull request #10254: URL: https://github.com/apache/kafka/pull/10254#discussion_r607278548 ## File path: core/src/main/scala/kafka/server/metadata/ClientQuotaCache.scala ## @@ -122,6 +122,14 @@ class ClientQuotaCache { entityFilters.put(en

[GitHub] [kafka] mumrah commented on a change in pull request #10254: KAFKA-12406 Integrate client quotas with raft broker

2021-04-05 Thread GitBox
mumrah commented on a change in pull request #10254: URL: https://github.com/apache/kafka/pull/10254#discussion_r607274751 ## File path: core/src/test/java/kafka/test/junit/RaftClusterInvocationContext.java ## @@ -77,7 +77,7 @@ public String getDisplayName(int invocationIndex)

[GitHub] [kafka] jsancio commented on a change in pull request #10468: Kafka 12373:Improve KafkaRaftClient handling of graceful shutdown

2021-04-05 Thread GitBox
jsancio commented on a change in pull request #10468: URL: https://github.com/apache/kafka/pull/10468#discussion_r607271981 ## File path: raft/src/test/java/org/apache/kafka/raft/KafkaRaftClientTest.java ## @@ -1673,6 +1673,69 @@ public void testLeaderGracefulShutdownTimeout()

[GitHub] [kafka] cmccabe commented on a change in pull request #10254: KAFKA-12406 Integrate client quotas with raft broker

2021-04-05 Thread GitBox
cmccabe commented on a change in pull request #10254: URL: https://github.com/apache/kafka/pull/10254#discussion_r607270847 ## File path: metadata/src/main/java/org/apache/kafka/controller/ClientQuotaControlManager.java ## @@ -170,8 +170,10 @@ private void alterClientQuotaEnti

[GitHub] [kafka] hachikuji merged pull request #10445: KAFKA-12548; Propagate record error messages to application

2021-04-05 Thread GitBox
hachikuji merged pull request #10445: URL: https://github.com/apache/kafka/pull/10445 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, plea

[GitHub] [kafka] cmccabe commented on a change in pull request #10254: KAFKA-12406 Integrate client quotas with raft broker

2021-04-05 Thread GitBox
cmccabe commented on a change in pull request #10254: URL: https://github.com/apache/kafka/pull/10254#discussion_r607267726 ## File path: core/src/test/scala/integration/kafka/server/RaftClusterTest.scala ## @@ -212,4 +214,103 @@ class RaftClusterTest { cluster.close()

[GitHub] [kafka] cmccabe commented on a change in pull request #10254: KAFKA-12406 Integrate client quotas with raft broker

2021-04-05 Thread GitBox
cmccabe commented on a change in pull request #10254: URL: https://github.com/apache/kafka/pull/10254#discussion_r607264162 ## File path: core/src/test/java/kafka/test/junit/RaftClusterInvocationContext.java ## @@ -77,7 +77,7 @@ public String getDisplayName(int invocationIndex)

[GitHub] [kafka] cmccabe commented on a change in pull request #10254: KAFKA-12406 Integrate client quotas with raft broker

2021-04-05 Thread GitBox
cmccabe commented on a change in pull request #10254: URL: https://github.com/apache/kafka/pull/10254#discussion_r607264162 ## File path: core/src/test/java/kafka/test/junit/RaftClusterInvocationContext.java ## @@ -77,7 +77,7 @@ public String getDisplayName(int invocationIndex)

[GitHub] [kafka] cmccabe commented on a change in pull request #10254: KAFKA-12406 Integrate client quotas with raft broker

2021-04-05 Thread GitBox
cmccabe commented on a change in pull request #10254: URL: https://github.com/apache/kafka/pull/10254#discussion_r607263869 ## File path: core/src/main/scala/kafka/server/metadata/ClientQuotaMetadataManager.scala ## @@ -121,16 +121,16 @@ class ClientQuotaMetadataManager(privat

[jira] [Resolved] (KAFKA-12548) Invalid record error message is not getting sent to application

2021-04-05 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-12548. - Resolution: Fixed > Invalid record error message is not getting sent to application > --

[jira] [Updated] (KAFKA-12548) Invalid record error message is not getting sent to application

2021-04-05 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-12548: Fix Version/s: 3.0.0 > Invalid record error message is not getting sent to application > -

[GitHub] [kafka] cmccabe commented on a change in pull request #10254: KAFKA-12406 Integrate client quotas with raft broker

2021-04-05 Thread GitBox
cmccabe commented on a change in pull request #10254: URL: https://github.com/apache/kafka/pull/10254#discussion_r607263189 ## File path: core/src/main/scala/kafka/server/metadata/ClientQuotaCache.scala ## @@ -122,6 +122,14 @@ class ClientQuotaCache { entityFilters.put(e

[GitHub] [kafka] junrao commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-05 Thread GitBox
junrao commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r606343187 ## File path: remote-storage/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogMetadataCache.java ## @@ -0,0 +1,331 @@ +/* + * Licensed to

[GitHub] [kafka] cmccabe commented on pull request #10366: KAFKA-12467: Add controller-side snapshot generation

2021-04-05 Thread GitBox
cmccabe commented on pull request #10366: URL: https://github.com/apache/kafka/pull/10366#issuecomment-813567129 Fix spotbugs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [kafka] hachikuji commented on a change in pull request #10085: KAFKA-12154: Snapshot Loading API

2021-04-05 Thread GitBox
hachikuji commented on a change in pull request #10085: URL: https://github.com/apache/kafka/pull/10085#discussion_r607232953 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -355,24 +373,29 @@ private void fireHandleResign(int epoch) { }

[GitHub] [kafka] ijuma commented on a change in pull request #10393: KAFKA-12539: Refactor KafkaRaftCllient handleVoteRequest to reduce cyclomatic complexity

2021-04-05 Thread GitBox
ijuma commented on a change in pull request #10393: URL: https://github.com/apache/kafka/pull/10393#discussion_r607233215 ## File path: raft/src/main/java/org/apache/kafka/raft/CandidateState.java ## @@ -235,6 +240,15 @@ public int epoch() { return highWatermark;

[GitHub] [kafka] ncliang commented on a change in pull request #10475: KAFKA-12610: Implement PluginClassLoader::getResource

2021-04-05 Thread GitBox
ncliang commented on a change in pull request #10475: URL: https://github.com/apache/kafka/pull/10475#discussion_r607232615 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/runtime/isolation/PluginsTest.java ## @@ -342,6 +343,47 @@ public void newPluginsSh

[jira] [Commented] (KAFKA-12602) The LICENSE and NOTICE files don't list everything they should

2021-04-05 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17315000#comment-17315000 ] John Roesler commented on KAFKA-12602: -- Thank you, [~jmclean] ,   There weren't t

[GitHub] [kafka] hachikuji commented on a change in pull request #10142: KAFKA-12294: forward auto topic request within envelope on behalf of clients

2021-04-05 Thread GitBox
hachikuji commented on a change in pull request #10142: URL: https://github.com/apache/kafka/pull/10142#discussion_r607224516 ## File path: core/src/main/scala/kafka/server/AutoTopicCreationManager.scala ## @@ -156,17 +169,44 @@ class DefaultAutoTopicCreationManager( .

  1   2   >