[GitHub] [kafka] showuon commented on pull request #10665: KAFKA-9009: increase replica.lag.time.max.ms to make the test reliable

2021-05-11 Thread GitBox
showuon commented on pull request #10665: URL: https://github.com/apache/kafka/pull/10665#issuecomment-837956607 Failed tests are un-related. Thanks. ``` Build / JDK 15 and Scala 2.13 / org.apache.kafka.common.record.MemoryRecordsBuilderTest.[6] magic=2, bufferOffset=0, compressi

[GitHub] [kafka] vitojeng opened a new pull request #10668: MINOR: Apply try-with-resource to KafkaStreamsTest

2021-05-11 Thread GitBox
vitojeng opened a new pull request #10668: URL: https://github.com/apache/kafka/pull/10668 In the PR #9821, @mjsax 's [comment](https://github.com/apache/kafka/pull/9821#discussion_r556200365) > We should use a try-with-resources clause to make sure close() is called. > > Seems

[GitHub] [kafka] jlprat opened a new pull request #10669: KAFKA-12769: Backport of KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while bui…

2021-05-11 Thread GitBox
jlprat opened a new pull request #10669: URL: https://github.com/apache/kafka/pull/10669 …lding SslTransportLayer This is a cherry picked commit of https://github.com/apache/kafka/pull/10059 to branch 2.8 Original commit message: This patch moves the `peerHost` helper defin

[GitHub] [kafka] DuongPTIT opened a new pull request #10670: KAFKA-10273 Connect Converters should produce actionable error messages

2021-05-11 Thread GitBox
DuongPTIT opened a new pull request #10670: URL: https://github.com/apache/kafka/pull/10670 @rhauch Hi, please review this PR for me. Many thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (KAFKA-12635) Mirrormaker 2 offset sync is incorrect if the target partition is empty

2021-05-11 Thread Angelos Kaltsikis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342374#comment-17342374 ] Angelos Kaltsikis commented on KAFKA-12635: --- Hello [~yangguo1220], Thanks for

[GitHub] [kafka] chia7712 opened a new pull request #10671: MINOR: exclude all `src/generated` and `src/generated-test`

2021-05-11 Thread GitBox
chia7712 opened a new pull request #10671: URL: https://github.com/apache/kafka/pull/10671 for https://github.com/apache/kafka/pull/10637#issuecomment-837691327 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test cover

[GitHub] [kafka] chia7712 commented on pull request #10637: MINOR: remove storage/src/generated from tracked files

2021-05-11 Thread GitBox
chia7712 commented on pull request #10637: URL: https://github.com/apache/kafka/pull/10637#issuecomment-837994346 > Is it possible to set up the gitignore to automatically ignore anything under a generated/ subdirectory? please take a look at #10671 -- This is an automated message

[GitHub] [kafka] DuongPTIT closed pull request #10670: KAFKA-10273 Connect Converters should produce actionable error messages

2021-05-11 Thread GitBox
DuongPTIT closed pull request #10670: URL: https://github.com/apache/kafka/pull/10670 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, plea

[GitHub] [kafka] jlprat opened a new pull request #10672: KAFKA-12769: Backport of KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while bui…

2021-05-11 Thread GitBox
jlprat opened a new pull request #10672: URL: https://github.com/apache/kafka/pull/10672 …lding SslTransportLayer This is a cherry picked commit of #10059 to branch 2.7 This patch moves the `peerHost` helper defined in `SslChannelBuilder` into `SslFactor`. `SaslChannelBuilder`

[GitHub] [kafka] jlprat commented on pull request #10669: KAFKA-12769: Backport too 2.8 of KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while bui…

2021-05-11 Thread GitBox
jlprat commented on pull request #10669: URL: https://github.com/apache/kafka/pull/10669#issuecomment-838046575 cc @vvcephei as mentioned on the mailing list, here there is a backport to 2.8 -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [kafka] jlprat commented on pull request #10672: KAFKA-12769: Backport to 2.7 of KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while bui…

2021-05-11 Thread GitBox
jlprat commented on pull request #10672: URL: https://github.com/apache/kafka/pull/10672#issuecomment-838046744 cc @vvcephei as mentioned on the mailing list, here there is a backport to 2.7 -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [kafka] cadonna commented on a change in pull request #10646: KAFKA-8897 Follow-up: Consolidate the global state stores

2021-05-11 Thread GitBox
cadonna commented on a change in pull request #10646: URL: https://github.com/apache/kafka/pull/10646#discussion_r629968874 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStateManagerImpl.java ## @@ -128,8 +129,7 @@ public void setGlobal

[GitHub] [kafka] cadonna commented on pull request #10651: MINOR: Kafka Streams code samples formating unification

2021-05-11 Thread GitBox
cadonna commented on pull request #10651: URL: https://github.com/apache/kafka/pull/10651#issuecomment-838118603 For the next time, I would recommend to open smaller PRs. Almost 2000 additions is a lot even for docs. In my experience smaller PRs tend to be merged faster. -- This is an a

[GitHub] [kafka] cadonna edited a comment on pull request #10651: MINOR: Kafka Streams code samples formating unification

2021-05-11 Thread GitBox
cadonna edited a comment on pull request #10651: URL: https://github.com/apache/kafka/pull/10651#issuecomment-838118603 Thank for the PR! For the next time, I would recommend to open smaller PRs. Almost 2000 additions is a lot even for docs. In my experience smaller PRs tend to be

[GitHub] [kafka] showuon commented on a change in pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-11 Thread GitBox
showuon commented on a change in pull request #10552: URL: https://github.com/apache/kafka/pull/10552#discussion_r629988069 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -40,6 +39,7 @@ private static fina

[GitHub] [kafka] cadonna commented on pull request #10651: MINOR: Kafka Streams code samples formating unification

2021-05-11 Thread GitBox
cadonna commented on pull request #10651: URL: https://github.com/apache/kafka/pull/10651#issuecomment-838119727 Do not worry about the test failures. First of all, your PR does not touch any code, so there is no reason a test would fail due to your PR. Additionally, the failing tests are

[GitHub] [kafka] jlprat commented on pull request #10651: MINOR: Kafka Streams code samples formating unification

2021-05-11 Thread GitBox
jlprat commented on pull request #10651: URL: https://github.com/apache/kafka/pull/10651#issuecomment-838120807 @cadonna I'll do it next time. I was doubting between providing a PR per file or a PR per folder. (ended up doing PR for the Streams folder). -- This is an automated message fr

[GitHub] [kafka] jlprat commented on pull request #10651: MINOR: Kafka Streams code samples formating unification

2021-05-11 Thread GitBox
jlprat commented on pull request #10651: URL: https://github.com/apache/kafka/pull/10651#issuecomment-838123810 Most of the changes are purely cosmetic: - using the right tags for for embedding a code snippet - escaping `<` and `>` characters to HTML encoded strings so they are proper

[GitHub] [kafka] showuon commented on a change in pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-11 Thread GitBox
showuon commented on a change in pull request #10552: URL: https://github.com/apache/kafka/pull/10552#discussion_r629990096 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -106,9 +104,9 @@ private boolean allSub

[GitHub] [kafka] showuon commented on a change in pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-11 Thread GitBox
showuon commented on a change in pull request #10552: URL: https://github.com/apache/kafka/pull/10552#discussion_r629990802 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -428,14 +372,18 @@ private boolean allS

[GitHub] [kafka] showuon commented on a change in pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-11 Thread GitBox
showuon commented on a change in pull request #10552: URL: https://github.com/apache/kafka/pull/10552#discussion_r629991853 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -469,73 +426,184 @@ private boolean all

[GitHub] [kafka] showuon commented on a change in pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-11 Thread GitBox
showuon commented on a change in pull request #10552: URL: https://github.com/apache/kafka/pull/10552#discussion_r629991853 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -469,73 +426,184 @@ private boolean all

[GitHub] [kafka] jlprat edited a comment on pull request #10651: MINOR: Kafka Streams code samples formating unification

2021-05-11 Thread GitBox
jlprat edited a comment on pull request #10651: URL: https://github.com/apache/kafka/pull/10651#issuecomment-838123810 Most of the changes are purely cosmetic: - using the right tags for for embedding a code snippet - escaping `<` and `>` characters to HTML encoded strings so they are

[GitHub] [kafka] showuon commented on a change in pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-11 Thread GitBox
showuon commented on a change in pull request #10552: URL: https://github.com/apache/kafka/pull/10552#discussion_r629994499 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -469,73 +426,184 @@ private boolean all

[GitHub] [kafka] showuon commented on pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-11 Thread GitBox
showuon commented on pull request #10552: URL: https://github.com/apache/kafka/pull/10552#issuecomment-838136488 @ableegoldman , I've done the code refinement and refactor. Basically is what we've discussed in constrained Assignor PR. Please take a look when available. Thanks. -- This

[GitHub] [kafka] showuon edited a comment on pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-11 Thread GitBox
showuon edited a comment on pull request #10552: URL: https://github.com/apache/kafka/pull/10552#issuecomment-838136488 @ableegoldman , I've done the code refinement and refactor. Basically is what we've discussed in constrained Assignor PR. Please take a look when available. cc @guozh

[GitHub] [kafka] jlprat commented on pull request #10669: KAFKA-12769: Backport too 2.8 of KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while bui…

2021-05-11 Thread GitBox
jlprat commented on pull request #10669: URL: https://github.com/apache/kafka/pull/10669#issuecomment-838186948 Test failure was: `Build / JDK 8 and Scala 2.12 / kafka.server.ListOffsetsRequestTest.testResponseIncludesLeaderEpoch()` This flaky test is already fixed (https://issues.ap

[GitHub] [kafka] DuongPTIT commented on pull request #10670: KAFKA-10273 Connect Converters should produce actionable error messages

2021-05-11 Thread GitBox
DuongPTIT commented on pull request #10670: URL: https://github.com/apache/kafka/pull/10670#issuecomment-838210303 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] jlprat commented on pull request #10672: KAFKA-12769: Backport to 2.7 of KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while bui…

2021-05-11 Thread GitBox
jlprat commented on pull request #10672: URL: https://github.com/apache/kafka/pull/10672#issuecomment-838221652 Test failure was: `Build / JDK 8 and Scala 2.12 / kafka.server.ListOffsetsRequestTest.testResponseIncludesLeaderEpoch()` This flaky test is already fixed (https://issues.ap

[GitHub] [kafka] dengziming commented on pull request #10667: KAFKA-12772: Move all transaction state transition rules into their states

2021-05-11 Thread GitBox
dengziming commented on pull request #10667: URL: https://github.com/apache/kafka/pull/10667#issuecomment-838294186 Hello @guozhangwang ,PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] showuon commented on pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-05-11 Thread GitBox
showuon commented on pull request #10552: URL: https://github.com/apache/kafka/pull/10552#issuecomment-838339697 I saw there are cooperative sticky tests failed. I'll update it, and add more tests into it tomorrow or later. Thanks. -- This is an automated message from the Apache Git Serv

[jira] [Commented] (KAFKA-9826) Log cleaning repeatedly picks same segment with no effect when first dirty offset is past start of active segment

2021-05-11 Thread zhangzhisheng (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342571#comment-17342571 ] zhangzhisheng commented on KAFKA-9826: -- we have same problems,kafka version 2.12_2.4

[GitHub] [kafka] vitojeng commented on pull request #10668: MINOR: Apply try-with-resource to KafkaStreamsTest

2021-05-11 Thread GitBox
vitojeng commented on pull request #10668: URL: https://github.com/apache/kafka/pull/10668#issuecomment-838538304 @mjsax , @ableegoldman Please take a look. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [kafka] chia7712 opened a new pull request #10673: MINOR: set replication.factor to 1 to make StreamsBrokerCompatibility…

2021-05-11 Thread GitBox
chia7712 opened a new pull request #10673: URL: https://github.com/apache/kafka/pull/10673 related to #10532 the default value of `replication.factor` was changed from `1` to `-1`. The old broker (< 2.4) does not support such configuration so this PR sets the `replication.factor` to

[GitHub] [kafka] chia7712 commented on a change in pull request #10668: MINOR: Apply try-with-resource to KafkaStreamsTest

2021-05-11 Thread GitBox
chia7712 commented on a change in pull request #10668: URL: https://github.com/apache/kafka/pull/10668#discussion_r630238771 ## File path: streams/src/test/java/org/apache/kafka/streams/KafkaStreamsTest.java ## @@ -553,141 +552,152 @@ public void testInitializesAndDestroysMetr

[GitHub] [kafka] omkreddy commented on pull request #10669: KAFKA-12769: Backport too 2.8 of KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while bui…

2021-05-11 Thread GitBox
omkreddy commented on pull request #10669: URL: https://github.com/apache/kafka/pull/10669#issuecomment-838656757 @jlprat Thanks for PR. I will directly push the original commit to 2.8 and 2.7 branches. We can close this PR. -- This is an automated message from the Apache Git Service. To

[GitHub] [kafka] omkreddy commented on pull request #10672: KAFKA-12769: Backport to 2.7 of KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while bui…

2021-05-11 Thread GitBox
omkreddy commented on pull request #10672: URL: https://github.com/apache/kafka/pull/10672#issuecomment-838657103 @jlprat Thanks for PR. I will directly push the original commit to 2.8 and 2.7 branches. We can close this PR. -- This is an automated message from the Apache Git Ser

[jira] [Updated] (KAFKA-8562) SASL_SSL still performs reverse DNS lookup despite KAFKA-5051

2021-05-11 Thread Manikumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar updated KAFKA-8562: - Fix Version/s: 2.8.1 2.7.2 > SASL_SSL still performs reverse DNS lookup despite KAFKA-

[GitHub] [kafka] jlprat closed pull request #10669: KAFKA-12769: Backport too 2.8 of KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while bui…

2021-05-11 Thread GitBox
jlprat closed pull request #10669: URL: https://github.com/apache/kafka/pull/10669 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [kafka] jlprat closed pull request #10672: KAFKA-12769: Backport to 2.7 of KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while bui…

2021-05-11 Thread GitBox
jlprat closed pull request #10672: URL: https://github.com/apache/kafka/pull/10672 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [kafka] jlprat commented on pull request #10672: KAFKA-12769: Backport to 2.7 of KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while bui…

2021-05-11 Thread GitBox
jlprat commented on pull request #10672: URL: https://github.com/apache/kafka/pull/10672#issuecomment-838708838 Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [kafka] jlprat commented on pull request #10669: KAFKA-12769: Backport too 2.8 of KAFKA-8562; SaslChannelBuilder - Avoid (reverse) DNS lookup while bui…

2021-05-11 Thread GitBox
jlprat commented on pull request #10669: URL: https://github.com/apache/kafka/pull/10669#issuecomment-838708152 Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [kafka] mumrah commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

2021-05-11 Thread GitBox
mumrah commented on a change in pull request #10638: URL: https://github.com/apache/kafka/pull/10638#discussion_r630302538 ## File path: build.gradle ## @@ -1345,6 +1349,62 @@ project(':raft') { } } +project(':server-common') { + archivesBaseName = "kafka-server-common"

[GitHub] [kafka] mumrah commented on a change in pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

2021-05-11 Thread GitBox
mumrah commented on a change in pull request #10638: URL: https://github.com/apache/kafka/pull/10638#discussion_r630302538 ## File path: build.gradle ## @@ -1345,6 +1349,62 @@ project(':raft') { } } +project(':server-common') { + archivesBaseName = "kafka-server-common"

[GitHub] [kafka] junrao merged pull request #10647: MINOR Removed copying storage libraries specifically as the task already copies them.

2021-05-11 Thread GitBox
junrao merged pull request #10647: URL: https://github.com/apache/kafka/pull/10647 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [kafka] ijuma commented on a change in pull request #10647: MINOR Removed copying storage libraries specifically as the task already copies them.

2021-05-11 Thread GitBox
ijuma commented on a change in pull request #10647: URL: https://github.com/apache/kafka/pull/10647#discussion_r630330236 ## File path: build.gradle ## @@ -1003,10 +1003,6 @@ project(':core') { from(project(':connect:mirror').configurations.runtimeClasspath) { into("libs/

[GitHub] [kafka] dejan2609 commented on pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-11 Thread GitBox
dejan2609 commented on pull request #10606: URL: https://github.com/apache/kafka/pull/10606#issuecomment-838776681 > Looks like there is a change in behavior in Gradle 7 related to resource files that's causing a bunch of tests to fail Indeed... on my local machine I received around

[GitHub] [kafka] jsancio commented on a change in pull request #10431: KAFKA-12543: Change RawSnapshotReader ownership model

2021-05-11 Thread GitBox
jsancio commented on a change in pull request #10431: URL: https://github.com/apache/kafka/pull/10431#discussion_r628546403 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -242,85 +246,116 @@ final class KafkaMetadataLog private ( } override de

[GitHub] [kafka] jsancio commented on a change in pull request #10431: KAFKA-12543: Change RawSnapshotReader ownership model

2021-05-11 Thread GitBox
jsancio commented on a change in pull request #10431: URL: https://github.com/apache/kafka/pull/10431#discussion_r628546403 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -242,85 +246,116 @@ final class KafkaMetadataLog private ( } override de

[GitHub] [kafka] jsancio commented on a change in pull request #10431: KAFKA-12543: Change RawSnapshotReader ownership model

2021-05-11 Thread GitBox
jsancio commented on a change in pull request #10431: URL: https://github.com/apache/kafka/pull/10431#discussion_r628550053 ## File path: raft/src/main/java/org/apache/kafka/snapshot/Snapshots.java ## @@ -104,18 +105,29 @@ public static Path createTempFile(Path logDir, OffsetA

[GitHub] [kafka] junrao merged pull request #10638: KAFKA-12758 Added `server-common` module to have server side common classes.

2021-05-11 Thread GitBox
junrao merged pull request #10638: URL: https://github.com/apache/kafka/pull/10638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[jira] [Resolved] (KAFKA-12758) Create a new `server-common` module and move ApiMessageAndVersion, RecordSerde, AbstractApiMessageSerde, and BytesApiMessageSerde to that module.

2021-05-11 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-12758. - Fix Version/s: 3.0.0 Resolution: Fixed merged the PR to trunk > Create a new `server-common` mod

[jira] [Created] (KAFKA-12773) Use UncheckedIOException when wrapping IOException

2021-05-11 Thread Jose Armando Garcia Sancio (Jira)
Jose Armando Garcia Sancio created KAFKA-12773: -- Summary: Use UncheckedIOException when wrapping IOException Key: KAFKA-12773 URL: https://issues.apache.org/jira/browse/KAFKA-12773 Project

[GitHub] [kafka] jsancio commented on a change in pull request #10431: KAFKA-12543: Change RawSnapshotReader ownership model

2021-05-11 Thread GitBox
jsancio commented on a change in pull request #10431: URL: https://github.com/apache/kafka/pull/10431#discussion_r630364576 ## File path: raft/src/main/java/org/apache/kafka/snapshot/FileRawSnapshotReader.java ## @@ -54,8 +54,12 @@ public Records records() { } @Ove

[jira] [Commented] (KAFKA-3539) KafkaProducer.send() may block even though it returns the Future

2021-05-11 Thread Moses Nakamura (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342739#comment-17342739 ] Moses Nakamura commented on KAFKA-3539: --- We've been wrestling with this problem for

[GitHub] [kafka] jsancio commented on a change in pull request #10431: KAFKA-12543: Change RawSnapshotReader ownership model

2021-05-11 Thread GitBox
jsancio commented on a change in pull request #10431: URL: https://github.com/apache/kafka/pull/10431#discussion_r630367487 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -161,19 +162,24 @@ final class KafkaMetadataLog private ( override def trun

[GitHub] [kafka] jsancio commented on a change in pull request #10431: KAFKA-12543: Change RawSnapshotReader ownership model

2021-05-11 Thread GitBox
jsancio commented on a change in pull request #10431: URL: https://github.com/apache/kafka/pull/10431#discussion_r630369703 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -242,85 +248,116 @@ final class KafkaMetadataLog private ( } override de

[GitHub] [kafka] jsancio commented on a change in pull request #10431: KAFKA-12543: Change RawSnapshotReader ownership model

2021-05-11 Thread GitBox
jsancio commented on a change in pull request #10431: URL: https://github.com/apache/kafka/pull/10431#discussion_r630369878 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -242,85 +248,116 @@ final class KafkaMetadataLog private ( } override de

[GitHub] [kafka] jsancio commented on a change in pull request #10431: KAFKA-12543: Change RawSnapshotReader ownership model

2021-05-11 Thread GitBox
jsancio commented on a change in pull request #10431: URL: https://github.com/apache/kafka/pull/10431#discussion_r630370013 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -242,85 +248,116 @@ final class KafkaMetadataLog private ( } override de

[GitHub] [kafka] dejan2609 edited a comment on pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-11 Thread GitBox
dejan2609 edited a comment on pull request #10606: URL: https://github.com/apache/kafka/pull/10606#issuecomment-838776681 > Looks like there is a change in behavior in Gradle 7 related to resource files that's causing a bunch of tests to fail Indeed... on my local machine I received

[GitHub] [kafka] dejan2609 commented on pull request #10626: KAFKA-12744: Breaking change dependency upgrade: "argparse4j" 0.7.0 -->> 0.9.0

2021-05-11 Thread GitBox
dejan2609 commented on pull request #10626: URL: https://github.com/apache/kafka/pull/10626#issuecomment-838902447 Changing status to draft (until test CLI, as mentioned in a comments above), -- This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [kafka] mingaliu opened a new pull request #10674: KAFKA-12713: Report the real fetch latency by removing the wait-time in purgatory.

2021-05-11 Thread GitBox
mingaliu opened a new pull request #10674: URL: https://github.com/apache/kafka/pull/10674 This is to help monitor the 'real' fetch latency by removing the waitTime when FetchRequest is in purgatory. The changes include: 1. Add waitTimeMs in FetchResponse() 2. In Kafka API handler

[jira] [Commented] (KAFKA-12713) Report "REAL" follower/consumer fetch latency

2021-05-11 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342778#comment-17342778 ] Ming Liu commented on KAFKA-12713: -- Here is PR [https://github.com/apache/kafka/pull/10

[GitHub] [kafka] Boojapho commented on pull request #10642: KAFKA-12756: Update Zookeeper to 3.6.3 or higher

2021-05-11 Thread GitBox
Boojapho commented on pull request #10642: URL: https://github.com/apache/kafka/pull/10642#issuecomment-838928359 > Also, we override the netty version, so I am not sure the ZK version needs to be bumped at all. So long as only one verson of netty remains in the release, that should

[jira] [Updated] (KAFKA-12766) Consider Disabling WAL-related Options in RocksDB

2021-05-11 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12766: --- Labels: newbie newbie++ (was: ) > Consider Disabling WAL-related Options in

[jira] [Assigned] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order

2021-05-11 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan reassigned KAFKA-12682: -- Assignee: Justine Olshan > Kraft MetadataPartitionsBuilder _localChanged and _localRe

[GitHub] [kafka] yeralin commented on a change in pull request #6592: KAFKA-8326: Introduce List Serde

2021-05-11 Thread GitBox
yeralin commented on a change in pull request #6592: URL: https://github.com/apache/kafka/pull/6592#discussion_r630523305 ## File path: clients/src/main/java/org/apache/kafka/common/serialization/ListDeserializer.java ## @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Softwar

[GitHub] [kafka] yeralin commented on a change in pull request #6592: KAFKA-8326: Introduce List Serde

2021-05-11 Thread GitBox
yeralin commented on a change in pull request #6592: URL: https://github.com/apache/kafka/pull/6592#discussion_r630539285 ## File path: clients/src/main/java/org/apache/kafka/common/serialization/ListSerializer.java ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] yeralin commented on a change in pull request #6592: KAFKA-8326: Introduce List Serde

2021-05-11 Thread GitBox
yeralin commented on a change in pull request #6592: URL: https://github.com/apache/kafka/pull/6592#discussion_r630539763 ## File path: clients/src/main/java/org/apache/kafka/common/serialization/ListSerializer.java ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] yeralin commented on a change in pull request #6592: KAFKA-8326: Introduce List Serde

2021-05-11 Thread GitBox
yeralin commented on a change in pull request #6592: URL: https://github.com/apache/kafka/pull/6592#discussion_r630542548 ## File path: clients/src/main/java/org/apache/kafka/common/serialization/Serdes.java ## @@ -125,6 +126,27 @@ public UUIDSerde() { } } +

[GitHub] [kafka] yeralin commented on a change in pull request #6592: KAFKA-8326: Introduce List Serde

2021-05-11 Thread GitBox
yeralin commented on a change in pull request #6592: URL: https://github.com/apache/kafka/pull/6592#discussion_r630542548 ## File path: clients/src/main/java/org/apache/kafka/common/serialization/Serdes.java ## @@ -125,6 +126,27 @@ public UUIDSerde() { } } +

[GitHub] [kafka] yeralin commented on a change in pull request #6592: KAFKA-8326: Introduce List Serde

2021-05-11 Thread GitBox
yeralin commented on a change in pull request #6592: URL: https://github.com/apache/kafka/pull/6592#discussion_r630550102 ## File path: clients/src/test/java/org/apache/kafka/common/serialization/SerializationTest.java ## @@ -106,6 +110,190 @@ public void stringSerdeShouldSupp

[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2

2021-05-11 Thread GitBox
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r630552512 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java ## @@ -525,6 +525,11 @@ private TransactionManager configu

[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2

2021-05-11 Thread GitBox
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r630553249 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java ## @@ -179,10 +179,18 @@ public void beginTransaction() thro

[GitHub] [kafka] ijuma commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2

2021-05-11 Thread GitBox
ijuma commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r630556204 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java ## @@ -525,6 +525,11 @@ private TransactionManager configureTrans

[GitHub] [kafka] cmccabe commented on a change in pull request #10504: KAFKA-12620 Allocate producer ids on the controller

2021-05-11 Thread GitBox
cmccabe commented on a change in pull request #10504: URL: https://github.com/apache/kafka/pull/10504#discussion_r630569991 ## File path: clients/src/main/resources/common/message/AllocateProducerIdsResponse.json ## @@ -0,0 +1,32 @@ +// Licensed to the Apache Software Foundati

[jira] [Commented] (KAFKA-9826) Log cleaning repeatedly picks same segment with no effect when first dirty offset is past start of active segment

2021-05-11 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342904#comment-17342904 ] Jun Rao commented on KAFKA-9826: [~zhangzs] : The issue was fixed in 2.4.2. Could you try

[GitHub] [kafka] cmccabe commented on a change in pull request #10504: KAFKA-12620 Allocate producer ids on the controller

2021-05-11 Thread GitBox
cmccabe commented on a change in pull request #10504: URL: https://github.com/apache/kafka/pull/10504#discussion_r630587756 ## File path: core/src/main/scala/kafka/controller/KafkaController.scala ## @@ -2376,6 +2375,82 @@ class KafkaController(val config: KafkaConfig, }

[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2

2021-05-11 Thread GitBox
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r630593372 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/MockProducer.java ## @@ -179,10 +179,18 @@ public void beginTransaction() thro

[GitHub] [kafka] cmccabe commented on pull request #10504: KAFKA-12620 Allocate producer ids on the controller

2021-05-11 Thread GitBox
cmccabe commented on pull request #10504: URL: https://github.com/apache/kafka/pull/10504#issuecomment-839261742 We need to gate the new behavior behind an IBP bump, right? Otherwise we won't have access to new producer IDs while rolling a cluster. -- This is an automated message from t

[GitHub] [kafka] mjsax commented on a change in pull request #10673: MINOR: set replication.factor to 1 to make StreamsBrokerCompatibility…

2021-05-11 Thread GitBox
mjsax commented on a change in pull request #10673: URL: https://github.com/apache/kafka/pull/10673#discussion_r630595939 ## File path: tests/kafkatest/services/streams.py ## @@ -466,6 +466,15 @@ def __init__(self, test_context, kafka, processingMode):

[GitHub] [kafka] cmccabe commented on a change in pull request #10550: MINOR: Add support for ZK Authorizer with KRaft

2021-05-11 Thread GitBox
cmccabe commented on a change in pull request #10550: URL: https://github.com/apache/kafka/pull/10550#discussion_r630598753 ## File path: tests/kafkatest/services/kafka/kafka.py ## @@ -863,7 +878,7 @@ def kafka_configs_cmd_with_optional_security_settings(self, node, force_use_

[GitHub] [kafka] cmccabe commented on a change in pull request #10550: MINOR: Add support for ZK Authorizer with KRaft

2021-05-11 Thread GitBox
cmccabe commented on a change in pull request #10550: URL: https://github.com/apache/kafka/pull/10550#discussion_r630601127 ## File path: tests/kafkatest/services/security/kafka_acls.py ## @@ -66,17 +67,46 @@ def add_cluster_acl(self, kafka, principal, force_use_zk_connection=

[GitHub] [kafka] ableegoldman opened a new pull request #10675: KAFKA-12574: remove internal Producer config and auto downgrade logic

2021-05-11 Thread GitBox
ableegoldman opened a new pull request #10675: URL: https://github.com/apache/kafka/pull/10675 Minor followup to [#10573](https://github.com/apache/kafka/pull/10573), see in particular [this comment thread](https://github.com/apache/kafka/pull/10573#discussion_r628777848). Removes this in

[GitHub] [kafka] mjsax commented on a change in pull request #10668: MINOR: Apply try-with-resource to KafkaStreamsTest

2021-05-11 Thread GitBox
mjsax commented on a change in pull request #10668: URL: https://github.com/apache/kafka/pull/10668#discussion_r630601302 ## File path: streams/src/test/java/org/apache/kafka/streams/KafkaStreamsTest.java ## @@ -491,25 +493,23 @@ public void testStateThreadClose() throws Except

[GitHub] [kafka] ableegoldman commented on a change in pull request #10573: KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2

2021-05-11 Thread GitBox
ableegoldman commented on a change in pull request #10573: URL: https://github.com/apache/kafka/pull/10573#discussion_r630601449 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java ## @@ -525,6 +525,11 @@ private TransactionManager configu

[GitHub] [kafka] mjsax commented on a change in pull request #10668: MINOR: Apply try-with-resource to KafkaStreamsTest

2021-05-11 Thread GitBox
mjsax commented on a change in pull request #10668: URL: https://github.com/apache/kafka/pull/10668#discussion_r630601702 ## File path: streams/src/test/java/org/apache/kafka/streams/KafkaStreamsTest.java ## @@ -525,14 +525,13 @@ public void testStateGlobalThreadClose() throws

[GitHub] [kafka] ableegoldman commented on a change in pull request #10673: MINOR: set replication.factor to 1 to make StreamsBrokerCompatibility…

2021-05-11 Thread GitBox
ableegoldman commented on a change in pull request #10673: URL: https://github.com/apache/kafka/pull/10673#discussion_r630601750 ## File path: tests/kafkatest/services/streams.py ## @@ -466,6 +466,15 @@ def __init__(self, test_context, kafka, processingMode):

[GitHub] [kafka] vitojeng commented on a change in pull request #10668: MINOR: Apply try-with-resource to KafkaStreamsTest

2021-05-11 Thread GitBox
vitojeng commented on a change in pull request #10668: URL: https://github.com/apache/kafka/pull/10668#discussion_r630602092 ## File path: streams/src/test/java/org/apache/kafka/streams/KafkaStreamsTest.java ## @@ -553,141 +552,152 @@ public void testInitializesAndDestroysMetr

[GitHub] [kafka] mjsax commented on a change in pull request #10668: MINOR: Apply try-with-resource to KafkaStreamsTest

2021-05-11 Thread GitBox
mjsax commented on a change in pull request #10668: URL: https://github.com/apache/kafka/pull/10668#discussion_r630602355 ## File path: streams/src/test/java/org/apache/kafka/streams/KafkaStreamsTest.java ## @@ -553,141 +552,152 @@ public void testInitializesAndDestroysMetrics

[GitHub] [kafka] cmccabe commented on a change in pull request #10550: MINOR: Add support for ZK Authorizer with KRaft

2021-05-11 Thread GitBox
cmccabe commented on a change in pull request #10550: URL: https://github.com/apache/kafka/pull/10550#discussion_r630602650 ## File path: core/src/main/scala/kafka/server/MetadataSupport.scala ## @@ -91,28 +93,42 @@ case class ZkSupport(adminManager: ZkAdminManager, override

[GitHub] [kafka] cmccabe commented on a change in pull request #10550: MINOR: Add support for ZK Authorizer with KRaft

2021-05-11 Thread GitBox
cmccabe commented on a change in pull request #10550: URL: https://github.com/apache/kafka/pull/10550#discussion_r630603137 ## File path: core/src/main/scala/kafka/server/MetadataSupport.scala ## @@ -91,28 +93,42 @@ case class ZkSupport(adminManager: ZkAdminManager, override

[GitHub] [kafka] vitojeng commented on a change in pull request #10668: MINOR: Apply try-with-resource to KafkaStreamsTest

2021-05-11 Thread GitBox
vitojeng commented on a change in pull request #10668: URL: https://github.com/apache/kafka/pull/10668#discussion_r630603389 ## File path: streams/src/test/java/org/apache/kafka/streams/KafkaStreamsTest.java ## @@ -491,25 +493,23 @@ public void testStateThreadClose() throws Exc

[GitHub] [kafka] vitojeng commented on a change in pull request #10668: MINOR: Apply try-with-resource to KafkaStreamsTest

2021-05-11 Thread GitBox
vitojeng commented on a change in pull request #10668: URL: https://github.com/apache/kafka/pull/10668#discussion_r630603482 ## File path: streams/src/test/java/org/apache/kafka/streams/KafkaStreamsTest.java ## @@ -525,14 +525,13 @@ public void testStateGlobalThreadClose() thro

[GitHub] [kafka] cmccabe commented on pull request #10550: MINOR: Add support for ZK Authorizer with KRaft

2021-05-11 Thread GitBox
cmccabe commented on pull request #10550: URL: https://github.com/apache/kafka/pull/10550#issuecomment-839281487 Thanks for working on this, @rondagostino , and sorry about the delays in reviewing. > We forward the Create/Remove operations to the controller, but this patch actually

[GitHub] [kafka] mjsax commented on pull request #10675: KAFKA-12574: remove internal Producer config and auto downgrade logic

2021-05-11 Thread GitBox
mjsax commented on pull request #10675: URL: https://github.com/apache/kafka/pull/10675#issuecomment-839282578 \cc @abbccdda and @guozhangwang to make sure we don't to anything bad removing this. -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [kafka] ableegoldman commented on a change in pull request #6592: KAFKA-8326: Introduce List Serde

2021-05-11 Thread GitBox
ableegoldman commented on a change in pull request #6592: URL: https://github.com/apache/kafka/pull/6592#discussion_r630605834 ## File path: clients/src/test/java/org/apache/kafka/common/serialization/SerializationTest.java ## @@ -106,6 +110,190 @@ public void stringSerdeShoul

[GitHub] [kafka] mjsax commented on a change in pull request #10673: MINOR: set replication.factor to 1 to make StreamsBrokerCompatibility…

2021-05-11 Thread GitBox
mjsax commented on a change in pull request #10673: URL: https://github.com/apache/kafka/pull/10673#discussion_r630610600 ## File path: tests/kafkatest/services/streams.py ## @@ -466,6 +466,15 @@ def __init__(self, test_context, kafka, processingMode):

[GitHub] [kafka] ijuma commented on a change in pull request #10675: KAFKA-12574: remove internal Producer config and auto downgrade logic

2021-05-11 Thread GitBox
ijuma commented on a change in pull request #10675: URL: https://github.com/apache/kafka/pull/10675#discussion_r630612053 ## File path: clients/src/main/java/org/apache/kafka/common/requests/TxnOffsetCommitRequest.java ## @@ -88,26 +84,13 @@ public Builder(final String transac

[GitHub] [kafka] ableegoldman commented on a change in pull request #10673: MINOR: set replication.factor to 1 to make StreamsBrokerCompatibility…

2021-05-11 Thread GitBox
ableegoldman commented on a change in pull request #10673: URL: https://github.com/apache/kafka/pull/10673#discussion_r630612173 ## File path: tests/kafkatest/services/streams.py ## @@ -466,6 +466,15 @@ def __init__(self, test_context, kafka, processingMode):

  1   2   >