[GitHub] [kafka] chia7712 commented on pull request #9204: KAFKA-6181 Examining log messages with {{--deep-iteration}} should show superset of fields

2020-11-16 Thread GitBox
chia7712 commented on pull request #9204: URL: https://github.com/apache/kafka/pull/9204#issuecomment-727809410 @iprithv Could you rebase code (or add trivial change) to trigger QA again? This is an automated message from the

[GitHub] [kafka] chia7712 commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-16 Thread GitBox
chia7712 commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-727825315 The last commit borrows some improvement from #9563. This is an automated message from the Apache Git Service. T

[jira] [Assigned] (KAFKA-10723) LogManager leaks internal thread pool activity during shutdown

2020-11-16 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Bentley reassigned KAFKA-10723: --- Assignee: Kowshik Prakasam > LogManager leaks internal thread pool activity during shutdown

[jira] [Assigned] (KAFKA-10720) Add note to Transformation docs to avoid mutating records or their members

2020-11-16 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Bentley reassigned KAFKA-10720: --- Assignee: Tom Bentley > Add note to Transformation docs to avoid mutating records or their

[GitHub] [kafka] cadonna commented on pull request #9508: KAFKA-10648: Add Prefix Scan support to State Stores

2020-11-16 Thread GitBox
cadonna commented on pull request #9508: URL: https://github.com/apache/kafka/pull/9508#issuecomment-727841634 @vamossagar12 I am really sorry. I haven't found the time yet, but it is on my ToDo list. I hope I will manage to make a pass this week. -

[jira] [Commented] (KAFKA-10683) Consumer.position() Ignores Transaction Marker with read_uncommitted

2020-11-16 Thread Timur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232635#comment-17232635 ] Timur commented on KAFKA-10683: --- [~grussell], absolutely - from the consumer side this mar

[jira] [Commented] (KAFKA-10062) Add a method to retrieve the current timestamp as known by the Streams app

2020-11-16 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232638#comment-17232638 ] Bruno Cadonna commented on KAFKA-10062: --- [~rohitdeshaws] I think [~wbottrell] and

[GitHub] [kafka] tombentley commented on a change in pull request #9549: KIP-145: Add SMTs, HeaderFrom, DropHeaders and InsertHeader

2020-11-16 Thread GitBox
tombentley commented on a change in pull request #9549: URL: https://github.com/apache/kafka/pull/9549#discussion_r524016349 ## File path: connect/transforms/src/test/java/org/apache/kafka/connect/transforms/HeaderFromTest.java ## @@ -0,0 +1,357 @@ +/* + * Licensed to the Apac

[GitHub] [kafka] tombentley commented on pull request #9549: KIP-145: Add SMTs, HeaderFrom, DropHeaders and InsertHeader

2020-11-16 Thread GitBox
tombentley commented on pull request #9549: URL: https://github.com/apache/kafka/pull/9549#issuecomment-727852419 @kkonstantine or perhaps @rhauch please could one of you take a look? This is an automated message from the Ap

[GitHub] [kafka] chia7712 commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-16 Thread GitBox
chia7712 commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-727856785 @hachikuji @ijuma @lbradstreet @dajac I have updated the perf result. The regression is reduced by last commit. Please take a look. -

[GitHub] [kafka] tombentley opened a new pull request #9597: KAFKA-10720: Document prohibition on header mutation by SMTs

2020-11-16 Thread GitBox
tombentley opened a new pull request #9597: URL: https://github.com/apache/kafka/pull/9597 Adds a sentence to the Javadoc for `Transformation` about not mutating headers. This is an automated message from the Apache Git Serv

[GitHub] [kafka] tombentley commented on pull request #9597: KAFKA-10720: Document prohibition on header mutation by SMTs

2020-11-16 Thread GitBox
tombentley commented on pull request #9597: URL: https://github.com/apache/kafka/pull/9597#issuecomment-727858963 @kkonstantine @rhauch please could you take look at this trivial PR? cc @C0urante. This is an automated

[GitHub] [kafka] lqjack commented on a change in pull request #9596: KAFKA-10723: Fix LogManager shutdown error handling

2020-11-16 Thread GitBox
lqjack commented on a change in pull request #9596: URL: https://github.com/apache/kafka/pull/9596#discussion_r524053292 ## File path: core/src/test/scala/unit/kafka/log/LogManagerTest.scala ## @@ -83,6 +87,51 @@ class LogManagerTest { log.appendAsLeader(TestUtils.singleto

[jira] [Created] (KAFKA-10726) How to detect heartbeat failure between broker/zookeeper leader

2020-11-16 Thread Keiichiro Wakasa (Jira)
Keiichiro Wakasa created KAFKA-10726: Summary: How to detect heartbeat failure between broker/zookeeper leader Key: KAFKA-10726 URL: https://issues.apache.org/jira/browse/KAFKA-10726 Project: Kafk

[jira] [Updated] (KAFKA-10726) How to detect heartbeat failure between broker/zookeeper leader

2020-11-16 Thread Keiichiro Wakasa (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keiichiro Wakasa updated KAFKA-10726: - Description: Hello experts, I'm not sure this is proper place to ask but I'd appreciate

[GitHub] [kafka] lijubjohn opened a new pull request #9598: fixes kafka-10701 : incorrect first line of detailed stats from consumer-perf-test.sh

2020-11-16 Thread GitBox
lijubjohn opened a new pull request #9598: URL: https://github.com/apache/kafka/pull/9598 Corrected the initialization of joinStart variable to fix the first line of the consumer performance stats. Since the joinStart was initialized with 0 , so when the first time onPartitionAssigned meth

[GitHub] [kafka] C0urante commented on a change in pull request #9597: KAFKA-10720: Document prohibition on header mutation by SMTs

2020-11-16 Thread GitBox
C0urante commented on a change in pull request #9597: URL: https://github.com/apache/kafka/pull/9597#discussion_r524160693 ## File path: connect/api/src/main/java/org/apache/kafka/connect/transforms/Transformation.java ## @@ -33,6 +33,9 @@ * Apply transformation to the {

[GitHub] [kafka] C0urante commented on a change in pull request #9597: KAFKA-10720: Document prohibition on header mutation by SMTs

2020-11-16 Thread GitBox
C0urante commented on a change in pull request #9597: URL: https://github.com/apache/kafka/pull/9597#discussion_r524160693 ## File path: connect/api/src/main/java/org/apache/kafka/connect/transforms/Transformation.java ## @@ -33,6 +33,9 @@ * Apply transformation to the {

[jira] [Commented] (KAFKA-10701) First line of detailed stats from consumer-perf-test.sh incorrect

2020-11-16 Thread Liju (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232698#comment-17232698 ] Liju commented on KAFKA-10701: -- Opened below pr for the fix https://github.com/apache/kafk

[jira] [Commented] (KAFKA-10652) Raft leader should flush accumulated writes after a min size is reached

2020-11-16 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232720#comment-17232720 ] Sagar Rao commented on KAFKA-10652: --- hi [~hachikuji], wanted to know if the approach m

[jira] [Created] (KAFKA-10727) Kafka clients throw AuthenticationException during Kerberos re-login

2020-11-16 Thread Rajini Sivaram (Jira)
Rajini Sivaram created KAFKA-10727: -- Summary: Kafka clients throw AuthenticationException during Kerberos re-login Key: KAFKA-10727 URL: https://issues.apache.org/jira/browse/KAFKA-10727 Project: Kaf

[jira] [Created] (KAFKA-10728) Mirroring data without decompressing with MirrorMaker 2.0

2020-11-16 Thread Eazhilan Nagarajan (Jira)
Eazhilan Nagarajan created KAFKA-10728: -- Summary: Mirroring data without decompressing with MirrorMaker 2.0 Key: KAFKA-10728 URL: https://issues.apache.org/jira/browse/KAFKA-10728 Project: Kafka

[GitHub] [kafka] C0urante opened a new pull request #9599: MINOR: Include connector name in error message

2020-11-16 Thread GitBox
C0urante opened a new pull request #9599: URL: https://github.com/apache/kafka/pull/9599 These log messages aren't triggered very frequently, but when they are it can indicate a serious problem with the connector, and it'd be nice to know exactly which connector is having that problem with

[GitHub] [kafka] kowshik commented on a change in pull request #9596: KAFKA-10723: Fix LogManager shutdown error handling

2020-11-16 Thread GitBox
kowshik commented on a change in pull request #9596: URL: https://github.com/apache/kafka/pull/9596#discussion_r524345459 ## File path: core/src/test/scala/unit/kafka/log/LogManagerTest.scala ## @@ -83,6 +87,51 @@ class LogManagerTest { log.appendAsLeader(TestUtils.singlet

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

2020-11-16 Thread GitBox
wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r524433881 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -366,6 +375,93 @@ public void setUncaughtExceptionHandler(final Thre

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

2020-11-16 Thread GitBox
wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r524435241 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamThreadTest.java ## @@ -1000,7 +1012,17 @@ public void restore(fina

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

2020-11-16 Thread GitBox
wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r524437940 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStreamThread.java ## @@ -311,6 +314,8 @@ public void run() {

[GitHub] [kafka] vvcephei commented on a change in pull request #9414: KAFKA-10585: Kafka Streams should clean up the state store directory from cleanup

2020-11-16 Thread GitBox
vvcephei commented on a change in pull request #9414: URL: https://github.com/apache/kafka/pull/9414#discussion_r524437121 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StateDirectoryTest.java ## @@ -186,19 +186,29 @@ public void shouldReport

[jira] [Resolved] (KAFKA-10704) Mirror maker with TLS at target

2020-11-16 Thread Tushar Bhasme (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tushar Bhasme resolved KAFKA-10704. --- Resolution: Not A Problem > Mirror maker with TLS at target > --

[jira] [Commented] (KAFKA-10704) Mirror maker with TLS at target

2020-11-16 Thread Tushar Bhasme (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232972#comment-17232972 ] Tushar Bhasme commented on KAFKA-10704: --- Sorry, I forgot to update this bug. We we

[GitHub] [kafka] vvcephei commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

2020-11-16 Thread GitBox
vvcephei commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r524448160 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -366,6 +377,90 @@ public void setUncaughtExceptionHandler(final Threa

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

2020-11-16 Thread GitBox
wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r524487609 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -559,18 +552,52 @@ void runLoop() {

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

2020-11-16 Thread GitBox
wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r524490211 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -366,6 +377,90 @@ public void setUncaughtExceptionHandler(final Thre

[jira] [Commented] (KAFKA-10722) Timestamped store is used even if not desired

2020-11-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232993#comment-17232993 ] Matthias J. Sax commented on KAFKA-10722: - Every record in Kafka Streams has a t

[jira] [Updated] (KAFKA-10721) Rewrite topology to allow for overlapping unequal topic subscriptions

2020-11-16 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-10721: -- Labels: newbie++ (was: ) > Rewrite topology to allow for overlapping unequal topic subscripti

[jira] [Comment Edited] (KAFKA-10722) Timestamped store is used even if not desired

2020-11-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232993#comment-17232993 ] Matthias J. Sax edited comment on KAFKA-10722 at 11/16/20, 6:50 PM: --

[jira] [Comment Edited] (KAFKA-10722) Timestamped store is used even if not desired

2020-11-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232993#comment-17232993 ] Matthias J. Sax edited comment on KAFKA-10722 at 11/16/20, 6:55 PM: --

[jira] [Comment Edited] (KAFKA-10722) Timestamped store is used even if not desired

2020-11-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232993#comment-17232993 ] Matthias J. Sax edited comment on KAFKA-10722 at 11/16/20, 6:59 PM: --

[GitHub] [kafka] splett2 commented on a change in pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-16 Thread GitBox
splett2 commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r521856417 ## File path: core/src/main/scala/kafka/network/SocketServer.scala ## @@ -1324,7 +1401,59 @@ class ConnectionQuotas(config: KafkaConfig, time: Time, metric

[GitHub] [kafka] splett2 commented on a change in pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-16 Thread GitBox
splett2 commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r521856417 ## File path: core/src/main/scala/kafka/network/SocketServer.scala ## @@ -1324,7 +1401,59 @@ class ConnectionQuotas(config: KafkaConfig, time: Time, metric

[GitHub] [kafka] lijubjohn commented on pull request #9598: KAFKA-10701 : First line of detailed stats from consumer-perf-test.sh incorrect

2020-11-16 Thread GitBox
lijubjohn commented on pull request #9598: URL: https://github.com/apache/kafka/pull/9598#issuecomment-728277968 @mumrah the return type of the method which is modified is void and the purpose of the method is to print the performance stats to console , so there isn't any direct hook to te

[GitHub] [kafka] lijubjohn edited a comment on pull request #9598: KAFKA-10701 : First line of detailed stats from consumer-perf-test.sh incorrect

2020-11-16 Thread GitBox
lijubjohn edited a comment on pull request #9598: URL: https://github.com/apache/kafka/pull/9598#issuecomment-728277968 @mumrah Thanks for reviewing the pr. The return type of the method which is modified is void and the purpose of the method is to print the performance stats to console

[GitHub] [kafka] abbccdda opened a new pull request #9600: KAFKA-10674: Controller API version bond with forwardable APIs

2020-11-16 Thread GitBox
abbccdda opened a new pull request #9600: URL: https://github.com/apache/kafka/pull/9600 To make sure the forwarded request could be properly handled by the controller, when forwarding is enabled, we should acquire the controller API versions to enforce as joint constraints back to the cli

[GitHub] [kafka] gardnervickers opened a new pull request #9601: MINOR: Enable flexible versioning for ListOffsetRequest/ListOffsetResponse.

2020-11-16 Thread GitBox
gardnervickers opened a new pull request #9601: URL: https://github.com/apache/kafka/pull/9601 This patch enables flexible versioning for ListOffsets req/response, as well as introducing a new IBP version allowing the replica fetchers to use this new ListOffsets version. -

[GitHub] [kafka] hachikuji commented on pull request #9589: KAFKA-10710 - Mirror Maker 2 - Create herders only if source->target.enabled=true

2020-11-16 Thread GitBox
hachikuji commented on pull request #9589: URL: https://github.com/apache/kafka/pull/9589#issuecomment-728294205 Thanks for the patch. The proposed fix makes sense. I am only trying to confirm that this is a defect and not intended. The code seems to explicitly allow the connector to be cr

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

2020-11-16 Thread GitBox
wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r524540416 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java ## @@ -255,8 +255,9 @@ public ByteBuffer s

[GitHub] [kafka] twobeeb commented on pull request #9589: KAFKA-10710 - Mirror Maker 2 - Create herders only if source->target.enabled=true

2020-11-16 Thread GitBox
twobeeb commented on pull request #9589: URL: https://github.com/apache/kafka/pull/9589#issuecomment-728339571 Thank you for your help @hachikuji. I agree with your analysis and it kind of makes sense for most use-cases with 2-3 clusters as described in the original KIP. I also understa

[GitHub] [kafka] ijuma commented on a change in pull request #9601: MINOR: Enable flexible versioning for ListOffsetRequest/ListOffsetResponse.

2020-11-16 Thread GitBox
ijuma commented on a change in pull request #9601: URL: https://github.com/apache/kafka/pull/9601#discussion_r524636462 ## File path: core/src/main/scala/kafka/api/ApiVersion.scala ## @@ -424,6 +426,13 @@ case object KAFKA_2_7_IV2 extends DefaultApiVersion { val id: Int = 30

[jira] [Commented] (KAFKA-10722) Timestamped store is used even if not desired

2020-11-16 Thread fml2 (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233116#comment-17233116 ] fml2 commented on KAFKA-10722: -- OK, I accept that Kafka Streams needs timestamps for the in

[GitHub] [kafka] ryannedolan commented on pull request #9589: KAFKA-10710 - Mirror Maker 2 - Create herders only if source->target.enabled=true

2020-11-16 Thread GitBox
ryannedolan commented on pull request #9589: URL: https://github.com/apache/kafka/pull/9589#issuecomment-728354718 > The code seems to explicitly allow the connector to be created even when the link is disabled. > @ryannedolan maybe you could clarify? My intention was to ensure th

[GitHub] [kafka] jolshan commented on a change in pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2020-11-16 Thread GitBox
jolshan commented on a change in pull request #9590: URL: https://github.com/apache/kafka/pull/9590#discussion_r524659149 ## File path: core/src/main/scala/kafka/log/LogCleaner.scala ## @@ -711,6 +723,9 @@ private[log] class Cleaner(val id: Int, shallowOffsetOfMaxTim

[GitHub] [kafka] jolshan commented on pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2020-11-16 Thread GitBox
jolshan commented on pull request #9590: URL: https://github.com/apache/kafka/pull/9590#issuecomment-728360755 I've updated the code to delay creating a new segment until there is a non-compacted record. If a segment is never created in cleanSegments, the old segments are simply deleted ra

[GitHub] [kafka] junrao commented on a change in pull request #9596: KAFKA-10723: Fix LogManager shutdown error handling

2020-11-16 Thread GitBox
junrao commented on a change in pull request #9596: URL: https://github.com/apache/kafka/pull/9596#discussion_r524673732 ## File path: core/src/main/scala/kafka/log/LogManager.scala ## @@ -477,27 +477,41 @@ class LogManager(logDirs: Seq[File], jobs(dir) = jobsForDir.map(

[jira] [Created] (KAFKA-10729) KIP-482: Bump remaining RPC's to use flexible versions

2020-11-16 Thread Gardner Vickers (Jira)
Gardner Vickers created KAFKA-10729: --- Summary: KIP-482: Bump remaining RPC's to use flexible versions Key: KAFKA-10729 URL: https://issues.apache.org/jira/browse/KAFKA-10729 Project: Kafka

[jira] [Updated] (KAFKA-10729) KIP-482: Bump remaining RPC's to use tagged fields

2020-11-16 Thread Gardner Vickers (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gardner Vickers updated KAFKA-10729: Summary: KIP-482: Bump remaining RPC's to use tagged fields (was: KIP-482: Bump remaining

[GitHub] [kafka] hachikuji commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-16 Thread GitBox
hachikuji commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-728383115 Posting allocation flame graphs from the producer before and after this patch: ![Screen Shot 2020-11-16 at 3 02 42 PM](https://user-images.githubusercontent.com/12502538/

[GitHub] [kafka] ijuma commented on a change in pull request #9566: KAFKA-10618: Update to Uuid class

2020-11-16 Thread GitBox
ijuma commented on a change in pull request #9566: URL: https://github.com/apache/kafka/pull/9566#discussion_r524722302 ## File path: clients/src/test/java/org/apache/kafka/common/UuidTest.java ## @@ -21,50 +21,50 @@ import static org.junit.Assert.assertEquals; import static

[GitHub] [kafka] hachikuji edited a comment on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-16 Thread GitBox
hachikuji edited a comment on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-728383115 Posting allocation flame graphs from the producer before and after this patch: ![Screen Shot 2020-11-16 at 3 02 42 PM](https://user-images.githubusercontent.com/12

[GitHub] [kafka] ijuma commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-16 Thread GitBox
ijuma commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-728385188 Nice! So what's the reason for the small regression in the PR description? This is an automated message from the Apa

[GitHub] [kafka] jolshan commented on a change in pull request #9566: KAFKA-10618: Update to Uuid class

2020-11-16 Thread GitBox
jolshan commented on a change in pull request #9566: URL: https://github.com/apache/kafka/pull/9566#discussion_r524727151 ## File path: clients/src/test/java/org/apache/kafka/common/UuidTest.java ## @@ -21,50 +21,50 @@ import static org.junit.Assert.assertEquals; import stati

[GitHub] [kafka] vvcephei commented on a change in pull request #9570: KAFKA-9274: Handle TimeoutException on commit

2020-11-16 Thread GitBox
vvcephei commented on a change in pull request #9570: URL: https://github.com/apache/kafka/pull/9570#discussion_r524564694 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -817,13 +817,15 @@ private void initializeMetadata(

[GitHub] [kafka] ijuma commented on a change in pull request #9566: KAFKA-10618: Update to Uuid class

2020-11-16 Thread GitBox
ijuma commented on a change in pull request #9566: URL: https://github.com/apache/kafka/pull/9566#discussion_r524738967 ## File path: clients/src/test/java/org/apache/kafka/common/UuidTest.java ## @@ -21,50 +21,50 @@ import static org.junit.Assert.assertEquals; import static

[jira] [Commented] (KAFKA-10062) Add a method to retrieve the current timestamp as known by the Streams app

2020-11-16 Thread Rohit Deshpande (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233158#comment-17233158 ] Rohit Deshpande commented on KAFKA-10062: - Thanks [~cadonna] I will wait for the

[jira] [Commented] (KAFKA-10062) Add a method to retrieve the current timestamp as known by the Streams app

2020-11-16 Thread William Bottrell (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233170#comment-17233170 ] William Bottrell commented on KAFKA-10062: -- Go ahead and take over. I had left

[GitHub] [kafka] hachikuji commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-16 Thread GitBox
hachikuji commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-728486374 Yeah, there is something strange going on, especially in regard to latency. Running the same producer performance test, I saw the following: ``` Patch: 25000 records

[GitHub] [kafka] jolshan commented on pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2020-11-16 Thread GitBox
jolshan commented on pull request #9590: URL: https://github.com/apache/kafka/pull/9590#issuecomment-728513284 Also looks like the test I added may be flaky, so I'll take a look at that. This is an automated message from

[GitHub] [kafka] ableegoldman commented on pull request #9583: [KAFKA-10705]: Make state stores not readable by others

2020-11-16 Thread GitBox
ableegoldman commented on pull request #9583: URL: https://github.com/apache/kafka/pull/9583#issuecomment-728523131 I don't think so. It would be nice to have if you happen to end up cutting a new RC, but I wouldn't delay the ongoing release over this -

[GitHub] [kafka] ijuma commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-16 Thread GitBox
ijuma commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-728531187 That is really weird. The difference seems significant enough that we need to understand it better before we can merge IMO.

[GitHub] [kafka] hachikuji commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-16 Thread GitBox
hachikuji commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-728545356 It would be helpful if someone can reproduce the tests I did to make sure it is not something funky in my environment. -

[GitHub] [kafka] ableegoldman commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

2020-11-16 Thread GitBox
ableegoldman commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r524814932 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -559,18 +552,52 @@ void runLoop() {

[GitHub] [kafka] ableegoldman commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

2020-11-16 Thread GitBox
ableegoldman commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r524817135 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -366,6 +374,63 @@ public void setUncaughtExceptionHandler(final T

[GitHub] [kafka] ableegoldman commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

2020-11-16 Thread GitBox
ableegoldman commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r524819063 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -559,18 +552,52 @@ void runLoop() {

[GitHub] [kafka] twobeeb commented on pull request #9589: KAFKA-10710 - Mirror Maker 2 - Create herders only if source->target.enabled=true

2020-11-16 Thread GitBox
twobeeb commented on pull request #9589: URL: https://github.com/apache/kafka/pull/9589#issuecomment-728593310 @ryannedolan @hachikuji If I understand correctly, when setting up a link A->B.enabled=true (with defaults settings regarding heartbeat), it creates a topic heartbeat which pr

[GitHub] [kafka] twobeeb edited a comment on pull request #9589: KAFKA-10710 - Mirror Maker 2 - Create herders only if source->target.enabled=true

2020-11-16 Thread GitBox
twobeeb edited a comment on pull request #9589: URL: https://github.com/apache/kafka/pull/9589#issuecomment-728593310 @ryannedolan @hachikuji If I understand correctly, when setting up a link A->B.enabled=true (with defaults settings regarding heartbeat), it creates a topic heartbeat wh

[GitHub] [kafka] gardnervickers commented on a change in pull request #9601: MINOR: Enable flexible versioning for ListOffsetRequest/ListOffsetResponse.

2020-11-16 Thread GitBox
gardnervickers commented on a change in pull request #9601: URL: https://github.com/apache/kafka/pull/9601#discussion_r524820388 ## File path: core/src/main/scala/kafka/api/ApiVersion.scala ## @@ -424,6 +426,13 @@ case object KAFKA_2_7_IV2 extends DefaultApiVersion { val id:

[GitHub] [kafka] twobeeb edited a comment on pull request #9589: KAFKA-10710 - Mirror Maker 2 - Create herders only if source->target.enabled=true

2020-11-16 Thread GitBox
twobeeb edited a comment on pull request #9589: URL: https://github.com/apache/kafka/pull/9589#issuecomment-728593310 @ryannedolan @hachikuji If I understand correctly, when setting up a link A->B.enabled=true (with defaults settings regarding heartbeat), it creates a topic heartbeat wh

[GitHub] [kafka] ableegoldman commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

2020-11-16 Thread GitBox
ableegoldman commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r524824649 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -559,18 +552,52 @@ void runLoop() {

[GitHub] [kafka] hachikuji edited a comment on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-16 Thread GitBox
hachikuji edited a comment on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-728383115 Posting allocation flame graphs from the producer before and after this patch: ![Screen Shot 2020-11-16 at 5 26 56 PM](https://user-images.githubusercontent.com/12

[GitHub] [kafka] hachikuji commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-16 Thread GitBox
hachikuji commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-728643380 I think the large difference in latency in my test is due to the producer's buffer pool getting exhausted. I was looking at the "bufferpool-wait-ratio" metric exposed in the pro

[GitHub] [kafka] kowshik commented on a change in pull request #9596: KAFKA-10723: Fix LogManager shutdown error handling

2020-11-16 Thread GitBox
kowshik commented on a change in pull request #9596: URL: https://github.com/apache/kafka/pull/9596#discussion_r524847888 ## File path: core/src/main/scala/kafka/log/LogManager.scala ## @@ -477,27 +477,41 @@ class LogManager(logDirs: Seq[File], jobs(dir) = jobsForDir.map

[GitHub] [kafka] kowshik commented on a change in pull request #9596: KAFKA-10723: Fix LogManager shutdown error handling

2020-11-16 Thread GitBox
kowshik commented on a change in pull request #9596: URL: https://github.com/apache/kafka/pull/9596#discussion_r524848584 ## File path: core/src/test/scala/unit/kafka/log/LogManagerTest.scala ## @@ -83,6 +87,51 @@ class LogManagerTest { log.appendAsLeader(TestUtils.singlet

[GitHub] [kafka] kowshik commented on a change in pull request #9596: KAFKA-10723: Fix LogManager shutdown error handling

2020-11-16 Thread GitBox
kowshik commented on a change in pull request #9596: URL: https://github.com/apache/kafka/pull/9596#discussion_r524853231 ## File path: core/src/test/scala/unit/kafka/log/LogManagerTest.scala ## @@ -83,6 +87,51 @@ class LogManagerTest { log.appendAsLeader(TestUtils.singlet

[jira] [Commented] (KAFKA-10062) Add a method to retrieve the current timestamp as known by the Streams app

2020-11-16 Thread Rohit Deshpande (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233230#comment-17233230 ] Rohit Deshpande commented on KAFKA-10062: - Thank you [~wbottrell]  > Add a meth

[jira] [Assigned] (KAFKA-10062) Add a method to retrieve the current timestamp as known by the Streams app

2020-11-16 Thread Rohit Deshpande (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohit Deshpande reassigned KAFKA-10062: --- Assignee: Rohit Deshpande (was: William Bottrell) > Add a method to retrieve the c

[GitHub] [kafka] kowshik commented on pull request #9596: KAFKA-10723: Fix LogManager shutdown error handling

2020-11-16 Thread GitBox
kowshik commented on pull request #9596: URL: https://github.com/apache/kafka/pull/9596#issuecomment-728654950 Thanks for the review @junrao! I have addressed the comments in f917f0c24cebbb0fb5eb7029ccb6676734b60b3e. This is

[GitHub] [kafka] chia7712 commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-16 Thread GitBox
chia7712 commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-728664200 @hachikuji @ijuma Thanks for all feedback. I'm going to do more tests :) This is an automated message from the Ap

[GitHub] [kafka] hachikuji commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-16 Thread GitBox
hachikuji commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-728669194 @chia7712 It might be worth checking the fancy new `toSend` implementation. I did a quick test and found that gc overhead actually increased with this change even though the new

[GitHub] [kafka] kowshik opened a new pull request #9602: MINOR: Use string interpolation in FinalizedFeatureCache

2020-11-16 Thread GitBox
kowshik opened a new pull request #9602: URL: https://github.com/apache/kafka/pull/9602 This is a small change. In this PR, I'm using string interpolation in `FinalizedFeatureCache` at places where string format was otherwise used. This just ensures uniformity, with this change we ensure t

[GitHub] [kafka] quanuw commented on pull request #9598: KAFKA-10701 : First line of detailed stats from consumer-perf-test.sh incorrect

2020-11-16 Thread GitBox
quanuw commented on pull request #9598: URL: https://github.com/apache/kafka/pull/9598#issuecomment-728698903 Hi @lijubjohn, can you explain how having joinStart initialized to 0L led to a negative fetchTimeMs? I'm new to the project and not sure how having joinStart initialized as 0L coul

[jira] [Commented] (KAFKA-10666) Kafka doesn't use keystore / key / truststore passwords for named SSL connections

2020-11-16 Thread lqjacklee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233277#comment-17233277 ] lqjacklee commented on KAFKA-10666: --- [~pfjason] Does https://issues.apache.org/jira/br

[GitHub] [kafka] lqjack commented on a change in pull request #9596: KAFKA-10723: Fix LogManager shutdown error handling

2020-11-16 Thread GitBox
lqjack commented on a change in pull request #9596: URL: https://github.com/apache/kafka/pull/9596#discussion_r524907470 ## File path: core/src/test/scala/unit/kafka/log/LogManagerTest.scala ## @@ -83,6 +87,51 @@ class LogManagerTest { log.appendAsLeader(TestUtils.singleto

[jira] [Resolved] (KAFKA-10709) Sender#sendProduceRequest should use auto-generated protocol directly

2020-11-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10709. Resolution: Won't Fix This fix will be included by https://github.com/apache/kafka/pull/94

[jira] [Commented] (KAFKA-10726) How to detect heartbeat failure between broker/zookeeper leader

2020-11-16 Thread lqjacklee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233300#comment-17233300 ] lqjacklee commented on KAFKA-10726: --- If you are seeing excessive pauses during garbage

[jira] [Commented] (KAFKA-10726) How to detect heartbeat failure between broker/zookeeper leader

2020-11-16 Thread Keiichiro Wakasa (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233304#comment-17233304 ] Keiichiro Wakasa commented on KAFKA-10726: -- [~Jack-Lee] Hello Jack, thank you

[jira] [Created] (KAFKA-10730) KafkaApis#handleProduceRequest should use auto-generated protocol

2020-11-16 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-10730: -- Summary: KafkaApis#handleProduceRequest should use auto-generated protocol Key: KAFKA-10730 URL: https://issues.apache.org/jira/browse/KAFKA-10730 Project: Kafka

[jira] [Updated] (KAFKA-10730) KafkaApis#handleProduceRequest should use auto-generated protocol

2020-11-16 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-10730: --- Description: This is follow-up of KAFKA-9628 the construction of ProduceResponse is able to

[GitHub] [kafka] dajac commented on pull request #9602: MINOR: Use string interpolation in FinalizedFeatureCache

2020-11-16 Thread GitBox
dajac commented on pull request #9602: URL: https://github.com/apache/kafka/pull/9602#issuecomment-728753741 Failed test is unrelated. Merging to trunk. This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] dajac merged pull request #9602: MINOR: Use string interpolation in FinalizedFeatureCache

2020-11-16 Thread GitBox
dajac merged pull request #9602: URL: https://github.com/apache/kafka/pull/9602 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the