[GitHub] [kafka] chia7712 commented on pull request #7703: MINOR: Avoid unnecessary tuple allocations in index binary search

2020-10-29 Thread GitBox
chia7712 commented on pull request #7703: URL: https://github.com/apache/kafka/pull/7703#issuecomment-718423727 @ijuma Is this still active? the method ```binarySearch``` is a hot method so this optimization LGTM This is an

[GitHub] [kafka] chia7712 commented on a change in pull request #9433: KAFKA-10607: Consistent behaviour for response errorCounts()

2020-10-29 Thread GitBox
chia7712 commented on a change in pull request #9433: URL: https://github.com/apache/kafka/pull/9433#discussion_r514055998 ## File path: clients/src/main/java/org/apache/kafka/common/requests/AbstractResponse.java ## @@ -55,6 +55,11 @@ public ByteBuffer serialize(ApiKeys apiKe

[GitHub] [kafka] chia7712 opened a new pull request #9527: MINOR: avoid unnecessary conversion and tuple when updating error met…

2020-10-29 Thread GitBox
chia7712 opened a new pull request #9527: URL: https://github.com/apache/kafka/pull/9527 The conversion (from java collection to scala collection) and tuple is unnecessary for hot method ```updateErrorMetrics``` ### Committer Checklist (excluded from commit message) - [ ] Verify

[GitHub] [kafka] yinchuanwang commented on pull request #6915: KAFKA-8334 Executor to retry delayed operations failed to obtain lock

2020-10-29 Thread GitBox
yinchuanwang commented on pull request #6915: URL: https://github.com/apache/kafka/pull/6915#issuecomment-718528255 @junrao We are facing the same issue(offset commit request timeout occasionally). You think the retry logic based on the flag tryCompletePending is the root cause. Sorry, i

[GitHub] [kafka] showuon commented on pull request #9104: KAFKA-10266: Update the connector config header.converter

2020-10-29 Thread GitBox
showuon commented on pull request #9104: URL: https://github.com/apache/kafka/pull/9104#issuecomment-718549066 @kkonstantine , I've updated the PR, please review again. Thanks. This is an automated message from the Apache Git

[GitHub] [kafka] showuon commented on a change in pull request #9104: KAFKA-10266: Update the connector config header.converter

2020-10-29 Thread GitBox
showuon commented on a change in pull request #9104: URL: https://github.com/apache/kafka/pull/9104#discussion_r514120305 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/ConnectorConfig.java ## @@ -89,11 +89,16 @@ public static final String VA

[GitHub] [kafka] showuon commented on a change in pull request #9525: KAFKA-10658 ErrantRecordReporter.report always return completed futur…

2020-10-29 Thread GitBox
showuon commented on a change in pull request #9525: URL: https://github.com/apache/kafka/pull/9525#discussion_r514121667 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/runtime/errors/ProcessingContextTest.java ## @@ -0,0 +1,55 @@ +/* + * Licensed to the

[GitHub] [kafka] chia7712 commented on a change in pull request #9525: KAFKA-10658 ErrantRecordReporter.report always return completed futur…

2020-10-29 Thread GitBox
chia7712 commented on a change in pull request #9525: URL: https://github.com/apache/kafka/pull/9525#discussion_r514132602 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/runtime/errors/ProcessingContextTest.java ## @@ -0,0 +1,55 @@ +/* + * Licensed to the

[GitHub] [kafka] chia7712 commented on pull request #6915: KAFKA-8334 Executor to retry delayed operations failed to obtain lock

2020-10-29 Thread GitBox
chia7712 commented on pull request #6915: URL: https://github.com/apache/kafka/pull/6915#issuecomment-718563890 @yinchuanwang Thanks for you report. #8657 offers another way to resolve the issue. Please take a look at #8657 and feel free to open jira if it can't resolve the issue you encou

[GitHub] [kafka] viktorsomogyi commented on a change in pull request #9519: KAFKA-10650: Use Murmur3 instead of MD5 in SkimpyOffsetMap

2020-10-29 Thread GitBox
viktorsomogyi commented on a change in pull request #9519: URL: https://github.com/apache/kafka/pull/9519#discussion_r514135717 ## File path: jmh-benchmarks/jmh.sh ## @@ -37,6 +37,6 @@ echo "gradle build done" echo "running JMH with args [$@]" -java -jar ${libDir}/kafka-jm

[GitHub] [kafka] chia7712 commented on a change in pull request #9519: KAFKA-10650: Use Murmur3 instead of MD5 in SkimpyOffsetMap

2020-10-29 Thread GitBox
chia7712 commented on a change in pull request #9519: URL: https://github.com/apache/kafka/pull/9519#discussion_r514141334 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/server/OffsetMapBenchmark.java ## @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Softwar

[GitHub] [kafka] shadikajevand commented on pull request #9359: kafka-10273 Connect Converters should produce actionable error messages

2020-10-29 Thread GitBox
shadikajevand commented on pull request #9359: URL: https://github.com/apache/kafka/pull/9359#issuecomment-718721824 @rhauch Thanks, I really appreciate your help. I have updated the code now. Please review when you have time. --

[GitHub] [kafka] lbradstreet commented on a change in pull request #9473: KAFKA-10545: Create topic IDs in ZooKeeper and Controller

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9473: URL: https://github.com/apache/kafka/pull/9473#discussion_r514302473 ## File path: core/src/main/scala/kafka/controller/ControllerContext.scala ## @@ -126,6 +130,26 @@ class ControllerContext { replicaStates.clear()

[GitHub] [kafka] lbradstreet commented on a change in pull request #9473: KAFKA-10545: Create topic IDs in ZooKeeper and Controller

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9473: URL: https://github.com/apache/kafka/pull/9473#discussion_r514302473 ## File path: core/src/main/scala/kafka/controller/ControllerContext.scala ## @@ -126,6 +130,26 @@ class ControllerContext { replicaStates.clear()

[GitHub] [kafka] vvcephei commented on a change in pull request #9426: MINOR: Fix flaky shouldRejectNonExistentStoreName

2020-10-29 Thread GitBox
vvcephei commented on a change in pull request #9426: URL: https://github.com/apache/kafka/pull/9426#discussion_r514316662 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/QueryableStateIntegrationTest.java ## @@ -470,6 +475,8 @@ public void shouldRejec

[GitHub] [kafka] vvcephei commented on pull request #9426: MINOR: Fix flaky shouldRejectNonExistentStoreName

2020-10-29 Thread GitBox
vvcephei commented on pull request #9426: URL: https://github.com/apache/kafka/pull/9426#issuecomment-718806219 Thanks for the reviews, all! @dongjinleekr : Sorry, I should have made the fix clear. It's just the one line where I changed `getStartedStreams` to `getRunningStreams`. The

[GitHub] [kafka] vvcephei commented on a change in pull request #9521: KAFKA-10638: Fix QueryableStateIntegrationTest

2020-10-29 Thread GitBox
vvcephei commented on a change in pull request #9521: URL: https://github.com/apache/kafka/pull/9521#discussion_r514323639 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StoreQueryIntegrationTest.java ## @@ -117,17 +126,34 @@ public void shouldQueryO

[GitHub] [kafka] vvcephei commented on a change in pull request #9521: KAFKA-10638: Fix QueryableStateIntegrationTest

2020-10-29 Thread GitBox
vvcephei commented on a change in pull request #9521: URL: https://github.com/apache/kafka/pull/9521#discussion_r514325710 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StoreQueryIntegrationTest.java ## @@ -153,51 +179,75 @@ public void shouldQuerySp

[GitHub] [kafka] vvcephei commented on pull request #9471: MINOR: Add Jenkinsfile to 2.6

2020-10-29 Thread GitBox
vvcephei commented on pull request #9471: URL: https://github.com/apache/kafka/pull/9471#issuecomment-718826291 Ok, I've run it a total of 5 times now, and it has passed two times and timed out three times. I did some analysis on the timed out logs, and found that every test that st

[GitHub] [kafka] hachikuji commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r514326022 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -210,65 +142,42 @@ public String toString() { }

[GitHub] [kafka] hachikuji commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
hachikuji commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-718828042 @chia7712 One thing that would be useful is running the producer-performance test, just to make sure the the performance is inline. Might be worth checking flame graphs as well.

[GitHub] [kafka] bbejeck opened a new pull request #9528: MINOR: Add releaseTarGz to args for building docs

2020-10-29 Thread GitBox
bbejeck opened a new pull request #9528: URL: https://github.com/apache/kafka/pull/9528 This PR adds the `releaseTarGz` command back into the `release.py::command_stage_docs` method when building docs for staging *Summary of testing strategy (including rationale) for the feature o

[GitHub] [kafka] abbccdda commented on pull request #9522: MINOR: revert initial principal PR

2020-10-29 Thread GitBox
abbccdda commented on pull request #9522: URL: https://github.com/apache/kafka/pull/9522#issuecomment-718869803 Reopening this PR to merge to 2.7 This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [kafka] abbccdda opened a new pull request #9529: Revert initial principal from 2.7

2020-10-29 Thread GitBox
abbccdda opened a new pull request #9529: URL: https://github.com/apache/kafka/pull/9529 Need to make sure 2.7 doesn't bump the request header unexpectedly. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage

[GitHub] [kafka] abbccdda commented on pull request #9529: Revert initial principal from 2.7

2020-10-29 Thread GitBox
abbccdda commented on pull request #9529: URL: https://github.com/apache/kafka/pull/9529#issuecomment-718871156 @bbejeck @hachikuji for a review This is an automated message from the Apache Git Service. To respond to the mess

[jira] [Created] (KAFKA-10660) Poll time out logstash

2020-10-29 Thread David (Jira)
David created KAFKA-10660: - Summary: Poll time out logstash Key: KAFKA-10660 URL: https://issues.apache.org/jira/browse/KAFKA-10660 Project: Kafka Issue Type: Bug Components: KafkaConnect

[GitHub] [kafka] vvcephei merged pull request #9471: MINOR: Add Jenkinsfile to 2.6

2020-10-29 Thread GitBox
vvcephei merged pull request #9471: URL: https://github.com/apache/kafka/pull/9471 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] vvcephei commented on pull request #9467: KAFKA-10515: Properly initialize nullable Serdes with default values

2020-10-29 Thread GitBox
vvcephei commented on pull request #9467: URL: https://github.com/apache/kafka/pull/9467#issuecomment-718882348 _Finally_ merged that Jenkinsfile PR (seeing a lot of timeouts, but decided just to go ahead and merge it). Merged in the 2.6 branch, and now the tests are running! We shou

[GitHub] [kafka] vvcephei commented on pull request #9521: KAFKA-10638: Fix QueryableStateIntegrationTest

2020-10-29 Thread GitBox
vvcephei commented on pull request #9521: URL: https://github.com/apache/kafka/pull/9521#issuecomment-718885978 The tests passed, and I answered the two review questions. I'll go ahead and merge. This is an automated message

[GitHub] [kafka] vvcephei merged pull request #9521: KAFKA-10638: Fix QueryableStateIntegrationTest

2020-10-29 Thread GitBox
vvcephei merged pull request #9521: URL: https://github.com/apache/kafka/pull/9521 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[jira] [Comment Edited] (KAFKA-10373) Kafka Reassign Partition is stuck with Java OutOfMemory error

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223041#comment-17223041 ] Brajesh Kumar edited comment on KAFKA-10373 at 10/29/20, 4:59 PM: ---

[jira] [Commented] (KAFKA-10373) Kafka Reassign Partition is stuck with Java OutOfMemory error

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223041#comment-17223041 ] Brajesh Kumar commented on KAFKA-10373: Hi What is your Environment configurati

[GitHub] [kafka] vvcephei commented on pull request #9414: KAFKA-10585: Kafka Streams should clean up the state store directory from cleanup

2020-10-29 Thread GitBox
vvcephei commented on pull request #9414: URL: https://github.com/apache/kafka/pull/9414#issuecomment-718889494 Hey @dongjinleekr , just a quick check-in: do you plan to take a look at the test failures? This is an automated

[jira] [Comment Edited] (KAFKA-9948) Gradle Issue

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223044#comment-17223044 ] Brajesh Kumar edited comment on KAFKA-9948 at 10/29/20, 5:02 PM: -

[jira] [Commented] (KAFKA-9948) Gradle Issue

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223044#comment-17223044 ] Brajesh Kumar commented on KAFKA-9948: --- [~dulvinw] Which command you are using to

[jira] [Updated] (KAFKA-10638) QueryableStateIntegrationTest fails due to stricter store checking

2020-10-29 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-10638: - Fix Version/s: (was: 2.7.1) (was: 2.8.0) 2.7.0 > Q

[jira] [Resolved] (KAFKA-10638) QueryableStateIntegrationTest fails due to stricter store checking

2020-10-29 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-10638. -- Resolution: Fixed > QueryableStateIntegrationTest fails due to stricter store checking > -

[GitHub] [kafka] vvcephei commented on pull request #9426: MINOR: Fix flaky shouldRejectNonExistentStoreName

2020-10-29 Thread GitBox
vvcephei commented on pull request #9426: URL: https://github.com/apache/kafka/pull/9426#issuecomment-718900823 The only build failure was: `Build / JDK 11 / org.apache.kafka.streams.integration.StreamTableJoinTopologyOptimizationIntegrationTest.shouldDoStreamTableJoinWithDifferentNumber

[jira] [Commented] (KAFKA-9892) Producer state snapshot needs to be forced to disk

2020-10-29 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223070#comment-17223070 ] Brajesh Kumar commented on KAFKA-9892: --- [~junrao] [~ijuma] Can I pick this up pl

[GitHub] [kafka] vvcephei merged pull request #9426: MINOR: Fix flaky shouldRejectNonExistentStoreName

2020-10-29 Thread GitBox
vvcephei merged pull request #9426: URL: https://github.com/apache/kafka/pull/9426 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[jira] [Commented] (KAFKA-9892) Producer state snapshot needs to be forced to disk

2020-10-29 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223094#comment-17223094 ] Jun Rao commented on KAFKA-9892: [~kbrajesh176]: Thanks for your interest. Just added you

[jira] [Created] (KAFKA-10661) Add resigned state to raft state machine to preserve leader/epoch information

2020-10-29 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-10661: --- Summary: Add resigned state to raft state machine to preserve leader/epoch information Key: KAFKA-10661 URL: https://issues.apache.org/jira/browse/KAFKA-10661 P

[GitHub] [kafka] hachikuji commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r514467652 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -210,65 +142,42 @@ public String toString() { }

[GitHub] [kafka] junrao commented on pull request #6915: KAFKA-8334 Executor to retry delayed operations failed to obtain lock

2020-10-29 Thread GitBox
junrao commented on pull request #6915: URL: https://github.com/apache/kafka/pull/6915#issuecomment-718933493 @yinchuanwang : The short answer is that the retry logic could cause a watcher event not to be triggered. If there is no future event triggering the same watcher, the corresponding

[GitHub] [kafka] abbccdda commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
abbccdda commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r513200010 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/DataInputStreamReadable.java ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Soft

[GitHub] [kafka] junrao commented on pull request #9495: KAFKA-10642: Expose the real stack trace if any exception occurred during SSL Client Trust Verification in extension

2020-10-29 Thread GitBox
junrao commented on pull request #9495: URL: https://github.com/apache/kafka/pull/9495#issuecomment-718942968 @rajinisivaram : What do you think? Thanks. This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] abbccdda commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
abbccdda commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514482551 ## File path: core/src/main/scala/kafka/raft/TimingWheelExpirationService.scala ## @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] senthilm-ms commented on pull request #9495: KAFKA-10642: Expose the real stack trace if any exception occurred during SSL Client Trust Verification in extension

2020-10-29 Thread GitBox
senthilm-ms commented on pull request #9495: URL: https://github.com/apache/kafka/pull/9495#issuecomment-718944686 > @rajinisivaram : What do you think? Thanks. @junrao @rajinisivaram - take a look at the stack trace mentioned in the JIRA bug where you don't see the extension/custom

[jira] [Created] (KAFKA-10662) Possible hanging test in 2.6 on JDK 11

2020-10-29 Thread John Roesler (Jira)
John Roesler created KAFKA-10662: Summary: Possible hanging test in 2.6 on JDK 11 Key: KAFKA-10662 URL: https://issues.apache.org/jira/browse/KAFKA-10662 Project: Kafka Issue Type: Bug Af

[GitHub] [kafka] cadonna opened a new pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
cadonna opened a new pull request #9530: URL: https://github.com/apache/kafka/pull/9530 The system test StreamsUpgradeTest.test_version_probing_upgrade tries to verify the wrong version for version probing. ### Committer Checklist (excluded from commit message) - [ ] Verify desi

[GitHub] [kafka] cadonna commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
cadonna commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-718960604 Call for review: @lct45 @ableegoldman @vvcephei This is an automated message from the Apache Git Service. To resp

[GitHub] [kafka] cadonna commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
cadonna commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-718961222 This fix needs to be cherry-picked to 2.7 and 2.6 This is an automated message from the Apache Git Service. To re

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514504593 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -1688,14 +1694,39 @@ private long pollCurrentState(long currentTimeMs)

[GitHub] [kafka] cadonna commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
cadonna commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-718965450 System tests run: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/4276/ This is an automated me

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514508755 ## File path: clients/src/main/java/org/apache/kafka/common/requests/AddOffsetsToTxnRequest.java ## @@ -57,6 +57,10 @@ public AddOffsetsToTxnRequest(S

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514512213 ## File path: clients/src/main/java/org/apache/kafka/common/metrics/stats/Percentiles.java ## @@ -113,11 +113,11 @@ protected HistogramSample newSample(l

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514514051 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/DataInputStreamReadable.java ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Sof

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514515763 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/DataInputStreamReadable.java ## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Sof

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514516570 ## File path: raft/src/main/java/org/apache/kafka/raft/BatchReader.java ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[jira] [Created] (KAFKA-10663) Flakey test ConsumerBounceTest#testSeekAndCommitWithBrokerFailures

2020-10-29 Thread Boyang Chen (Jira)
Boyang Chen created KAFKA-10663: --- Summary: Flakey test ConsumerBounceTest#testSeekAndCommitWithBrokerFailures Key: KAFKA-10663 URL: https://issues.apache.org/jira/browse/KAFKA-10663 Project: Kafka

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514518360 ## File path: raft/src/main/java/org/apache/kafka/raft/BatchReader.java ## @@ -0,0 +1,116 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] [kafka] abbccdda merged pull request #9529: Revert initial principal from 2.7

2020-10-29 Thread GitBox
abbccdda merged pull request #9529: URL: https://github.com/apache/kafka/pull/9529 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] hachikuji opened a new pull request #9531: KAFKA-10661; Add new resigned state for graceful shutdown/initialization

2020-10-29 Thread GitBox
hachikuji opened a new pull request #9531: URL: https://github.com/apache/kafka/pull/9531 When initializing the raft state machine after shutting down as a leader, we were previously entering the "unattached" state, which means we have no leader and no voted candidate. This was a bug becau

[GitHub] [kafka] bbejeck opened a new pull request #9532: MINOR: Move upgraded docs from site to kafak docs

2020-10-29 Thread GitBox
bbejeck opened a new pull request #9532: URL: https://github.com/apache/kafka/pull/9532 For the 2.7 release, we need to migrate some docs changes that went to `kafka-site` but didn't go into `kafka/docs` This PR covers the `documentation.html` and `uprade.html` changes. Once these

[GitHub] [kafka] lct45 commented on a change in pull request #9487: KAFKA-9331 add a streams handler

2020-10-29 Thread GitBox
lct45 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r514527605 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -346,26 +351,89 @@ public void setStateListener(final KafkaStreams.State

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514533712 ## File path: raft/src/main/java/org/apache/kafka/raft/internals/RecordsBatchReader.java ## @@ -0,0 +1,210 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514535761 ## File path: raft/src/main/java/org/apache/kafka/raft/internals/ThresholdPurgatory.java ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514536828 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -228,35 +234,81 @@ private void updateLeaderEndOffsetAndTimestamp(

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514537694 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -228,35 +234,81 @@ private void updateLeaderEndOffsetAndTimestamp(

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514539737 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -329,8 +387,9 @@ private void appendLeaderChangeMessage(LeaderState stat

[GitHub] [kafka] lct45 commented on a change in pull request #9487: KAFKA-9331 add a streams handler

2020-10-29 Thread GitBox
lct45 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r514535737 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -282,6 +284,15 @@ public boolean isRunning() {

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331 add a streams handler

2020-10-29 Thread GitBox
wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r514567028 ## File path: streams/src/main/java/org/apache/kafka/streams/errors/StreamsUncaughtExceptionHandler.java ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apac

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9487: KAFKA-9331 add a streams handler

2020-10-29 Thread GitBox
wcarlson5 commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r514566700 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -346,26 +351,89 @@ public void setStateListener(final KafkaStreams.S

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514574814 ## File path: core/src/main/scala/kafka/network/RequestChannel.scala ## @@ -162,7 +161,7 @@ object RequestChannel extends Logging { } } -

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514575372 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514576105 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514577021 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514577992 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] bbejeck commented on pull request #9532: MINOR: Move upgraded docs from site to kafak docs

2020-10-29 Thread GitBox
bbejeck commented on pull request #9532: URL: https://github.com/apache/kafka/pull/9532#issuecomment-719035513 Note that most of this PR already exists in 2.6 and doesn't need a close look This is an automated message from th

[GitHub] [kafka] bbejeck commented on a change in pull request #9532: MINOR: Move upgraded docs from site to kafak docs

2020-10-29 Thread GitBox
bbejeck commented on a change in pull request #9532: URL: https://github.com/apache/kafka/pull/9532#discussion_r514580970 ## File path: docs/upgrade.html ## @@ -21,6 +21,48 @@ Notable changes in 2.8.0 +Upgrading to 2.7.0 from any version 0.8.x through 2.6.x Review comme

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514584469 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -1688,14 +1694,39 @@ private long pollCurrentState(long currentTimeMs)

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514585353 ## File path: raft/src/main/java/org/apache/kafka/raft/RaftClient.java ## @@ -26,30 +24,53 @@ interface Listener { /** - * Callbac

[GitHub] [kafka] hachikuji commented on a change in pull request #9482: KAFKA-10632; Raft client should push all committed data to state machines

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9482: URL: https://github.com/apache/kafka/pull/9482#discussion_r514585353 ## File path: raft/src/main/java/org/apache/kafka/raft/RaftClient.java ## @@ -26,30 +24,53 @@ interface Listener { /** - * Callbac

[GitHub] [kafka] gitlw opened a new pull request #9533: Show log end offset during truncation to help estimate data loss during ULE

2020-10-29 Thread GitBox
gitlw opened a new pull request #9533: URL: https://github.com/apache/kafka/pull/9533 During Unclean Leader Election, there could be data loss due to truncation at the resigned leader. This PR tries to add more logs to understand the scale of message loss during an unclean leader electi

[GitHub] [kafka] hachikuji commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-10-29 Thread GitBox
hachikuji commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r514594596 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ProduceRequest.java ## @@ -210,65 +142,42 @@ public String toString() { }

[GitHub] [kafka] vvcephei merged pull request #9467: KAFKA-10515: Properly initialize nullable Serdes with default values

2020-10-29 Thread GitBox
vvcephei merged pull request #9467: URL: https://github.com/apache/kafka/pull/9467 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[jira] [Updated] (KAFKA-10515) NPE: Foreign key join serde may not be initialized with default serde if application is distributed

2020-10-29 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-10515: - Fix Version/s: 2.6.1 > NPE: Foreign key join serde may not be initialized with default serde if

[GitHub] [kafka] vvcephei commented on pull request #9467: KAFKA-10515: Properly initialize nullable Serdes with default values

2020-10-29 Thread GitBox
vvcephei commented on pull request #9467: URL: https://github.com/apache/kafka/pull/9467#issuecomment-719063925 Merged to 2.6 (cc @mimaison ) Thanks again for the contribution, @thake ! This is an automated message fro

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514610499 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514614216 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514617626 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] ableegoldman commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
ableegoldman commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-719074084 One flaky test failed: `EosBetaUpgradeIntegrationTest.shouldUpgradeFromEosAlphaToEosBeta[true]` Confirmed systems test passed. Merging to trunk --

[GitHub] [kafka] ableegoldman merged pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
ableegoldman merged pull request #9530: URL: https://github.com/apache/kafka/pull/9530 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514619891 ## File path: core/src/test/scala/unit/kafka/network/RequestConvertToJsonTest.scala ## @@ -0,0 +1,188 @@ +/** + * Licensed to the Apache Software Found

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514621561 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [kafka] mjsax commented on pull request #9518: KAFKA-10645: Add null check to the array/Iterable values in RecordHeaders constructor

2020-10-29 Thread GitBox
mjsax commented on pull request #9518: URL: https://github.com/apache/kafka/pull/9518#issuecomment-719084829 @showuon Seems we worked on this in parallel. Sorry for that... This is an automated message from the Apache Git Ser

[GitHub] [kafka] ableegoldman commented on pull request #9530: MINOR: Fix version verification in system test

2020-10-29 Thread GitBox
ableegoldman commented on pull request #9530: URL: https://github.com/apache/kafka/pull/9530#issuecomment-719085324 Cherrypicked to 2.6 & 2.7 cc/ @bbejeck This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] mjsax commented on pull request #9513: MINOR: improve `null` checks for headers

2020-10-29 Thread GitBox
mjsax commented on pull request #9513: URL: https://github.com/apache/kafka/pull/9513#issuecomment-719085653 > However, it may be overkill since users don't create ConsumerRecord. Yeah, I tend to agree. Would not hurt to add it, but should not be strictly necessary. Will merge this a

[GitHub] [kafka] mjsax merged pull request #9513: MINOR: improve `null` checks for headers

2020-10-29 Thread GitBox
mjsax merged pull request #9513: URL: https://github.com/apache/kafka/pull/9513 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] lbradstreet commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-10-29 Thread GitBox
lbradstreet commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r514635784 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,343 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

  1   2   >