[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

2021-11-22 Thread Matthew Sheppard (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17447243#comment-17447243 ] Matthew Sheppard commented on KAFKA-13289: -- Hi Eugen, I'm unsure I'm afraid. T

[GitHub] [kafka] dajac commented on pull request #11497: MINOR: Bump 2.7 to use 2.7.2

2021-11-22 Thread GitBox
dajac commented on pull request #11497: URL: https://github.com/apache/kafka/pull/11497#issuecomment-975299898 @mimaison I have uploaded `kafka-streams-2.7.2-test.jar`, `kafka_2.12-2.7.2.tgz`, and `kafka_2.13-2.7.2.tgz` to `s3://kafka-packages`. -- This is an automated message from the A

[GitHub] [kafka] dajac commented on pull request #11495: MINOR: Bump 2.6 to use 2.6.3

2021-11-22 Thread GitBox
dajac commented on pull request #11495: URL: https://github.com/apache/kafka/pull/11495#issuecomment-975302265 @mimaison I have uploaded `kafka-streams-2.6.3-test.jar`, `kafka_2.12-2.6.3.tgz`, and `kafka_2.13-2.6.3.tgz` to s3://kafka-packages. -- This is an automated message from the Apa

[GitHub] [kafka] dajac merged pull request #11519: MINOR: Update error log in `DefaultSslEngineFactory#createTrustStoreFromPem`

2021-11-22 Thread GitBox
dajac merged pull request #11519: URL: https://github.com/apache/kafka/pull/11519 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...

[GitHub] [kafka] dajac commented on pull request #11504: KAFKA-13457: socketChannel in Acceptor#accept is not closed upon IOException

2021-11-22 Thread GitBox
dajac commented on pull request #11504: URL: https://github.com/apache/kafka/pull/11504#issuecomment-975322642 @functioner It seems that the code does not compile. I seems to be due to https://github.com/apache/kafka/commit/30a9085d505ac301acee18e4cdcc98d8060e0388 which added an parameter

[GitHub] [kafka] dajac commented on a change in pull request #11504: KAFKA-13457: socketChannel in Acceptor#accept is not closed upon IOException

2021-11-22 Thread GitBox
dajac commented on a change in pull request #11504: URL: https://github.com/apache/kafka/pull/11504#discussion_r754109304 ## File path: core/src/test/scala/unit/kafka/network/SocketServerTest.scala ## @@ -867,6 +868,39 @@ class SocketServerTest { } } + @Test + def t

[GitHub] [kafka] mimaison commented on pull request #11495: MINOR: Bump 2.6 to use 2.6.3

2021-11-22 Thread GitBox
mimaison commented on pull request #11495: URL: https://github.com/apache/kafka/pull/11495#issuecomment-975375937 Awesome, thanks @dajac ! Uploading the Streams artifacts to that Confluent bucket is not great. Also it's always a bit annoying to chase people to get it done. I did not know

[GitHub] [kafka] mimaison merged pull request #11495: MINOR: Bump 2.6 to use 2.6.3

2021-11-22 Thread GitBox
mimaison merged pull request #11495: URL: https://github.com/apache/kafka/pull/11495 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr

[GitHub] [kafka] mimaison merged pull request #11497: MINOR: Bump 2.7 to use 2.7.2

2021-11-22 Thread GitBox
mimaison merged pull request #11497: URL: https://github.com/apache/kafka/pull/11497 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr

[GitHub] [kafka] dengziming opened a new pull request #11522: KAFKA-13242: Ensure UpdateFeatures is properly handled in KRaft

2021-11-22 Thread GitBox
dengziming opened a new pull request #11522: URL: https://github.com/apache/kafka/pull/11522 *More detailed description of your change* This patch fixes several problems with the UpdateFeatures API in KRaft: KafkaApis did not properly forward this request type to the controller.

[GitHub] [kafka] dengziming closed pull request #11274: KAFKA-13242: Handle UpdateFeatureRequest in kraft

2021-11-22 Thread GitBox
dengziming closed pull request #11274: URL: https://github.com/apache/kafka/pull/11274 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubs

[GitHub] [kafka] joshuagrisham commented on pull request #9492: KAFKA-10627: Added support for Connect TimestampConverter to convert multiple fields using a comma-separated list, and multiple input fo

2021-11-22 Thread GitBox
joshuagrisham commented on pull request #9492: URL: https://github.com/apache/kafka/pull/9492#issuecomment-975429363 I have done a rebase and then merged two new commits into my changes now. There were a lot of changes to Kafka's trunk branch since 13 months ago so the rebase has added a

[GitHub] [kafka] joshuagrisham edited a comment on pull request #9492: KAFKA-10627: Added support for Connect TimestampConverter to convert multiple fields using a comma-separated list, and multiple i

2021-11-22 Thread GitBox
joshuagrisham edited a comment on pull request #9492: URL: https://github.com/apache/kafka/pull/9492#issuecomment-975429363 Thanks again @rhauch I have done a rebase and then merged two new commits into my changes now. There were a lot of changes to Kafka's trunk branch since 13 month

[GitHub] [kafka] camilesing commented on pull request #8575: KAFKA-8713 KIP-581 Add accept.optional.null to solve optional null

2021-11-22 Thread GitBox
camilesing commented on pull request #8575: URL: https://github.com/apache/kafka/pull/8575#issuecomment-975447966 Do you have any updates or possible workaround? @rhauch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [kafka] dengziming commented on pull request #11522: KAFKA-13242: Ensure UpdateFeatures is properly handled in KRaft

2021-11-22 Thread GitBox
dengziming commented on pull request #11522: URL: https://github.com/apache/kafka/pull/11522#issuecomment-975485415 Hello @cmccabe ,PTAL, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] cadonna commented on a change in pull request #11455: KAFKA-13423: GlobalThread should not log ERROR on clean shutdown

2021-11-22 Thread GitBox
cadonna commented on a change in pull request #11455: URL: https://github.com/apache/kafka/pull/11455#discussion_r754267786 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -655,10 +656,11 @@ public synchronized void onChange(final Thread thr

[jira] [Commented] (KAFKA-10603) Re-design KStream.process() and K*.transform*() operations

2021-11-22 Thread Anders Kirkeby (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17447393#comment-17447393 ] Anders Kirkeby commented on KAFKA-10603: With the deprecation of the _org.apache

[GitHub] [kafka] cadonna commented on a change in pull request #11455: KAFKA-13423: GlobalThread should not log ERROR on clean shutdown

2021-11-22 Thread GitBox
cadonna commented on a change in pull request #11455: URL: https://github.com/apache/kafka/pull/11455#discussion_r754267786 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -655,10 +656,11 @@ public synchronized void onChange(final Thread thr

[jira] [Comment Edited] (KAFKA-10603) Re-design KStream.process() and K*.transform*() operations

2021-11-22 Thread Anders Kirkeby (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17447393#comment-17447393 ] Anders Kirkeby edited comment on KAFKA-10603 at 11/22/21, 1:26 PM: ---

[GitHub] [kafka] cadonna commented on a change in pull request #11455: KAFKA-13423: GlobalThread should not log ERROR on clean shutdown

2021-11-22 Thread GitBox
cadonna commented on a change in pull request #11455: URL: https://github.com/apache/kafka/pull/11455#discussion_r754277424 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -655,10 +656,11 @@ public synchronized void onChange(final Thread thr

[jira] [Comment Edited] (KAFKA-10603) Re-design KStream.process() and K*.transform*() operations

2021-11-22 Thread Anders Kirkeby (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17447393#comment-17447393 ] Anders Kirkeby edited comment on KAFKA-10603 at 11/22/21, 1:27 PM: ---

[GitHub] [kafka] cadonna commented on a change in pull request #11455: KAFKA-13423: GlobalThread should not log ERROR on clean shutdown

2021-11-22 Thread GitBox
cadonna commented on a change in pull request #11455: URL: https://github.com/apache/kafka/pull/11455#discussion_r754277424 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -655,10 +656,11 @@ public synchronized void onChange(final Thread thr

[GitHub] [kafka] cadonna commented on pull request #11455: KAFKA-13423: GlobalThread should not log ERROR on clean shutdown

2021-11-22 Thread GitBox
cadonna commented on pull request #11455: URL: https://github.com/apache/kafka/pull/11455#issuecomment-975529501 > In that sense, I feel we can still safely remove the whole block I agree with you. But we need tests to verify this behavior. -- This is an automated message from the

[GitHub] [kafka] mimaison merged pull request #11500: KAFKA-13455: Add steps to run Kafka Connect to quickstart

2021-11-22 Thread GitBox
mimaison merged pull request #11500: URL: https://github.com/apache/kafka/pull/11500 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr

[jira] [Resolved] (KAFKA-13455) The Apache Kafka quickstart guide does not contain any steps for running Kafka Connect

2021-11-22 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-13455. Fix Version/s: 3.2.0 Resolution: Fixed > The Apache Kafka quickstart guide does not

[GitHub] [kafka] mimaison commented on pull request #11500: KAFKA-13455: Add steps to run Kafka Connect to quickstart

2021-11-22 Thread GitBox
mimaison commented on pull request #11500: URL: https://github.com/apache/kafka/pull/11500#issuecomment-975537693 @katheris Let's also update the website, otherwise we'll have to wait for the next release. Can you open a similar PR on https://github.com/apache/kafka-site? Thanks -- This

[GitHub] [kafka] rhauch commented on pull request #9492: KAFKA-10627: Added support for Connect TimestampConverter to convert multiple fields using a comma-separated list, and multiple input formats w

2021-11-22 Thread GitBox
rhauch commented on pull request #9492: URL: https://github.com/apache/kafka/pull/9492#issuecomment-975566960 Thanks, Joshua. Normally, to fix merge conflicts it's best to use `git merge trunk` rather than rebase. It does add a merge commit, but GitHub is smart enough to keep showing only

[GitHub] [kafka] rhauch edited a comment on pull request #9492: KAFKA-10627: Added support for Connect TimestampConverter to convert multiple fields using a comma-separated list, and multiple input fo

2021-11-22 Thread GitBox
rhauch edited a comment on pull request #9492: URL: https://github.com/apache/kafka/pull/9492#issuecomment-975566960 Thanks, Joshua. Normally, to fix merge conflicts it's best to use `git merge trunk` rather than rebase. It does add a merge commit, but GitHub is smart enough to keep showin

[GitHub] [kafka] C0urante commented on a change in pull request #11046: KAFKA-12980: Return empty record batch from Consumer::poll when position advances due to aborted transactions

2021-11-22 Thread GitBox
C0urante commented on a change in pull request #11046: URL: https://github.com/apache/kafka/pull/11046#discussion_r754402147 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java ## @@ -1235,9 +1236,15 @@ public void assign(Collection partit

[GitHub] [kafka] C0urante commented on pull request #11046: KAFKA-12980: Return empty record batch from Consumer::poll when position advances due to aborted transactions

2021-11-22 Thread GitBox
C0urante commented on pull request #11046: URL: https://github.com/apache/kafka/pull/11046#issuecomment-975659468 Thanks @hachikuji. I've addressed the nits and tried to address the more substantial comment regarding logging; interested in your thoughts when you have a moment. -- This i

[GitHub] [kafka] C0urante commented on a change in pull request #11382: KAFKA-13348: Allow Source Tasks to Handle Producer Exceptions

2021-11-22 Thread GitBox
C0urante commented on a change in pull request #11382: URL: https://github.com/apache/kafka/pull/11382#discussion_r754408473 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java ## @@ -366,7 +367,11 @@ private boolean sendRecords()

[GitHub] [kafka] rondagostino commented on a change in pull request #11503: KAFKA-13456: Tighten KRaft config checks/constraints

2021-11-22 Thread GitBox
rondagostino commented on a change in pull request #11503: URL: https://github.com/apache/kafka/pull/11503#discussion_r754409736 ## File path: core/src/main/scala/kafka/server/KafkaConfig.scala ## @@ -2018,12 +2019,68 @@ class KafkaConfig private(doLog: Boolean, val props: jav

[jira] [Created] (KAFKA-13469) End-of-life offset commit for source task can take place before all records are flushed

2021-11-22 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-13469: - Summary: End-of-life offset commit for source task can take place before all records are flushed Key: KAFKA-13469 URL: https://issues.apache.org/jira/browse/KAFKA-13469

[GitHub] [kafka] jeqo commented on a change in pull request #11481: KAFKA-13117: migrate TupleForwarder and CacheFlushListener to new Record API

2021-11-22 Thread GitBox
jeqo commented on a change in pull request #11481: URL: https://github.com/apache/kafka/pull/11481#discussion_r754432136 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamSlidingWindowAggregate.java ## @@ -467,27 +464,26 @@ private boolean r

[GitHub] [kafka] naanagon commented on pull request #11516: MINOR: Use MessageDigest equals when comparing signature

2021-11-22 Thread GitBox
naanagon commented on pull request #11516: URL: https://github.com/apache/kafka/pull/11516#issuecomment-975692771 @showuon, thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [kafka] functioner commented on a change in pull request #11504: KAFKA-13457: socketChannel in Acceptor#accept is not closed upon IOException

2021-11-22 Thread GitBox
functioner commented on a change in pull request #11504: URL: https://github.com/apache/kafka/pull/11504#discussion_r754464724 ## File path: core/src/test/scala/unit/kafka/network/SocketServerTest.scala ## @@ -867,6 +868,39 @@ class SocketServerTest { } } + @Test +

[GitHub] [kafka] joshuagrisham opened a new pull request #11523: KAFKA-10627: Added support for Connect TimestampConverter to convert multiple fields using a comma-separated list, and multiple input f

2021-11-22 Thread GitBox
joshuagrisham opened a new pull request #11523: URL: https://github.com/apache/kafka/pull/11523 > Note this PR is copied from #9492 where I mistakenly did a rebase on a very old change which added thousands of commits, so this is a second attempt to get a very clean PR for this change inst

[GitHub] [kafka] joshuagrisham closed pull request #9492: KAFKA-10627: Added support for Connect TimestampConverter to convert multiple fields using a comma-separated list, and multiple input formats

2021-11-22 Thread GitBox
joshuagrisham closed pull request #9492: URL: https://github.com/apache/kafka/pull/9492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsub

[GitHub] [kafka] joshuagrisham commented on pull request #9492: KAFKA-10627: Added support for Connect TimestampConverter to convert multiple fields using a comma-separated list, and multiple input fo

2021-11-22 Thread GitBox
joshuagrisham commented on pull request #9492: URL: https://github.com/apache/kafka/pull/9492#issuecomment-975729890 Thanks @rhauch sorry about the confusion! I will close this one now and I have opened a new PR #11523 where I will tag you to take a look! -- This is an automated message

[GitHub] [kafka] joshuagrisham commented on pull request #11523: KAFKA-10627: Added support for Connect TimestampConverter to convert multiple fields using a comma-separated list, and multiple input f

2021-11-22 Thread GitBox
joshuagrisham commented on pull request #11523: URL: https://github.com/apache/kafka/pull/11523#issuecomment-975732575 @rhauch this is the new PR to replace #9492 per your comment there -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [kafka] functioner commented on a change in pull request #11504: KAFKA-13457: socketChannel in Acceptor#accept is not closed upon IOException

2021-11-22 Thread GitBox
functioner commented on a change in pull request #11504: URL: https://github.com/apache/kafka/pull/11504#discussion_r754481036 ## File path: core/src/main/scala/kafka/network/SocketServer.scala ## @@ -734,9 +730,21 @@ private[kafka] class Acceptor(val endPoint: EndPoint,

[GitHub] [kafka] functioner commented on a change in pull request #11504: KAFKA-13457: socketChannel in Acceptor#accept is not closed upon IOException

2021-11-22 Thread GitBox
functioner commented on a change in pull request #11504: URL: https://github.com/apache/kafka/pull/11504#discussion_r754482445 ## File path: core/src/test/scala/unit/kafka/network/SocketServerTest.scala ## @@ -867,6 +868,39 @@ class SocketServerTest { } } + @Test +

[GitHub] [kafka] junrao commented on a change in pull request #11345: Allow empty last segment to have missing offset index during recovery

2021-11-22 Thread GitBox
junrao commented on a change in pull request #11345: URL: https://github.com/apache/kafka/pull/11345#discussion_r754503632 ## File path: core/src/main/scala/kafka/log/UnifiedLog.scala ## @@ -1504,22 +1504,30 @@ class UnifiedLog(@volatile var logStartOffset: Long, /** *

[jira] [Commented] (KAFKA-13467) Clients never refresh cached bootstrap IPs

2021-11-22 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17447547#comment-17447547 ] Matthias J. Sax commented on KAFKA-13467: - I am not familiar with the details. [

[GitHub] [kafka] C0urante opened a new pull request #11524: KAFKA-13469: Block for in-flight record delivery before end-of-life source task offset commit

2021-11-22 Thread GitBox
C0urante opened a new pull request #11524: URL: https://github.com/apache/kafka/pull/11524 [Jira](https://issues.apache.org/jira/browse/KAFKA-13469) Although committing source task offsets without blocking on the delivery of all in-flight records is beneficial most of the time, it ca

[GitHub] [kafka] C0urante commented on a change in pull request #11524: KAFKA-13469: Block for in-flight record delivery before end-of-life source task offset commit

2021-11-22 Thread GitBox
C0urante commented on a change in pull request #11524: URL: https://github.com/apache/kafka/pull/11524#discussion_r754528381 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java ## @@ -260,6 +259,10 @@ public void execute() {

[jira] [Created] (KAFKA-13470) Fix the handling of secure controller ports

2021-11-22 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-13470: Summary: Fix the handling of secure controller ports Key: KAFKA-13470 URL: https://issues.apache.org/jira/browse/KAFKA-13470 Project: Kafka Issue Type: Bug

[GitHub] [kafka] C0urante commented on a change in pull request #11524: KAFKA-13469: Block for in-flight record delivery before end-of-life source task offset commit

2021-11-22 Thread GitBox
C0urante commented on a change in pull request #11524: URL: https://github.com/apache/kafka/pull/11524#discussion_r754530063 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SubmittedRecords.java ## @@ -132,6 +144,27 @@ public CommittableOffsets com

[GitHub] [kafka] jsancio opened a new pull request #11525: MINOR: Share notification code for coordinators

2021-11-22 Thread GitBox
jsancio opened a new pull request #11525: URL: https://github.com/apache/kafka/pull/11525 Leader election and resignation logic for the Group Coordinator and Transaction Coordinator is the same. Share this logic by refactoring this code to a higher order method. ### Committer Checkl

[jira] [Updated] (KAFKA-13470) Test the handling of secure controller ports

2021-11-22 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe updated KAFKA-13470: - Summary: Test the handling of secure controller ports (was: Fix the handling of secure controll

[jira] [Updated] (KAFKA-13470) Test the handling of secure controller ports

2021-11-22 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe updated KAFKA-13470: - Description: It should be possible to configure the controller to use secure ports. We need a j

[GitHub] [kafka] naanagon edited a comment on pull request #11516: MINOR: Use MessageDigest equals when comparing signature

2021-11-22 Thread GitBox
naanagon edited a comment on pull request #11516: URL: https://github.com/apache/kafka/pull/11516#issuecomment-975692771 @showuon, thank you. @dajac, Could you please take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [kafka] cmccabe commented on pull request #11503: KAFKA-13456: Tighten KRaft config checks/constraints

2021-11-22 Thread GitBox
cmccabe commented on pull request #11503: URL: https://github.com/apache/kafka/pull/11503#issuecomment-975815825 @rondagostino : I think we need to discuss the question of whether to include `controller.listener.names` in `listener.names` a bit more. The case for including it: 1.

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

2021-11-22 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17447566#comment-17447566 ] Matthias J. Sax commented on KAFKA-13289: - [~eugendueck] could your issue be rel

[GitHub] [kafka] wcarlson5 commented on a change in pull request #11479: KAFKA-12648: Make changing the named topologies blocking

2021-11-22 Thread GitBox
wcarlson5 commented on a change in pull request #11479: URL: https://github.com/apache/kafka/pull/11479#discussion_r754573181 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/namedtopology/KafkaStreamsNamedTopologyWrapper.java ## @@ -118,40 +133

[GitHub] [kafka] rondagostino commented on a change in pull request #11503: KAFKA-13456: Tighten KRaft config checks/constraints

2021-11-22 Thread GitBox
rondagostino commented on a change in pull request #11503: URL: https://github.com/apache/kafka/pull/11503#discussion_r754608137 ## File path: core/src/main/scala/kafka/server/KafkaConfig.scala ## @@ -1992,18 +1993,6 @@ class KafkaConfig private(doLog: Boolean, val props: java

[jira] [Created] (KAFKA-13471) Test rolling change of KRaft controller endpoints

2021-11-22 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-13471: Summary: Test rolling change of KRaft controller endpoints Key: KAFKA-13471 URL: https://issues.apache.org/jira/browse/KAFKA-13471 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-13471) Test rolling change of KRaft controller endpoints

2021-11-22 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe updated KAFKA-13471: - Component/s: controller kraft Labels: test (was: ) > Test rolling cha

[GitHub] [kafka] rondagostino commented on a change in pull request #11503: KAFKA-13456: Tighten KRaft config checks/constraints

2021-11-22 Thread GitBox
rondagostino commented on a change in pull request #11503: URL: https://github.com/apache/kafka/pull/11503#discussion_r754649102 ## File path: core/src/main/scala/kafka/server/KafkaConfig.scala ## @@ -2007,8 +2007,13 @@ class KafkaConfig private(doLog: Boolean, val props: java

[GitHub] [kafka] rondagostino commented on a change in pull request #11503: KAFKA-13456: Tighten KRaft config checks/constraints

2021-11-22 Thread GitBox
rondagostino commented on a change in pull request #11503: URL: https://github.com/apache/kafka/pull/11503#discussion_r754649102 ## File path: core/src/main/scala/kafka/server/KafkaConfig.scala ## @@ -2007,8 +2007,13 @@ class KafkaConfig private(doLog: Boolean, val props: java

[jira] [Created] (KAFKA-13472) Connect can lose track of last committed offsets for topic partitions after partial consumer revocation

2021-11-22 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-13472: - Summary: Connect can lose track of last committed offsets for topic partitions after partial consumer revocation Key: KAFKA-13472 URL: https://issues.apache.org/jira/browse/KAFK

[GitHub] [kafka] C0urante opened a new pull request #11526: KAFKA-13472: Correct last-committed offsets tracking for sink tasks after partial revocation

2021-11-22 Thread GitBox
C0urante opened a new pull request #11526: URL: https://github.com/apache/kafka/pull/11526 [Jira](https://issues.apache.org/jira/browse/KAFKA-13472) The `WorkerSinkTask.lastCommittedOffsets` field is now added to (via `Map::putAll`) after a successful offset commit, instead of being

[GitHub] [kafka] cmccabe merged pull request #11525: MINOR: Share notification code for coordinators

2021-11-22 Thread GitBox
cmccabe merged pull request #11525: URL: https://github.com/apache/kafka/pull/11525 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr.

[GitHub] [kafka] cmccabe opened a new pull request #11527: MINOR: store producer IDs in broker snapshots

2021-11-22 Thread GitBox
cmccabe opened a new pull request #11527: URL: https://github.com/apache/kafka/pull/11527 When creating snapshots, controllers generate a ProducerIdsRecord indicating the highest producer ID that has been used so far. Brokers should generate the same record, so that the snapshots can be

[jira] [Updated] (KAFKA-13472) Connect can lose track of last committed offsets for topic partitions after partial consumer revocation

2021-11-22 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-13472: -- Fix Version/s: 3.1.0 3.0.1 > Connect can lose track of last committed offse

[jira] [Updated] (KAFKA-13472) Connect can lose track of last committed offsets for topic partitions after partial consumer revocation

2021-11-22 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-13472: -- Priority: Blocker (was: Major) > Connect can lose track of last committed offsets for topic p

[GitHub] [kafka] rondagostino commented on pull request #11503: KAFKA-13456: Tighten KRaft config checks/constraints

2021-11-22 Thread GitBox
rondagostino commented on pull request #11503: URL: https://github.com/apache/kafka/pull/11503#issuecomment-975956591 > I think we need to discuss the question of whether to include controller.listener.names in listener.names a bit more. @cmccabe As discussed offline, it turns out th

[GitHub] [kafka] jeqo commented on a change in pull request #11481: KAFKA-13117: migrate TupleForwarder and CacheFlushListener to new Record API

2021-11-22 Thread GitBox
jeqo commented on a change in pull request #11481: URL: https://github.com/apache/kafka/pull/11481#discussion_r754687365 ## File path: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregateProcessorTest.java ## @@ -116,6 +116,7 @@ privat

[GitHub] [kafka] jsancio commented on a change in pull request #11527: KAFKA-13357: store producer IDs in broker snapshots

2021-11-22 Thread GitBox
jsancio commented on a change in pull request #11527: URL: https://github.com/apache/kafka/pull/11527#discussion_r754683139 ## File path: metadata/src/main/java/org/apache/kafka/image/ProducerIdsImage.java ## @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [kafka] rondagostino commented on pull request #11503: KAFKA-13456: Tighten KRaft config checks/constraints

2021-11-22 Thread GitBox
rondagostino commented on pull request #11503: URL: https://github.com/apache/kafka/pull/11503#issuecomment-975982872 @cmccabe As discussed offline, I added a `CONTROLLER:PLAINTEXT` mapping to `listener.security.protocol.map` for KRaft by default as a convenience. I added tests as well.

[GitHub] [kafka] junrao commented on pull request #11345: Allow empty last segment to have missing offset index during recovery

2021-11-22 Thread GitBox
junrao commented on pull request #11345: URL: https://github.com/apache/kafka/pull/11345#issuecomment-975986830 @ccding : It seems there are compilation errors? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [kafka] jsancio commented on a change in pull request #11457: MINOR: guard against calls to exit in QuorumTestHarness tests

2021-11-22 Thread GitBox
jsancio commented on a change in pull request #11457: URL: https://github.com/apache/kafka/pull/11457#discussion_r754698733 ## File path: core/src/test/scala/integration/kafka/server/QuorumTestHarness.scala ## @@ -163,6 +163,26 @@ abstract class QuorumTestHarness extends Loggin

[GitHub] [kafka] ccding commented on pull request #11345: Allow empty last segment to have missing offset index during recovery

2021-11-22 Thread GitBox
ccding commented on pull request #11345: URL: https://github.com/apache/kafka/pull/11345#issuecomment-975995094 @junrao I accidentally changed the `core/src/main/scala/kafka/utils/Logging.scala` file which made it not compiling. Fixed the compilation issue now. -- This is an automated m

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

2021-11-22 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17447676#comment-17447676 ] Eugen Dück commented on KAFKA-13289: Thanks Matthew for your comments! Too bad Kafk

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

2021-11-22 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17447678#comment-17447678 ] Eugen Dück commented on KAFKA-13289: Matthias To quote from the issue: > left/oute

[GitHub] [kafka] hachikuji commented on a change in pull request #11452: KAFKA-13412; Ensure initTransactions() safe for retry after timeout

2021-11-22 Thread GitBox
hachikuji commented on a change in pull request #11452: URL: https://github.com/apache/kafka/pull/11452#discussion_r754709824 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java ## @@ -964,9 +964,10 @@ private void throwIfProducerClosed() {

[GitHub] [kafka] cmccabe commented on a change in pull request #11457: MINOR: guard against calls to exit in QuorumTestHarness tests

2021-11-22 Thread GitBox
cmccabe commented on a change in pull request #11457: URL: https://github.com/apache/kafka/pull/11457#discussion_r754712726 ## File path: core/src/test/scala/integration/kafka/server/QuorumTestHarness.scala ## @@ -163,6 +163,26 @@ abstract class QuorumTestHarness extends Loggin

[GitHub] [kafka] cmccabe commented on a change in pull request #11527: KAFKA-13357: store producer IDs in broker snapshots

2021-11-22 Thread GitBox
cmccabe commented on a change in pull request #11527: URL: https://github.com/apache/kafka/pull/11527#discussion_r754714034 ## File path: metadata/src/main/java/org/apache/kafka/image/ProducerIdsImage.java ## @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [kafka] cmccabe commented on a change in pull request #11527: KAFKA-13357: store producer IDs in broker snapshots

2021-11-22 Thread GitBox
cmccabe commented on a change in pull request #11527: URL: https://github.com/apache/kafka/pull/11527#discussion_r754714660 ## File path: metadata/src/main/java/org/apache/kafka/image/ProducerIdsImage.java ## @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] [kafka] guozhangwang commented on a change in pull request #11479: KAFKA-12648: Make changing the named topologies blocking

2021-11-22 Thread GitBox
guozhangwang commented on a change in pull request #11479: URL: https://github.com/apache/kafka/pull/11479#discussion_r754719711 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TopologyMetadata.java ## @@ -108,6 +124,44 @@ private void unlock()

[GitHub] [kafka] satishd commented on a change in pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2021-11-22 Thread GitBox
satishd commented on a change in pull request #11390: URL: https://github.com/apache/kafka/pull/11390#discussion_r754749020 ## File path: core/src/main/scala/kafka/log/CleanableIndex.scala ## @@ -0,0 +1,42 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] satishd commented on a change in pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2021-11-22 Thread GitBox
satishd commented on a change in pull request #11390: URL: https://github.com/apache/kafka/pull/11390#discussion_r754749244 ## File path: core/src/main/scala/kafka/server/ReplicaFetcherThread.scala ## @@ -54,12 +56,12 @@ class ReplicaFetcherThread(name: String,

[GitHub] [kafka] mjsax commented on a change in pull request #11455: KAFKA-13423: GlobalThread should not log ERROR on clean shutdown

2021-11-22 Thread GitBox
mjsax commented on a change in pull request #11455: URL: https://github.com/apache/kafka/pull/11455#discussion_r754758851 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStreamThread.java ## @@ -294,7 +294,7 @@ public void run() {

[GitHub] [kafka] jsancio commented on a change in pull request #11457: MINOR: guard against calls to exit in QuorumTestHarness tests

2021-11-22 Thread GitBox
jsancio commented on a change in pull request #11457: URL: https://github.com/apache/kafka/pull/11457#discussion_r754759742 ## File path: core/src/test/scala/integration/kafka/server/QuorumTestHarness.scala ## @@ -163,6 +163,26 @@ abstract class QuorumTestHarness extends Loggin

[GitHub] [kafka] mjsax commented on a change in pull request #11455: KAFKA-13423: GlobalThread should not log ERROR on clean shutdown

2021-11-22 Thread GitBox
mjsax commented on a change in pull request #11455: URL: https://github.com/apache/kafka/pull/11455#discussion_r754759809 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/GlobalKTableIntegrationTest.java ## @@ -337,7 +341,18 @@ public void shouldGetToRu

[GitHub] [kafka] wcarlson5 commented on a change in pull request #11479: KAFKA-12648: Make changing the named topologies blocking

2021-11-22 Thread GitBox
wcarlson5 commented on a change in pull request #11479: URL: https://github.com/apache/kafka/pull/11479#discussion_r754759558 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TopologyMetadata.java ## @@ -108,6 +124,44 @@ private void unlock() {

[GitHub] [kafka] jsancio commented on a change in pull request #11457: MINOR: guard against calls to exit in QuorumTestHarness tests

2021-11-22 Thread GitBox
jsancio commented on a change in pull request #11457: URL: https://github.com/apache/kafka/pull/11457#discussion_r754698733 ## File path: core/src/test/scala/integration/kafka/server/QuorumTestHarness.scala ## @@ -163,6 +163,26 @@ abstract class QuorumTestHarness extends Loggin

[GitHub] [kafka] vvcephei commented on a change in pull request #11481: KAFKA-13117: migrate TupleForwarder and CacheFlushListener to new Record API

2021-11-22 Thread GitBox
vvcephei commented on a change in pull request #11481: URL: https://github.com/apache/kafka/pull/11481#discussion_r754781412 ## File path: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregateProcessorTest.java ## @@ -116,6 +116,7 @@ pr

[GitHub] [kafka] vvcephei merged pull request #11481: KAFKA-13117: migrate TupleForwarder and CacheFlushListener to new Record API

2021-11-22 Thread GitBox
vvcephei merged pull request #11481: URL: https://github.com/apache/kafka/pull/11481 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr

[GitHub] [kafka] vvcephei commented on a change in pull request #11514: Tracking latest seen offset in state stores

2021-11-22 Thread GitBox
vvcephei commented on a change in pull request #11514: URL: https://github.com/apache/kafka/pull/11514#discussion_r754789750 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/InMemoryKeyValueStore.java ## @@ -62,6 +67,12 @@ public void init(final Pro

[GitHub] [kafka] peterwanner commented on pull request #11517: KAFKA-13466:when kafka-console-producer.sh, batch.size defaults is In…

2021-11-22 Thread GitBox
peterwanner commented on pull request #11517: URL: https://github.com/apache/kafka/pull/11517#issuecomment-976189946 > @showuon ok ,and what shoud the lowest version like 2,x which the batch.size is not removed but still have the same error(default error) -- This is an automat

[GitHub] [kafka] showuon commented on pull request #11517: KAFKA-13466:when kafka-console-producer.sh, batch.size defaults is In…

2021-11-22 Thread GitBox
showuon commented on pull request #11517: URL: https://github.com/apache/kafka/pull/11517#issuecomment-976196050 > what shoud the lowest version like 2,x which the batch.size is not removed but still have the same error(default error) Sorry, I don't get your question. What is the er

[GitHub] [kafka] showuon commented on pull request #11408: KAFKA-13374: update doc to allow read from leader/followers

2021-11-22 Thread GitBox
showuon commented on pull request #11408: URL: https://github.com/apache/kafka/pull/11408#issuecomment-976197244 @mumrah @hachikuji @dajac , please take a look at the 1 line change PR. Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [kafka] showuon opened a new pull request #11528: MINOR: update rocksDb memory management doc

2021-11-22 Thread GitBox
showuon opened a new pull request #11528: URL: https://github.com/apache/kafka/pull/11528 In the RocksDb memory management [doc](https://kafka.apache.org/30/documentation/streams/developer-guide/memory-mgmt.html#rocksdb), we mentioned in the footnote that there's a rocksdb bug caused the

[GitHub] [kafka] showuon commented on a change in pull request #11528: MINOR: update rocksDb memory management doc

2021-11-22 Thread GitBox
showuon commented on a change in pull request #11528: URL: https://github.com/apache/kafka/pull/11528#discussion_r754843740 ## File path: docs/streams/developer-guide/memory-mgmt.html ## @@ -195,14 +195,14 @@ RocksDB1. INDEX_FILTER_BLOCK_RATIO can be used to set a fraction of

[GitHub] [kafka] showuon commented on a change in pull request #11528: MINOR: update rocksDb memory management doc

2021-11-22 Thread GitBox
showuon commented on a change in pull request #11528: URL: https://github.com/apache/kafka/pull/11528#discussion_r754844183 ## File path: docs/streams/developer-guide/memory-mgmt.html ## @@ -195,14 +195,14 @@ RocksDB1. INDEX_FILTER_BLOCK_RATIO can be used to set a fraction of

[GitHub] [kafka] showuon commented on pull request #11528: MINOR: update rocksDb memory management doc

2021-11-22 Thread GitBox
showuon commented on pull request #11528: URL: https://github.com/apache/kafka/pull/11528#issuecomment-976213077 @cadonna , please help review. Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [kafka] camilesing edited a comment on pull request #8575: KAFKA-8713 KIP-581 Add accept.optional.null to solve optional null

2021-11-22 Thread GitBox
camilesing edited a comment on pull request #8575: URL: https://github.com/apache/kafka/pull/8575#issuecomment-975447966 Do you have any updates or possible workaround? @rhauch @ijuma @hachikuji @guozhangwang @mjsax @rajinisivaram @cmccabe -- This is an automated message from the A

[GitHub] [kafka] peterwanner commented on pull request #11517: KAFKA-13466:when kafka-console-producer.sh, batch.size defaults is In…

2021-11-22 Thread GitBox
peterwanner commented on pull request #11517: URL: https://github.com/apache/kafka/pull/11517#issuecomment-976224503 > > what shoud the lowest version like 2,x which the batch.size is not removed but still have the same error(default error) > > Sorry, I don't get your question. What