[GitHub] [kafka] dajac commented on a change in pull request #11599: KAFKA-13527: Add top-level error code field to DescribeLogDirsResponse

2022-02-01 Thread GitBox
dajac commented on a change in pull request #11599: URL: https://github.com/apache/kafka/pull/11599#discussion_r796347687 ## File path: core/src/test/scala/integration/kafka/api/AuthorizerIntegrationTest.scala ## @@ -223,7 +223,7 @@ class AuthorizerIntegrationTest extends Base

[GitHub] [kafka] dajac commented on a change in pull request #11695: KAFKA-13595: Allow producing records with null fields in ConsoleProducer

2022-02-01 Thread GitBox
dajac commented on a change in pull request #11695: URL: https://github.com/apache/kafka/pull/11695#discussion_r796350195 ## File path: core/src/main/scala/kafka/tools/ConsoleProducer.scala ## @@ -362,11 +374,22 @@ object ConsoleProducer { } private def splitHeaders

[GitHub] [kafka] showuon commented on pull request #11691: KAFKA-13598: enable idempotence producer by default and validate the configs

2022-02-01 Thread GitBox
showuon commented on pull request #11691: URL: https://github.com/apache/kafka/pull/11691#issuecomment-1026588225 @hachikuji , thanks for your comments. I'll update it after Chinese new year holidays. And I agree with @ijuma to fix in v3.0.1, v3.1.1, because this feature was origina

[jira] [Commented] (KAFKA-9542) ZSTD Compression Not Working

2022-02-01 Thread kaushik srinivas (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485114#comment-17485114 ] kaushik srinivas commented on KAFKA-9542: - [~lucasbradstreet]  We see this issue

[GitHub] [kafka] dajac commented on a change in pull request #11725: KAFKA-13221; Implement `PartitionsWithLateTransactionsCount` metric

2022-02-01 Thread GitBox
dajac commented on a change in pull request #11725: URL: https://github.com/apache/kafka/pull/11725#discussion_r796391527 ## File path: core/src/main/scala/kafka/log/ProducerStateManager.scala ## @@ -498,12 +499,23 @@ class ProducerStateManager(val topicPartition: TopicPartiti

[GitHub] [kafka] dajac merged pull request #11723: MINOR: replace statement lambda with expression lambda

2022-02-01 Thread GitBox
dajac merged pull request #11723: URL: https://github.com/apache/kafka/pull/11723 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...

[GitHub] [kafka] dajac merged pull request #11547: fixed typo "Exsiting" -> "Existing"

2022-02-01 Thread GitBox
dajac merged pull request #11547: URL: https://github.com/apache/kafka/pull/11547 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...

[jira] [Commented] (KAFKA-13633) Merging multiple KStreams in one operation

2022-02-01 Thread Nicholas Telford (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485163#comment-17485163 ] Nicholas Telford commented on KAFKA-13633: -- I actually did create a KIP, I just

[jira] [Updated] (KAFKA-13633) Merging multiple KStreams in one operation

2022-02-01 Thread Nicholas Telford (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Telford updated KAFKA-13633: - Labels: (was: needs-kip) > Merging multiple KStreams in one operation > --

[GitHub] [kafka] satishd commented on a change in pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2022-02-01 Thread GitBox
satishd commented on a change in pull request #11390: URL: https://github.com/apache/kafka/pull/11390#discussion_r796465473 ## File path: clients/src/main/java/org/apache/kafka/common/record/RemoteLogInputStream.java ## @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] satishd commented on a change in pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2022-02-01 Thread GitBox
satishd commented on a change in pull request #11390: URL: https://github.com/apache/kafka/pull/11390#discussion_r796479255 ## File path: clients/src/main/java/org/apache/kafka/common/record/RemoteLogInputStream.java ## @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] Indupa commented on pull request #7898: KAFKA-9366: Change log4j dependency into log4j2

2022-02-01 Thread GitBox
Indupa commented on pull request #7898: URL: https://github.com/apache/kafka/pull/7898#issuecomment-1026731878 Hi @dongjinleekr , Can you please help me to build this latest patch you have released on top of Kafka_2.8,1. I tried using this following command to compile kafka source c

[GitHub] [kafka] mimaison commented on pull request #11695: KAFKA-13595: Allow producing records with null fields in ConsoleProducer

2022-02-01 Thread GitBox
mimaison commented on pull request #11695: URL: https://github.com/apache/kafka/pull/11695#issuecomment-1026794372 Thanks @dajac for the review! I've pushed an update. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[jira] [Created] (KAFKA-13635) Make Consumer Group Protocol resilient to disk issues with __consumer_offsets

2022-02-01 Thread Akhilesh Dubey (Jira)
Akhilesh Dubey created KAFKA-13635: -- Summary: Make Consumer Group Protocol resilient to disk issues with __consumer_offsets Key: KAFKA-13635 URL: https://issues.apache.org/jira/browse/KAFKA-13635 Pr

[jira] [Updated] (KAFKA-13635) Make Consumer Group Protocol resilient to disk issues with __consumer_offsets

2022-02-01 Thread Akhilesh Dubey (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akhilesh Dubey updated KAFKA-13635: --- Affects Version/s: 2.7.1 > Make Consumer Group Protocol resilient to disk issues with __cons

[jira] [Updated] (KAFKA-13635) Make Consumer Group Protocol resilient to disk issues with __consumer_offsets

2022-02-01 Thread Akhilesh Dubey (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akhilesh Dubey updated KAFKA-13635: --- Description: While working with 2.7.1, we experienced offset reset on some consumer groups

[GitHub] [kafka] cadonna commented on pull request #11690: KAFKA-13599: Upgrade RocksDB to 6.27.3

2022-02-01 Thread GitBox
cadonna commented on pull request #11690: URL: https://github.com/apache/kafka/pull/11690#issuecomment-1026894087 I re-triggered the builds. Once the builds are acceptable, I will merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Commented] (KAFKA-13600) Rebalances while streams is in degraded state can cause stores to be reassigned and restore from scratch

2022-02-01 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485343#comment-17485343 ] Bruno Cadonna commented on KAFKA-13600: --- [~tim.patterson] To come up with a perfec

[GitHub] [kafka] mumrah closed pull request #11550: KRaft to KRaft upgrades

2022-02-01 Thread GitBox
mumrah closed pull request #11550: URL: https://github.com/apache/kafka/pull/11550 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr..

[GitHub] [kafka] mumrah commented on pull request #11550: KRaft to KRaft upgrades

2022-02-01 Thread GitBox
mumrah commented on pull request #11550: URL: https://github.com/apache/kafka/pull/11550#issuecomment-1027091285 Closed in favor of #11677 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [kafka] hachikuji commented on a change in pull request #11725: KAFKA-13221; Implement `PartitionsWithLateTransactionsCount` metric

2022-02-01 Thread GitBox
hachikuji commented on a change in pull request #11725: URL: https://github.com/apache/kafka/pull/11725#discussion_r796828017 ## File path: core/src/main/scala/kafka/log/ProducerStateManager.scala ## @@ -498,12 +499,23 @@ class ProducerStateManager(val topicPartition: TopicPar

[GitHub] [kafka] Indupa edited a comment on pull request #7898: KAFKA-9366: Change log4j dependency into log4j2

2022-02-01 Thread GitBox
Indupa edited a comment on pull request #7898: URL: https://github.com/apache/kafka/pull/7898#issuecomment-1026731878 Hi @dongjinleekr , Can you please help me to build this latest patch you have released on top of Kafka_2.8,1. I tried using this following command to compile kafka so

[GitHub] [kafka] mumrah commented on a change in pull request #11677: KAFKA-13410 Add KRaft metadata.version for KIP-778

2022-02-01 Thread GitBox
mumrah commented on a change in pull request #11677: URL: https://github.com/apache/kafka/pull/11677#discussion_r796836599 ## File path: core/src/main/scala/kafka/server/metadata/BrokerMetadataListener.scala ## @@ -73,7 +74,7 @@ class BrokerMetadataListener( /** * The c

[GitHub] [kafka] mumrah commented on a change in pull request #11677: KAFKA-13410 Add KRaft metadata.version for KIP-778

2022-02-01 Thread GitBox
mumrah commented on a change in pull request #11677: URL: https://github.com/apache/kafka/pull/11677#discussion_r796836599 ## File path: core/src/main/scala/kafka/server/metadata/BrokerMetadataListener.scala ## @@ -73,7 +74,7 @@ class BrokerMetadataListener( /** * The c

[GitHub] [kafka] mumrah commented on a change in pull request #11677: KAFKA-13410 Add KRaft metadata.version for KIP-778

2022-02-01 Thread GitBox
mumrah commented on a change in pull request #11677: URL: https://github.com/apache/kafka/pull/11677#discussion_r796852603 ## File path: core/src/main/scala/kafka/server/metadata/MetadataVersionManager.scala ## @@ -0,0 +1,104 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [kafka] mimaison merged pull request #11599: KAFKA-13527: Add top-level error code field to DescribeLogDirsResponse

2022-02-01 Thread GitBox
mimaison merged pull request #11599: URL: https://github.com/apache/kafka/pull/11599 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr

[GitHub] [kafka] mumrah commented on a change in pull request #11677: KAFKA-13410 Add KRaft metadata.version for KIP-778

2022-02-01 Thread GitBox
mumrah commented on a change in pull request #11677: URL: https://github.com/apache/kafka/pull/11677#discussion_r796852603 ## File path: core/src/main/scala/kafka/server/metadata/MetadataVersionManager.scala ## @@ -0,0 +1,104 @@ +/** + * Licensed to the Apache Software Foundat

[GitHub] [kafka] mimaison merged pull request #11695: KAFKA-13595: Allow producing records with null fields in ConsoleProducer

2022-02-01 Thread GitBox
mimaison merged pull request #11695: URL: https://github.com/apache/kafka/pull/11695 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr

[jira] [Resolved] (KAFKA-13527) Add top-level error code field to DescribeLogDirsResponse

2022-02-01 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-13527. Fix Version/s: 3.2.0 Resolution: Fixed > Add top-level error code field to Describe

[jira] [Resolved] (KAFKA-13595) Allow producing records with null values in Kafka Console Producer

2022-02-01 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-13595. Fix Version/s: 3.2.0 Resolution: Fixed > Allow producing records with null values i

[GitHub] [kafka] dajac commented on a change in pull request #11725: KAFKA-13221; Implement `PartitionsWithLateTransactionsCount` metric

2022-02-01 Thread GitBox
dajac commented on a change in pull request #11725: URL: https://github.com/apache/kafka/pull/11725#discussion_r796878885 ## File path: core/src/main/scala/kafka/log/ProducerStateManager.scala ## @@ -498,12 +499,23 @@ class ProducerStateManager(val topicPartition: TopicPartiti

[jira] [Created] (KAFKA-13636) Committed offsets could be deleted during a rebalance if a group did not commit for a while

2022-02-01 Thread Damien Gasparina (Jira)
Damien Gasparina created KAFKA-13636: Summary: Committed offsets could be deleted during a rebalance if a group did not commit for a while Key: KAFKA-13636 URL: https://issues.apache.org/jira/browse/KAFKA-1363

[GitHub] [kafka] mimaison commented on pull request #11672: KAFKA-13577: Replace easymock with mockito in kafka:core - part 1

2022-02-01 Thread GitBox
mimaison commented on pull request #11672: URL: https://github.com/apache/kafka/pull/11672#issuecomment-1027218635 @tombentley @dajac Can you take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Assigned] (KAFKA-13636) Committed offsets could be deleted during a rebalance if a group did not commit for a while

2022-02-01 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot reassigned KAFKA-13636: --- Assignee: David Jacot > Committed offsets could be deleted during a rebalance if a group di

[jira] [Commented] (KAFKA-12879) Compatibility break in Admin.listOffsets()

2022-02-01 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485491#comment-17485491 ] Ismael Juma commented on KAFKA-12879: - My suggestion is to do 3 and to make sure the

[GitHub] [kafka] ijuma commented on pull request #10951: KAFKA-12841: NPE from the provided metadata in client callback in case of ApiException

2022-02-01 Thread GitBox
ijuma commented on pull request #10951: URL: https://github.com/apache/kafka/pull/10951#issuecomment-1027303692 @kirktrue Given how it works for the producer interceptor, it seems reasonable to do something similar for the consumer. @cmccabe do you agree? -- This is an automated message

[jira] [Commented] (KAFKA-12879) Compatibility break in Admin.listOffsets()

2022-02-01 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485496#comment-17485496 ] Colin McCabe commented on KAFKA-12879: -- Let me give a little context here on the be

[jira] [Commented] (KAFKA-12879) Compatibility break in Admin.listOffsets()

2022-02-01 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485498#comment-17485498 ] Colin McCabe commented on KAFKA-12879: -- bq. My suggestion is to do 3 and to make su

[GitHub] [kafka] mjsax commented on pull request #11424: KAFKA-13152: Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-02-01 Thread GitBox
mjsax commented on pull request #11424: URL: https://github.com/apache/kafka/pull/11424#issuecomment-1027334009 Seems this PR breaks backward compatibility. Will need to revert it for now. Sorry. @vamossagar12 -- Can you do a new PR containing the commit plus the required fixes? Hap

[jira] [Reopened] (KAFKA-13152) Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-02-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-13152: - The PR broke backward compatibility. Needed to revert the commit. Sorry about that. Lets d

[jira] [Commented] (KAFKA-12879) Compatibility break in Admin.listOffsets()

2022-02-01 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485514#comment-17485514 ] Ismael Juma commented on KAFKA-12879: - [~cmccabe] It's straightforward to do `descri

[GitHub] [kafka] mjsax commented on pull request #11424: KAFKA-13152: Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-02-01 Thread GitBox
mjsax commented on pull request #11424: URL: https://github.com/apache/kafka/pull/11424#issuecomment-1027357002 Btw: this PR does not update the docs -- we should docs updates, too. (Also ok to do in a follow up PR.) To ensure backward compatibility, it might also be good to split th

[GitHub] [kafka] kirktrue closed pull request #10951: KAFKA-12841: NPE from the provided metadata in client callback in case of ApiException

2022-02-01 Thread GitBox
kirktrue closed pull request #10951: URL: https://github.com/apache/kafka/pull/10951 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr

[GitHub] [kafka] kirktrue commented on pull request #10951: KAFKA-12841: NPE from the provided metadata in client callback in case of ApiException

2022-02-01 Thread GitBox
kirktrue commented on pull request #10951: URL: https://github.com/apache/kafka/pull/10951#issuecomment-1027464655 Closing this PR in favor of #11689. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [kafka] philipnee commented on pull request #10951: KAFKA-12841: NPE from the provided metadata in client callback in case of ApiException

2022-02-01 Thread GitBox
philipnee commented on pull request #10951: URL: https://github.com/apache/kafka/pull/10951#issuecomment-1027466090 > @kirktrue Given how it works for the producer interceptor, it seems reasonable to do something similar for the consumer. @cmccabe do you agree? let's move the convers

[GitHub] [kafka] artemlivshits commented on a change in pull request #11721: KAFKA-13629: use faster algorithm for ByteUtils sizeOfXxx algorithm

2022-02-01 Thread GitBox
artemlivshits commented on a change in pull request #11721: URL: https://github.com/apache/kafka/pull/11721#discussion_r797190190 ## File path: clients/src/main/java/org/apache/kafka/common/utils/ByteUtils.java ## @@ -386,18 +386,39 @@ public static void writeDouble(double valu

[GitHub] [kafka] hachikuji commented on a change in pull request #11725: KAFKA-13221; Implement `PartitionsWithLateTransactionsCount` metric

2022-02-01 Thread GitBox
hachikuji commented on a change in pull request #11725: URL: https://github.com/apache/kafka/pull/11725#discussion_r797197548 ## File path: core/src/main/scala/kafka/log/ProducerStateManager.scala ## @@ -498,12 +499,23 @@ class ProducerStateManager(val topicPartition: TopicPar

[GitHub] [kafka] hachikuji commented on a change in pull request #11725: KAFKA-13221; Implement `PartitionsWithLateTransactionsCount` metric

2022-02-01 Thread GitBox
hachikuji commented on a change in pull request #11725: URL: https://github.com/apache/kafka/pull/11725#discussion_r797197548 ## File path: core/src/main/scala/kafka/log/ProducerStateManager.scala ## @@ -498,12 +499,23 @@ class ProducerStateManager(val topicPartition: TopicPar

[GitHub] [kafka] jasonk000 commented on a change in pull request #11721: KAFKA-13629: use faster algorithm for ByteUtils sizeOfXxx algorithm

2022-02-01 Thread GitBox
jasonk000 commented on a change in pull request #11721: URL: https://github.com/apache/kafka/pull/11721#discussion_r797211462 ## File path: clients/src/main/java/org/apache/kafka/common/utils/ByteUtils.java ## @@ -386,18 +386,39 @@ public static void writeDouble(double value, B

[GitHub] [kafka] jasonk000 commented on a change in pull request #11721: KAFKA-13629: use faster algorithm for ByteUtils sizeOfXxx algorithm

2022-02-01 Thread GitBox
jasonk000 commented on a change in pull request #11721: URL: https://github.com/apache/kafka/pull/11721#discussion_r797211462 ## File path: clients/src/main/java/org/apache/kafka/common/utils/ByteUtils.java ## @@ -386,18 +386,39 @@ public static void writeDouble(double value, B

[jira] [Created] (KAFKA-13637) User default.api.timeout.ms config as default timeout for KafkaConsumer.endOffsets

2022-02-01 Thread dengziming (Jira)
dengziming created KAFKA-13637: -- Summary: User default.api.timeout.ms config as default timeout for KafkaConsumer.endOffsets Key: KAFKA-13637 URL: https://issues.apache.org/jira/browse/KAFKA-13637 Projec

[GitHub] [kafka] dengziming opened a new pull request #11726: KAFKA-13637: Use default.api.timeout.ms as default timeout value for KafkaConsumer.endOffsets

2022-02-01 Thread GitBox
dengziming opened a new pull request #11726: URL: https://github.com/apache/kafka/pull/11726 *More detailed description of your change* In KafkaConsumer, we use `request.timeout.ms` in `endOffsets` and `default.api.timeout.ms` when in `beginningOffsets`, we should use `default.api.timeo

[GitHub] [kafka] satishd commented on a change in pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2022-02-01 Thread GitBox
satishd commented on a change in pull request #11390: URL: https://github.com/apache/kafka/pull/11390#discussion_r784750507 ## File path: core/src/main/scala/kafka/log/UnifiedLog.scala ## @@ -288,13 +294,24 @@ class UnifiedLog(@volatile var logStartOffset: Long, @volatile