[jira] [Created] (KAFKA-17125) Streams Rebalance Protocol

Lucas Brutschy created KAFKA-17125: -- Summary: Streams Rebalance Protocol Key: KAFKA-17125 URL: https://issues.apache.org/jira/browse/KAFKA-17125 Project: Kafka Issue Type: Task Co

[DISCUSS] KIP-1071: Streams Rebalance Protocol

Hi all, I would like to start a discussion thread on KIP-1071: Streams Rebalance Protocol. With this KIP, we aim to bring the principles laid down by KIP-848 to Kafka Streams, to make rebalances more reliable and scalable, and make Kafka Streams overall easier to deploy and operate. The KIP propos

Re: [VOTE] KIP-1070: Deprecate MockProcessorContext

Sounds good to me! +1 (binding) On Fri, Jul 12, 2024 at 12:55 AM Bill Bejeck wrote: > > +1 (binding) > > On Thu, Jul 11, 2024 at 5:07 PM Sophie Blee-Goldman > wrote: > > > Makes sense to me, +1 (binding) > > > > On Thu, Jul 11, 2024 at 9:24 AM Matthias J. Sax wrote: > > > > > Hi, > > > > > > I

[jira] [Created] (KAFKA-17126) hande piggybacked acknowledgements asynchronously in Share Fetch and properly combine the results of fetch and acknowledge

Chirag Wadhwa created KAFKA-17126: - Summary: hande piggybacked acknowledgements asynchronously in Share Fetch and properly combine the results of fetch and acknowledge Key: KAFKA-17126 URL: https://issues.apache.o

[jira] [Created] (KAFKA-17127) Add check for whether the requested partition exists or not in Share Fetch request

Chirag Wadhwa created KAFKA-17127: - Summary: Add check for whether the requested partition exists or not in Share Fetch request Key: KAFKA-17127 URL: https://issues.apache.org/jira/browse/KAFKA-17127

[VOTE] 3.8.0 RC0

Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 3.8.0. Some of the major features included in this release are: * KIP-1028: Docker Official Image for Apache Kafka * KIP-974: Docker Image for GraalVM based Native Kafka Broker * KIP-1036:

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #3099

See

Re: [DISCUSS] KIP-1066: Mechanism to cordon brokers and log directories

Hi Kamal, Thanks for taking a look at the KIP! I briefly considered that option initially but I found it not very practical once you have more than a few cordoned log directories. I find your example is already not very easy to read, and it only has 2 entries. Also if the configuration is at the

Re: [DISCUSS] KIP-1062: Introduce Pagination for some requests used by Admin API

Hey Omnia, thanks for the KIP! I think this will be a really nice improvement for operators. DA1: In "Public Interfaces" you say "max.request.pagination.size.limit" controls the max items to return by default. It's not clear to me if this is just a default, or if it is a hard limit. In KIP-966, th

Re: [DISCUSS] Apache Kafka 3.9.0 release

> oh, sorry that there are 2 binding votes :( updated: KIP-1067 (https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=311627623) has 3 binding votes now. The PR is ready (https://github.com/apache/kafka/pull/16572) Hence, could KIP-1067 be included in 3.9.0? thanks! Best, Chia

[jira] [Resolved] (KAFKA-17096) Fix kafka_log4j_appender.py

[ https://issues.apache.org/jira/browse/KAFKA-17096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17096. Fix Version/s: 3.9.0 Resolution: Fixed > Fix kafka_log4j_appender.py >

[jira] [Created] (KAFKA-17128) Make node.id immutable after removing Zookeeper

TengYao Chi created KAFKA-17128: --- Summary: Make node.id immutable after removing Zookeeper Key: KAFKA-17128 URL: https://issues.apache.org/jira/browse/KAFKA-17128 Project: Kafka Issue Type: Imp

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

Can we update the KIP to clearly document these decisions? Thanks, Justine On Tue, Jul 9, 2024 at 9:25 AM Andrew Schofield wrote: > Hi Chris, > As it stands, the error handling for transactions in KafkaProducer is not > ideal. There’s no reason why a failed operation should fail a transaction

Re: [DISCUSS] KAFKA-17094: How should unregistered broker nodes be handled in KRaft quorum state?

Hi Igor, I think getting the list of all registered brokers is an administrator requirement. Typically end users are not interested in offline brokers. As highlighted in the Jira, the main goal for retrieving this information is to get broker Ids to unregister. On the other hand, I see the Describ

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

Hi Brenden, Thanks for the KIP. It fills a gap in the metrics for the new consumer nicely. AS1. If using the CLASSIC protocol, and thus the LegacyKafkaConsumer, I would expect that the metrics do not exist at all. Maybe say something like “These metrics are for the new consumer implementation usin

[jira] [Created] (KAFKA-17129) Revisit FindCoordinatorResponse in KafkaConsumerTest

PoAn Yang created KAFKA-17129: - Summary: Revisit FindCoordinatorResponse in KafkaConsumerTest Key: KAFKA-17129 URL: https://issues.apache.org/jira/browse/KAFKA-17129 Project: Kafka Issue Type: Su

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

Hi Andrew, Thank you for the feedback and your question. AS1. Great idea, I will get that added. AS2. For unsent-events-age-max, age will be calculated once the event is sent, so you are correct. AS3. I agree, I think that would be a helpful metric to add, thank you! I will get that added. Plea

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

Hey Brenden, thanks for the KIP! These will be great to observe and debug the background thread of the new consumer. LB1. `time-between-network-thread-poll-max` → I see several similar metrics including the unit in the metric name (ms or us). We could consider this, although it's probably not str

Re: [DISCUSS] KIP-1066: Mechanism to cordon brokers and log directories

Mickael, thanks for the KIP! I think this could be quite a useful feature. DA1: Having to know each of the log dirs for a broker seems a bit inconvenient for cases where we want to cordon off a whole broker. I do think having the ability to cordon off a specific log dir is useful for JBOD, but I i

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

Hey Brenden, thanks for the KIP! Great to get more visibility into the new consumer. LM1. +1 on Lucas's suggestion for including the unit in the name, seems clearer and consistent (I do see several time metrics including ms) LM2. What about a new metric for application-event-queue-time-ms. It wou

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

Hi Lucas, Thank you for the feedback! I have addressed your comments: LB1. Good catch there, I will update the names as needed. LB2. Good catch again! I will update the name to be more consistent. LB3. Thank you for pointing this out, I realized that all metric values will actually be set to

Re: [DISCUSS] KIP-1068: KIP-1068: New JMX Metrics for AsyncKafkaConsumer

Hey Lianet, Thank you for your suggestions and feedback! LM1. This has now been addressed. LM2. I think that would be a valuable addition to the current set of metrics, I will get that added. LM3. Again great idea, that would certainly be helpful. Will add that as well. Let me know if you

[jira] [Created] (KAFKA-17130) Connect workers do not properly ensure group membership before responding to health checks

Chris Egerton created KAFKA-17130: - Summary: Connect workers do not properly ensure group membership before responding to health checks Key: KAFKA-17130 URL: https://issues.apache.org/jira/browse/KAFKA-17130

Re: [DISCUSS] KIP-1059: Enable the Producer flush() method to clear the latest send() error

Hi all, Alieh, thanks for the KIP! And everyone else, thanks for the robust discussion. I understand that there are situations in which users desire that the pipeline "just keep working" and skip errors. However, I question whether it is appropriate to support/encourage this behavior via inclusio

Re: [PR] Fix update email notifications [kafka-merge-queue-sandbox]

mumrah merged PR #28: URL: https://github.com/apache/kafka-merge-queue-sandbox/pull/28 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@k

Re: [VOTE] KIP-1070: Deprecate MockProcessorContext

I just realized, that there is more interfaces with a similar situation: - Transformer - TransformerSupplier - ValueTransformer - ValueTransfomerSupplier - ValueTransformerWithKey - ValueTransfromerWithKeySupplier Given that `KStream#transform` and `KStream#transformValues` are deprecated, it s

Re: [DISCUSS] KIP-1043: Administration of groups

Hey Andrew, thanks for the KIP, we definitely need visibility from a higher level now that groups are growing. LM1. Should we have the existing org.apache.kafka.clients.admin.ConsumerGroupListing extend the GroupListing you’re proposing? ConsumerGroupListing already exists with a very similar shap

[jira] [Created] (KAFKA-17131) Cleanup `ProcessorContext` an related interfaces

Matthias J. Sax created KAFKA-17131: --- Summary: Cleanup `ProcessorContext` an related interfaces Key: KAFKA-17131 URL: https://issues.apache.org/jira/browse/KAFKA-17131 Project: Kafka Issue

[jira] [Created] (KAFKA-17132) Revisit testMissingOffsetNoResetPolicy for AsyncConsumer

Chia-Ping Tsai created KAFKA-17132: -- Summary: Revisit testMissingOffsetNoResetPolicy for AsyncConsumer Key: KAFKA-17132 URL: https://issues.apache.org/jira/browse/KAFKA-17132 Project: Kafka

[jira] [Created] (KAFKA-17133) add unit test to make sure `ConsumerRecords#recoreds` returns immutable object

Chia-Ping Tsai created KAFKA-17133: -- Summary: add unit test to make sure `ConsumerRecords#recoreds` returns immutable object Key: KAFKA-17133 URL: https://issues.apache.org/jira/browse/KAFKA-17133 P

Re: [DISCUSS] KIP-1065: Add "retry" return-option to ProductionExceptionHandler

After talking to a few people in-person, it seems there was concerns that KAFKA-16508 should be considered a backward incompatible change. While this KIP should address these incompatibility issue, the suggestion was to keep KAFKA-16508 in `trunk` for now, but revert it in the 3.9 branch after

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #3101

See

Proposal to Include Record Headers in ProducerInterceptor onAcknowledgement Method

Dear Kafka Development Community, I propose enhancing the Kafka ProducerInterceptor interface, specifically to make Record Headers available in the onAcknowledgement method. I would appreciate your feedback on this proposal. If the feedback is positive, I will follow up with a detailed discussion