Re: [DISCUSS] Apache Kafka 4.0.0 release

Hi PoAn/Ziming, Thanks for adding those to the release plan! Best, David On Tue, Nov 26, 2024 at 4:25 AM Ziming Deng wrote: > > Hi David & Luke, > > Thanks for driving the release, let’s add KIP-1011 to the release plan which > should have beed released in 3.7.0, after postpone doing it we can

Re: [DISCUSS] Apache Kafka 4.0.0 release

Hi David & Luke, Thanks for driving the release, let’s add KIP-1011 to the release plan which should have beed released in 3.7.0, after postpone doing it we can directly remove deprecated alterConfigs from ConfgCommand to avoid making our code dirty. --, Ziming > On Nov 20, 2024, at 18:04, Da

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.9 #119

See Changes: -- [...truncated 549530 lines...] [2024-11-26T02:29:05.077Z] [2024-11-26T02:29:05.077Z] Gradle Test Run :core:test > Gradle Test Executor 72 > ZkMigrationClient

[jira] [Resolved] (KAFKA-17953) Docker image fails when setting env vars

[ https://issues.apache.org/jira/browse/KAFKA-17953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-17953. --- Resolution: Not A Problem Improving doc in this PR: https://github.com/apache/kafka/pull/17894 > Do

Re: [VOTE] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

Hi everyone, Thanks for your voting. The KIP passes with 4 +1 (binding) votes from Chia-Ping Tsai, David Jacot, Lianet M., Jeff Kim and 1 +1 (non-binding) from Andrew Schofield. Thank you. Best, PoAn > On Nov 26, 2024, at 12:39 AM, Jeff Kim wrote: > > Thanks! > > +1 (binding) > > Jeff > >

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.8 #116

See

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.7 #212

See

Re: [DISCUSS] Apache Kafka 4.0.0 release

Hi David, Based on your comment on KIP-1099. I’ve added KIP-1099 to the release plan. Thanks, PoAn > On Nov 20, 2024, at 8:11 PM, Divij Vaidya wrote: > > Based on initial feedback from the community, I don't anticipate > the

Re: Optimize KafkaProducer Performance

Hi, Generally, max production throughput will be calculated as follows: - max_requests_per_sec = 1000 / request_latency_ms * 5 (max.in.flight.requests) - max_throughput = max_requests_per_sec * messages-per-request So I suggest identifying which is the bottleneck first, req-latency or too-small-

[jira] [Resolved] (KAFKA-17898) Seperate Epoch Bump Scenarios and Error Handling in TV2

[ https://issues.apache.org/jira/browse/KAFKA-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-17898. Resolution: Fixed > Seperate Epoch Bump Scenarios and Error Handling in TV2 >

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.7 #211

See

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.9 #118

See Changes: -- [...truncated 3792 lines...] [2024-11-25T20:44:00.593Z] > Task :connect:mirror:spotbugsTest SKIPPED [2024-11-25T20:44:03.458Z] > Task :tools:compileTestJava [2

[jira] [Created] (KAFKA-18088) Add consumer metric about paused partitions

Matthias J. Sax created KAFKA-18088: --- Summary: Add consumer metric about paused partitions Key: KAFKA-18088 URL: https://issues.apache.org/jira/browse/KAFKA-18088 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-1030: Change constraints and default values for various configurations

Hi, Divij, Thanks for the reply. A few more comments. 10. num.recovery.threads.per.data.dir still seems to depend on the number of cores. 11. Some of the configs on the server side exist at different levels (static, broker, topic, etc) with slightly different names. It would be useful to be clea

[jira] [Created] (KAFKA-18087) Revert unnecessary integration testing of Consumers using both group protocols

Kirk True created KAFKA-18087: - Summary: Revert unnecessary integration testing of Consumers using both group protocols Key: KAFKA-18087 URL: https://issues.apache.org/jira/browse/KAFKA-18087 Project: Kaf

[jira] [Resolved] (KAFKA-18085) Abort inflight requests when rebootstrapping due to timeout or error

[ https://issues.apache.org/jira/browse/KAFKA-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-18085. Reviewer: Manikumar Resolution: Fixed > Abort inflight requests when rebootstrapping

[jira] [Resolved] (KAFKA-18011) Remove ZooKeeper from the docs

[ https://issues.apache.org/jira/browse/KAFKA-18011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18011. Fix Version/s: 4.0.0 Resolution: Fixed > Remove ZooKeeper from the docs > -

Re: [VOTE] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

Thanks! +1 (binding) Jeff On 2024/11/25 15:46:04 "Lianet M." wrote: > Thanks for the KIP and updates PoAn! > > +1 (binding) > > Lianet > > On Mon, Nov 25, 2024, 10:43 a.m. Andrew Schofield < > andrew_schofield_j...@outlook.com> wrote: > > > +1 (non-binding) > > > > Thanks, > > Andrew > > > >

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.8 #115

See

Re: [VOTE] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

Thanks for the KIP and updates PoAn! +1 (binding) Lianet On Mon, Nov 25, 2024, 10:43 a.m. Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: > +1 (non-binding) > > Thanks, > Andrew > > > From: David Jacot > Sent: 25 November 2024 15:38 > To: d

Re: [VOTE] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

+1 (non-binding) Thanks, Andrew From: David Jacot Sent: 25 November 2024 15:38 To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group +1 (binding) In gmail, the discuss and t

[jira] [Created] (KAFKA-18086) Enable propagation of the error message when writing state

Chia-Ping Tsai created KAFKA-18086: -- Summary: Enable propagation of the error message when writing state Key: KAFKA-18086 URL: https://issues.apache.org/jira/browse/KAFKA-18086 Project: Kafka

Re: [DISCUSS] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

Hi David, Yes, it would be an option to have a ShareMemberDescription. Because ShareGroupDescription is still an evolving interface, it's acceptable to change its definition in a minor release, so no rush to get this done in the next 2 weeks. I suggest that I change KIP-932 to introduce ShareMembe

Re: [VOTE] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

+1 (binding) In gmail, the discuss and the vote threads got merged somehow. On Wed, Nov 20, 2024 at 4:47 AM Chia-Ping Tsai wrote: > +1 (binding) > > On 2024/11/18 14:52:22 PoAn Yang wrote: > > Hi All, > > > > I would like to start vote on KIP-1099: > > > https://cwiki.apache.org/confluence/disp

Re: [DISCUSS] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

Thanks, LGTM. On Mon, Nov 25, 2024 at 4:26 PM PoAn Yang wrote: > Hi David, > > Thanks for the feedback. > > Yes, I agree that the “UPGRADED” column name is better. I updated the KIP. > > Best, > PoAn > > > On Nov 25, 2024, at 11:11 PM, PoAn Yang wrote: > > > > Hi Andrew, > > > > Thanks for the

Re: [DISCUSS] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

Hi Andrew, Thanks for the feedback. AS4: Change TARGET-ASSIGNMENT-EPOCH to ASSIGNMENT-EPOCH for “—describe —state —verbose" in kafka-share-groups.sh. AS5: Remove TARGET-EPOCH and TARGET-ASSIGNMENT. Change CURRENT-EPOCH to MEMBER-EPOCH and CURRENT-ASSIGNMENT to ASSIGNMENT for “—describe —member

Re: [DISCUSS] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

Hi David, Thanks for the feedback. Yes, I agree that the “UPGRADED” column name is better. I updated the KIP. Best, PoAn > On Nov 25, 2024, at 11:11 PM, PoAn Yang wrote: > > Hi Andrew, > > Thanks for the feedback. > > AS4: Change TARGET-ASSIGNMENT-EPOCH to ASSIGNMENT-EPOCH for “—describe —s

Re: [DISCUSS] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

Hi PoAn, Thanks for the update. Regarding the CLI, I wonder whether we should rather have a column called "Upgraded" which is only displayed when some members use the classic protocol. The displayed value would be True for consumer members and False for classic members. I think that this may be m

Re: [DISCUSS] KIP-1099: Extend kafka-consumer-groups command line tool to support new consumer group

Hi PoAn, I've taken another pass over the KIP and have some more comments around the share group support. AS4: Share groups do not have the concept of a target assignment. As a result, the verbose state TARGET-ASSIGNMENT-EPOCH is a bit misleading from my point of view. ASSIGNMENT-EPOCH is better.

[jira] [Resolved] (KAFKA-18077) Remove deprecated JmxReporter(String)

[ https://issues.apache.org/jira/browse/KAFKA-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18077. Fix Version/s: 4.0.0 Resolution: Fixed > Remove deprecated JmxReporter(String) > --

Optimize KafkaProducer Performance

Hi Team, I am currently pushing 50,000 records to a Kafka producer, but the process is taking approximately 5 minutes to complete. Could you please assist me in optimizing the Kafka producer configurations to reduce the processing time? Any recommendations or best practices would be greatly appre

[jira] [Resolved] (KAFKA-18045) Add 0.11, 1.0, 1.1, and 2.0 back to streams_upgrade_test.py

[ https://issues.apache.org/jira/browse/KAFKA-18045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18045. Fix Version/s: 4.0.0 Resolution: Fixed > Add 0.11, 1.0, 1.1, and 2.0 back to stream

[jira] [Created] (KAFKA-18085) Abort inflight requests when rebootstrapping due to timeout or error

Rajini Sivaram created KAFKA-18085: -- Summary: Abort inflight requests when rebootstrapping due to timeout or error Key: KAFKA-18085 URL: https://issues.apache.org/jira/browse/KAFKA-18085 Project: Kaf

Re: [DISCUSS] KIP-1098: Reverse Checkpointing in MirrorMaker

Hi VIktor, SVV3. I think that ReplicationPolicy is all about naming topics, and not about what should be included in the replication flow - the *Filter plugins have the responsibility to decide about including/excluding resources. Additionally, pushing this responsibility to the policy would lead

[jira] [Created] (KAFKA-18084) Null and leaked AcquisitionLockTimerTask causes hanging AcknowledgeRequest and corrupted state of batch

Chia-Ping Tsai created KAFKA-18084: -- Summary: Null and leaked AcquisitionLockTimerTask causes hanging AcknowledgeRequest and corrupted state of batch Key: KAFKA-18084 URL: https://issues.apache.org/jira/browse/KA