[jira] [Resolved] (KAFKA-17987) Remove assorted ZK-related files

[ https://issues.apache.org/jira/browse/KAFKA-17987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17987. Fix Version/s: 4.0.0 Resolution: Fixed > Remove assorted ZK-related files > ---

[jira] [Resolved] (KAFKA-18004) Use version 3.8 to run the ZooKeeper service for end-to-end tests

[ https://issues.apache.org/jira/browse/KAFKA-18004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18004. Fix Version/s: 4.0.0 Resolution: Fixed > Use version 3.8 to run the ZooKeeper servi

[jira] [Resolved] (KAFKA-17885) Enable clients to rebootstrap based on timeout or error code

[ https://issues.apache.org/jira/browse/KAFKA-17885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-17885. Fix Version/s: 4.0.0 Reviewer: Manikumar Resolution: Fixed > Enable client

[jira] [Resolved] (KAFKA-17829) Verify ShareFetch requests return a completed/erroneous future on purgatory close

[ https://issues.apache.org/jira/browse/KAFKA-17829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhinav Dixit resolved KAFKA-17829. --- Resolution: Not A Bug > Verify ShareFetch requests return a completed/erroneous future on pu

[jira] [Resolved] (KAFKA-17957) Console share consumer unable to consume all messages when using multiple brokers

[ https://issues.apache.org/jira/browse/KAFKA-17957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhinav Dixit resolved KAFKA-17957. --- Resolution: Invalid > Console share consumer unable to consume all messages when using multi

[jira] [Created] (KAFKA-18005) Return metadata of sensitive config when describe config

Luke Chen created KAFKA-18005: - Summary: Return metadata of sensitive config when describe config Key: KAFKA-18005 URL: https://issues.apache.org/jira/browse/KAFKA-18005 Project: Kafka Issue Type

Re: [ANNOUNCE] Apache Kafka 3.9.0

Hi Chia-Ping, I just uploaded the artifacts to https://s3-us-west-2.amazonaws.com/kafka-packages. Best, Bruno On 12.11.24 12:16, Chia-Ping Tsai wrote: hi Colin 3.9.1 is nonexistent in the https://s3-us-west-2.amazonaws.com/kafka-packages Could you please check this? I'd like to add version

Re: [DISCUSS] KIP-1050: Consistent error handling for Transactions

Hello, thanks for the updates. Just a minor comment left LM5. Should we add the change to the AuthorizationException parent to the public interfaces section? Same for the InvalidMetadataException. Thanks! Lianet On Tue, Nov 12, 2024 at 1:52 PM Kaushik Raina wrote: > Thanks Lianet for review

Re: [DISCUSS]KIP-1107: Adding record-level acks for producers

Hi TaiJuWu, Thanks for the KIP. The motivation is not clear to me. Could you please elaborate a bit more on it? My concern is that it adds a lot of complexity and the added value seems to be low. Moreover, it will make reasoning about an application from the server side more difficult because we

[jira] [Resolved] (KAFKA-15718) KRaft support in UncleanLeaderElectionTest

[ https://issues.apache.org/jira/browse/KAFKA-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gantigmaa Selenge resolved KAFKA-15718. --- Fix Version/s: 3.8.0 Resolution: Fixed > KRaft support in UncleanLeaderElecti

Re: [ANNOUNCE] Apache Kafka 3.9.0

Thank you for your assistance, Bruno! Best, Chia-Ping Bruno Cadonna 於 2024年11月13日 週三 下午6:16寫道： > Hi Chia-Ping, > > I just uploaded the artifacts to > https://s3-us-west-2.amazonaws.com/kafka-packages. > > Best, > Bruno > > On 12.11.24 12:16, Chia-Ping Tsai wrote: > > hi Colin > > > > 3.9.1 is n

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Hi, Manikumar, Thanks for the updated KIP. JR10. The new AutoOffsetReset class. Could the following constructors be private? protected AutoOffsetReset(String name, long duration) protected AutoOffsetReset(String name) Could AutoOffsetReset(String name, long duration) just be AutoOffsetRes

[jira] [Created] (KAFKA-18009) Remove spurious public constructor for KafkaShareConsumer

Andrew Schofield created KAFKA-18009: Summary: Remove spurious public constructor for KafkaShareConsumer Key: KAFKA-18009 URL: https://issues.apache.org/jira/browse/KAFKA-18009 Project: Kafka

Re: [DISCUSS] KIP-1103: Additional metrics for cooperative consumption

Hi, Apoorv, Thanks for the updated KIP. Just one minor comment. Why is FetchLockRatio a Histogram? Currently, all ratios are measured as Meter. Jun On Thu, Nov 7, 2024 at 4:05 PM Apoorv Mittal wrote: > Hi Jun, > > 11. Got it, agree. I was thinking either to have FetchLockRatio or > FetchLockI

[jira] [Resolved] (KAFKA-17985) Set default value for share.auto.offset.reset in ShareRoundTripWorker.

[ https://issues.apache.org/jira/browse/KAFKA-17985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivsundar R resolved KAFKA-17985. -- Resolution: Fixed > Set default value for share.auto.offset.reset in ShareRoundTripWorker. > -

Re: [DISCUSS] KIP-1103: Additional metrics for cooperative consumption

HI Jun, I think it would be worth to see the distribution as well for FetchLockRatio as we have the distribution captured for RequestTopicPartitionsFetchRatio. The latter has been already marked as Histogram, per your earlier suggestion. Regards, Apoorv Mittal On Wed, Nov 13, 2024 at 5:53 PM Jun

Re: [DISCUSS] KIP-1103: Additional metrics for cooperative consumption

Hi, Apoorv, There is some difference between the two. For RequestTopicPartitionsFetchRatio, we measure a ratio per request and it makes sense to aggregate them as a histogram at the group level. For FetchLockRatio, if a request locks a partition for a certain amount of time, its impact is directly

Re: [DISCUSS] KIP-1103: Additional metrics for cooperative consumption

Sure Jun, My thought process was regarding if there exists a subset of fetch requests which takes considerably higher fetch time than others, then Meter will not reflect that but Histogram can. However I agree on the aggregation part and moved the metric to Meter. Regards, Apoorv Mittal On Wed,

[jira] [Resolved] (KAFKA-17518) AsyncKafkaConsumer cannot reliably leave group when closed with small timeout

[ https://issues.apache.org/jira/browse/KAFKA-17518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans resolved KAFKA-17518. Resolution: Fixed > AsyncKafkaConsumer cannot reliably leave group when closed with small

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

HI Jun, JR10: I was trying to follow KS conventions where we use non-public sub-class with necessary getters. Please check Matthias's suggestion here: https://lists.apache.org/thread/mln783cbswgj14wt4hykcks4kfyl9zpf There will be an AutoOffsetResetInternal class which will have getter method. We c

Re: [VOTE] KIP-1103: Additional metrics for cooperative consumption

Hi, Apoorv, Thanks for the KIP. LGTM Jun On Thu, Nov 7, 2024 at 3:16 AM Apoorv Mittal wrote: > Hi All, > I would like to start vote on KIP-1103: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1103%3A+Additional+metrics+for+cooperative+consumption > > Discussion thread: > https://lis

[jira] [Created] (KAFKA-18006) add 3.9.0 to end-to-end test

Chia-Ping Tsai created KAFKA-18006: -- Summary: add 3.9.0 to end-to-end test Key: KAFKA-18006 URL: https://issues.apache.org/jira/browse/KAFKA-18006 Project: Kafka Issue Type: Improvement

Re: [DISCUSS]KIP-1107: Adding record-level acks for producers

Hi TaiJuWu, I find this adding lot's of complexity and I am still not convinced by the added value. IMO creating a producer instance per ack level is not problematic and the behavior is clear for developers. What would be the added value of the proposed change ? Regards, On Wed, Nov 6, 2024 at

Re: [DISCUSS] Apache Kafka 4.0.0 release

Hi all, This is a friendly reminder that the KIP freeze is on 20th November 2024, one week from today. Best, David On Thu, Nov 7, 2024 at 5:09 AM Luke Chen wrote: > Hi Kamal, > > Just added them in to the release plan. > > Thanks. > Luke > > On Thu, Nov 7, 2024 at 11:11 AM Kamal Chandraprakash

[jira] [Created] (KAFKA-18007) MirrorCheckpointConnector fails with “Timeout while loading consumer groups” after upgrading to Kafka 3.9.0

Asker created KAFKA-18007: - Summary: MirrorCheckpointConnector fails with “Timeout while loading consumer groups” after upgrading to Kafka 3.9.0 Key: KAFKA-18007 URL: https://issues.apache.org/jira/browse/KAFKA-18007

[jira] [Created] (KAFKA-18008) Flaky testMultiConsumerSessionTimeoutOnStopPolling

Lianet Magrans created KAFKA-18008: -- Summary: Flaky testMultiConsumerSessionTimeoutOnStopPolling Key: KAFKA-18008 URL: https://issues.apache.org/jira/browse/KAFKA-18008 Project: Kafka Issue

Re: [DISCUSS]KIP-1107: Adding record-level acks for producers

Hi TaiJuWu, I've been thinking for a while about this KIP before jumping into the discussion. I'm afraid that I don't think the approach in the KIP is the best, given the design of the Kafka protocol in this area. Essentially, each Produce request contains the acks value at the top level, and ma

[jira] [Resolved] (KAFKA-17959) Avoid Reverse DNS Lookup for IP-Based SSL Authentication in Kraft Mode

[ https://issues.apache.org/jira/browse/KAFKA-17959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Narula resolved KAFKA-17959. --- Fix Version/s: 3.8.0 Resolution: Fixed > Avoid Reverse DNS Lookup for IP-Based SSL Au

Re: [DISCUSS]KIP-1107: Adding record-level acks for producers

hi All This KIP is based on our use case where an edge application with many sensors wants to use a single producer to deliver ‘few but varied’ records with different acks settings. The reason for using a single producer is to minimize resource usage on edge devices with limited hardware capabi

Jenkins build is still unstable: Kafka » Kafka PowerPC Daily » test-powerpc #117

See

[jira] [Reopened] (KAFKA-17962) test_pause_and_resume_sink fails with "Failed to consume messages after resuming sink connector" with CONSUMER group protocol

[ https://issues.apache.org/jira/browse/KAFKA-17962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reopened KAFKA-17962: --- > test_pause_and_resume_sink fails with "Failed to consume messages after > resuming sink connector" wi

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Hi Jun, JR10: Yes, we can remove the name field. Updated the KIP. Thanks, On Thu, Nov 14, 2024 at 3:57 AM Jun Rao wrote: > Hi, Manikumar, > > Thanks for the explanation. We can follow the existing KStreams pattern. > > Could AutoOffsetReset(String name, long duration) just > be AutoOffsetRese

Re: [VOTE] KIP-1103: Additional metrics for cooperative consumption

hi Apoorv Could you please add the link of vote thread to the KIP-1103 page? thanks, Chia-Ping On 2024/11/07 11:14:03 Apoorv Mittal wrote: > Hi All, > I would like to start vote on KIP-1103: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1103%3A+Additional+metrics+for+cooperative+consu

Re: [DISCUSS] KIP-1103: Additional metrics for cooperative consumption

Thanks for the update, Apoorv. The KIP looks to me. Jun On Wed, Nov 13, 2024 at 11:36 AM Apoorv Mittal wrote: > Sure Jun, > My thought process was regarding if there exists a subset of fetch requests > which takes considerably higher fetch time than others, then Meter will not > reflect that b

Re: [DISCUSS] KIP-1106: Add duration based offset reset option for consumer clients

Hi, Manikumar, Thanks for the explanation. We can follow the existing KStreams pattern. Could AutoOffsetReset(String name, long duration) just be AutoOffsetReset(long duration)? Jun On Wed, Nov 13, 2024 at 11:53 AM Manikumar wrote: > HI Jun, > > JR10: I was trying to follow KS conventions whe

Re: [DISCUSS]KIP-1107: Adding record-level acks for producers

Hi all, Thanks for your feeback and @Chia-Ping's help. . I also agree topic-level acks config is more reasonable and it can simply the story. When I try implementing record-level acks, I notice I don't have good idea to avoid iterating batches for get partition information (need by *RecordAccumula

Re: How to push forward KIP-1011 complete implementation

Hi Ziming, We should go through the depreciation period before removing/failing it. I remember the `alterConfigs` will still exist in v4.0.0, right? If so, I think we should go with option (1). Thanks. Luke On Thu, Nov 14, 2024 at 2:38 PM ziming deng wrote: > Hi Kafka devs, I have initiated K

How to push forward KIP-1011 complete implementation

Hi Kafka devs, I have initiated KIP-1011 in which we have concluded that we will replace deprecated `alterConfigs` with `incrementalAlterConfigs` in command line tool `CommandConfig`. In the design, if the server doesn’t support `incrementalAlterConfigs`, we would fallback to `alterConfigs` fo

Re: [VOTE] KIP-1103: Additional metrics for cooperative consumption

Hi Apoorv, Thanks for the KIP. +1 (binding) Thanks On Thu, Nov 14, 2024 at 11:11 AM Chia-Ping Tsai wrote: > hi Apoorv > > Could you please add the link of vote thread to the KIP-1103 page? > > thanks, > Chia-Ping > > On 2024/11/07 11:14:03 Apoorv Mittal wrote: > > Hi All, > > I would like to