Problematic new HWM increment behaviour introduced by KIP-207 and KIP-966

We identified a bug/new behaviour that would lead to consumer lagging for a long time and ListOffsets requests failing during that time frame. While the ListOffsets requests failure is expected and has been introduced by KIP-207

[jira] [Created] (KAFKA-17532) Move ShareGroupConfig to share module

Chirag Wadhwa created KAFKA-17532: - Summary: Move ShareGroupConfig to share module Key: KAFKA-17532 URL: https://issues.apache.org/jira/browse/KAFKA-17532 Project: Kafka Issue Type: Sub-task

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #3295

See

Re: [VOTE] KIP-1085: Fix leaking *_DOC variables in StreamsConfig

Hi everyone, The vote is now closed. The KIP has been accepted with 4 binding votes from Lucas, Matthias, Luke, and Chia-Ping. Thanks to everyone for participating! Regards, Jiunn-Yang > Lucas Brutschy 於 2024年9月10日 晚上10:14 寫道： > > +1 (binding). Thanks for the PR! > > On Fri, Sep 6, 2024 at 4:

Re: [VOTE] KIP-1052: Enable warmup in producer performance test

+1 (binding) from me. Thanks, Justine On Wed, Sep 4, 2024 at 3:22 PM Welch, Matt wrote: > Hi Kafka devs, > > Bumping this VOTE thread again for visibility. > > Thanks, > Matt > > -Original Message- > From: Welch, Matt > Sent: Friday, August 23, 2024 4:26 PM > To: dev@kafka.apache.org >

[jira] [Resolved] (KAFKA-15073) Automation for old/inactive PRs

[ https://issues.apache.org/jira/browse/KAFKA-15073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur resolved KAFKA-15073. -- Resolution: Fixed Resolving this since we've merged the change to actually close old PRs > Au

[jira] [Resolved] (KAFKA-16332) Remove Deprecated builder methods for Time/Session/Join/SlidingWindows

[ https://issues.apache.org/jira/browse/KAFKA-16332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-16332. - Resolution: Fixed > Remove Deprecated builder methods for Time/Session/Join/SlidingWindo

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.9 #65

See

[jira] [Created] (KAFKA-17531) Remove deprecates methods JoinWindows#of and #grace

Matthias J. Sax created KAFKA-17531: --- Summary: Remove deprecates methods JoinWindows#of and #grace Key: KAFKA-17531 URL: https://issues.apache.org/jira/browse/KAFKA-17531 Project: Kafka Iss

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.8 #88

See

[jira] [Resolved] (KAFKA-17290) Add integration test for ShareGroupFetch/Acknowledge requests

[ https://issues.apache.org/jira/browse/KAFKA-17290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chirag Wadhwa resolved KAFKA-17290. --- Resolution: Fixed The task has been completed and the PR has been merged > Add integration

[jira] [Resolved] (KAFKA-17401) Add config for purge interval for Share Fetch Purgatory

[ https://issues.apache.org/jira/browse/KAFKA-17401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhinav Dixit resolved KAFKA-17401. --- Fix Version/s: 4.0.0 Resolution: Fixed > Add config for purge interval for Share Fetc

Re: [DISCUSS] Regarding Old PRs

Hi David, I checked the PR, looks good! And I think the times set are adequate. Best, -- Josep Prat Open Source Engineering Director, Aiven josep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa,

[jira] [Resolved] (KAFKA-17400) Introduce a purgatory to deal with share fetch requests that cannot be completed instantaneously

[ https://issues.apache.org/jira/browse/KAFKA-17400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-17400. - Fix Version/s: 4.0.0 Resolution: Fixed merged the PR to trunk. > Introduce a purgatory to deal w

[jira] [Resolved] (KAFKA-17231) Share consumer node latency metrics

[ https://issues.apache.org/jira/browse/KAFKA-17231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17231. Fix Version/s: 4.0.0 Resolution: Fixed > Share consumer node latency metrics >

Re: [DISCUSS] Regarding Old PRs

Hey folks, I wanted to revive this old thread. I'd like to do the following: * Change our stale workflow to start with the oldest PRs and move forward * Enable closing of stale PRs (over 120 days) Here's a patch with these changes: https://github.com/apache/kafka/pull/17166 Docs for actions/stal

[jira] [Resolved] (KAFKA-17392) Remove whitelist in ConsoleConsumerOptions

[ https://issues.apache.org/jira/browse/KAFKA-17392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17392. Fix Version/s: 4.0.0 Resolution: Fixed > Remove whitelist in ConsoleConsumerOptions

[jira] [Created] (KAFKA-17530) Remove blacklist/whitelist from ReplaceField

Chia-Ping Tsai created KAFKA-17530: -- Summary: Remove blacklist/whitelist from ReplaceField Key: KAFKA-17530 URL: https://issues.apache.org/jira/browse/KAFKA-17530 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-17529) Remove blacklist from MM2

Chia-Ping Tsai created KAFKA-17529: -- Summary: Remove blacklist from MM2 Key: KAFKA-17529 URL: https://issues.apache.org/jira/browse/KAFKA-17529 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-17528) Remove whitelist/blacklist from JMXReporter

Chia-Ping Tsai created KAFKA-17528: -- Summary: Remove whitelist/blacklist from JMXReporter Key: KAFKA-17528 URL: https://issues.apache.org/jira/browse/KAFKA-17528 Project: Kafka Issue Type: S

Re: [DISCUSS] KIP-1043: Administration of groups

Hi Andrew, Took another full pass here after the latest changes (thanks for addressing all the feedback!). Only minor comment left is to add to the reject alternatives the INCONSISTENT_GROUP_PROTOCOL error that we discussed and discarded (see LM9). Also agree with the 2 minor points made by Lucas

[jira] [Resolved] (KAFKA-17347) Add omitted --client-metrics option to kafka-configs.sh

[ https://issues.apache.org/jira/browse/KAFKA-17347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17347. Resolution: Fixed > Add omitted --client-metrics option to kafka-configs.sh >

Re: New release branch 3.9

Hi Colin, we found another blocker bug: https://issues.apache.org/jira/browse/KAFKA-17527 Working on a fix. -Matthias On 9/9/24 6:51 PM, David Arthur wrote: Colin, I found a race condition in the migration while investigating a flaky test. If encountered, the migration driver will get stu

[jira] [Created] (KAFKA-17527) Kafka Streams fails with NPE for missing RecordContext

Matthias J. Sax created KAFKA-17527: --- Summary: Kafka Streams fails with NPE for missing RecordContext Key: KAFKA-17527 URL: https://issues.apache.org/jira/browse/KAFKA-17527 Project: Kafka

[jira] [Created] (KAFKA-17526) make ConfigCommandIntegrationTest.java test use correct arguments in testing alias

Chia-Ping Tsai created KAFKA-17526: -- Summary: make ConfigCommandIntegrationTest.java test use correct arguments in testing alias Key: KAFKA-17526 URL: https://issues.apache.org/jira/browse/KAFKA-17526

Re: [DISCUSS] Single broker failures causing offline partitions

Hi Haruki and Kamal, It's very interesting to see how you're handling this Haruki. - For proactive broker replacement, we are doing something similar. We monitor certain metrics, mostly related to slow disks, and automatically replace the machines if those alerts fire but Kafka state is still safe

Re: [DISCUSS] Single broker failures causing offline partitions

Hi Justine and Calvin, Thank you both for the feedback, I agree that KIP-966 should solve #2. I'll file a JIRA ticket for #1 (currently in the process of registering my account). Best, Martin On Tue, Sep 10, 2024 at 9:36 PM Justine Olshan wrote: > Hey Calvin and Martin, > > Makes sense. So KIP

[jira] [Created] (KAFKA-17525) Convert the UnknownServerException to InvalidRequestException when altering client-metrics config at runtime

Chia-Ping Tsai created KAFKA-17525: -- Summary: Convert the UnknownServerException to InvalidRequestException when altering client-metrics config at runtime Key: KAFKA-17525 URL: https://issues.apache.org/jira/brow

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.9 #64

See

[jira] [Resolved] (KAFKA-17506) KRaftMigrationDriver initialization race condition

[ https://issues.apache.org/jira/browse/KAFKA-17506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur resolved KAFKA-17506. -- Resolution: Fixed > KRaftMigrationDriver initialization race condition > -

[jira] [Created] (KAFKA-17524) StreamThreadTest shouldReturnErrorIfProducerInstanceIdNotInitialized hanging

David Arthur created KAFKA-17524: Summary: StreamThreadTest shouldReturnErrorIfProducerInstanceIdNotInitialized hanging Key: KAFKA-17524 URL: https://issues.apache.org/jira/browse/KAFKA-17524 Project:

Re: [DISCUSS] KIP-1043: Administration of groups

Hi Andrew, thanks for the KIP! It is looking good from my side! I like the simplification, and that we added the new error but only of the Describe RPCs. It's a good pragmatic improvement of the current state of things. I only have very minor comments: - nit: In `GroupListing`, you seem to impo

[jira] [Created] (KAFKA-17523) Connectors should be able to override offset.flush.interval.ms

Viktor Somogyi-Vass created KAFKA-17523: --- Summary: Connectors should be able to override offset.flush.interval.ms Key: KAFKA-17523 URL: https://issues.apache.org/jira/browse/KAFKA-17523 Project:

[jira] [Created] (KAFKA-17522) Share partition acquire() need not return a future

Abhinav Dixit created KAFKA-17522: - Summary: Share partition acquire() need not return a future Key: KAFKA-17522 URL: https://issues.apache.org/jira/browse/KAFKA-17522 Project: Kafka Issue Ty

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #3292

See

[jira] [Created] (KAFKA-17521) bootstrap-controller option buggy behavior

Michal Medvecky created KAFKA-17521: --- Summary: bootstrap-controller option buggy behavior Key: KAFKA-17521 URL: https://issues.apache.org/jira/browse/KAFKA-17521 Project: Kafka Issue Type:

Re: [DISCUSS] Single broker failures causing offline partitions

Hi Kamal, > Is the leader election automated to find the replica with the highest offset and latest epoch? > And, finding the eligible replica manually, will increase the outage mitigation time That's right, but this procedure is not yet automated in our operation. Because, we found that "choosin

[jira] [Created] (KAFKA-17520) align the low bound of ducktape version

Chia-Ping Tsai created KAFKA-17520: -- Summary: align the low bound of ducktape version Key: KAFKA-17520 URL: https://issues.apache.org/jira/browse/KAFKA-17520 Project: Kafka Issue Type: Impr

Re: [VOTE] KIP-1064 - Upgrade slf4j to 2.x

Hi Chia, thank you for the vote. I have updated kip motivation with it. Thanks, Murali On Wed, Sep 11, 2024 at 8:15 AM Chia-Ping Tsai wrote: > +1 (binding) > > BTW, could you please add "compatibility issue" to the motivation? > > the version of slf4j provider must be equal to slf4j API [0]. In

[VOTE] KIP-1034: Dead letter queue in Kafka Streams

Hi all, We would like to start a vote for KIP-1034: Dead letter queue in Kafka Streams The KIP is available on https://cwiki.apache.org/confluence/display/KAFKA/KIP-1034%3A+Dead+letter+queue+in+Kaf

Re: [DISCUSS] Single broker failures causing offline partitions

Hi Haruki, We are also interested in this issue. > The problem is how to identify such "eligible" replicas. Is the leader election automated to find the replica with the highest offset and latest epoch? If yes, could you please open a PR for it? When a broker goes down, it might be serving lead