Re: [VOTE] KIP-1090 Flaky Test Management

2024-09-30 Thread TengYao Chi
+1 (non-binding) Thank you, David Best, TengYao 吳岱儒 於 2024年9月30日 週一,下午6:01寫道: > +1 (non-binding) > Thanks David, I do love this flaky tag. > > TJ >

[jira] [Created] (KAFKA-17662) config.providers configuration missing from the docs

2024-09-30 Thread Mickael Maison (Jira)
Mickael Maison created KAFKA-17662: -- Summary: config.providers configuration missing from the docs Key: KAFKA-17662 URL: https://issues.apache.org/jira/browse/KAFKA-17662 Project: Kafka Issu

Re: [DISCUSS] Apache Kafka 3.8.1 release

2024-09-30 Thread Josep Prat
Hi there, I'll attempt to cut the first RC for 3.8.1 this Wednesday. If you have any bug fix that you'd like to backport to the 3.8 branch and you'd need more time, please let me know. Best, On Tue, Sep 24, 2024 at 1:15 PM Josep Prat wrote: > Hi folks! > As promised, here you have the release p

[jira] [Created] (KAFKA-17660) Kafka not able to import partitioned data

2024-09-30 Thread Adarsh Shukla (Jira)
Adarsh Shukla created KAFKA-17660: - Summary: Kafka not able to import partitioned data Key: KAFKA-17660 URL: https://issues.apache.org/jira/browse/KAFKA-17660 Project: Kafka Issue Type: Bug

[jira] [Created] (KAFKA-17661) Fix flaky BufferPoolTest.testBlockTimeout

2024-09-30 Thread Yu-Lin Chen (Jira)
Yu-Lin Chen created KAFKA-17661: --- Summary: Fix flaky BufferPoolTest.testBlockTimeout Key: KAFKA-17661 URL: https://issues.apache.org/jira/browse/KAFKA-17661 Project: Kafka Issue Type: Bug

Re: [VOTE] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

2024-09-30 Thread Francois Visconte
Could we vote on this? This is causing a bunch of tiered storage read issues as many consumers default to READ_COMMITTED (eg. librdkafka) Thanks, F. On Mon, Sep 16, 2024 at 7:20 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Bumping this thread for vote. PTAL. > > On Mon, Sep

Re: [VOTE] KIP-1090 Flaky Test Management

2024-09-30 Thread 吳岱儒
+1 (non-binding) Thanks David, I do love this flaky tag. TJ

[jira] [Resolved] (KAFKA-17560) testCurrentLag flaky if enabled for new consumer

2024-09-30 Thread TengYao Chi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TengYao Chi resolved KAFKA-17560. - Resolution: Fixed > testCurrentLag flaky if enabled for new consumer > -

[jira] [Created] (KAFKA-17663) Add Metadata caching in admin.internals.PartitionLeaderStrategy

2024-09-30 Thread Nicolas Guyomar (Jira)
Nicolas Guyomar created KAFKA-17663: --- Summary: Add Metadata caching in admin.internals.PartitionLeaderStrategy Key: KAFKA-17663 URL: https://issues.apache.org/jira/browse/KAFKA-17663 Project: Kafka

[jira] [Created] (KAFKA-17665) Kafka Tiered Storage retry back off is not used

2024-09-30 Thread Jorge Machado (Jira)
Jorge Machado created KAFKA-17665: - Summary: Kafka Tiered Storage retry back off is not used Key: KAFKA-17665 URL: https://issues.apache.org/jira/browse/KAFKA-17665 Project: Kafka Issue Type:

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Mickael Maison
Congratulations Kamal! On Mon, Sep 30, 2024 at 2:37 PM Luke Chen wrote: > > Hi all, > > The PMC of Apache Kafka is pleased to announce a new Kafka committer, Kamal > Chandraprakash. > > Kamal has been a Kafka contributor since May 2017. He has made significant > contributions to the tiered storag

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread kafka
Congrats Kamal! - Gaurav > On 30 Sep 2024, at 18:10, Mickael Maison wrote: > > Congratulations Kamal! > > On Mon, Sep 30, 2024 at 2:37 PM Luke Chen wrote: >> >> Hi all, >> >> The PMC of Apache Kafka is pleased to announce a new Kafka committer, Kamal >> Chandraprakash. >> >> Kamal has been

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Josep Prat
Congrats Kamal! On Mon, Sep 30, 2024 at 2:46 PM wrote: > Congrats Kamal! > > - > Gaurav > > > On 30 Sep 2024, at 18:10, Mickael Maison > wrote: > > > > Congratulations Kamal! > > > > On Mon, Sep 30, 2024 at 2:37 PM Luke Chen wrote: > >> > >> Hi all, > >> > >> The PMC of Apache Kafka is pleased

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Murali Basani
Congratulations Kamal. On Mon, Sep 30, 2024 at 3:21 PM Chia-Ping Tsai wrote: > Congratulations Kamal!!! > > Satish Duggana 於 2024年9月30日 週一 下午9:01寫道: > > > Congratulations Kamal! > > > > On Mon, 30 Sept 2024 at 18:18, Josep Prat > > wrote: > > > > > Congrats Kamal! > > > > > > On Mon, Sep 30, 2

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Bruno Cadonna
Congrats! Best, Bruno On 9/30/24 3:23 PM, Murali Basani wrote: Congratulations Kamal. On Mon, Sep 30, 2024 at 3:21 PM Chia-Ping Tsai wrote: Congratulations Kamal!!! Satish Duggana 於 2024年9月30日 週一 下午9:01寫道: Congratulations Kamal! On Mon, 30 Sept 2024 at 18:18, Josep Prat wrote: Congr

Re: [DISCUSS] KIP-1092: Extend Consumer#close with an option to leave the group or not

2024-09-30 Thread Matthias J. Sax
I am also in favor of consistent APIs. That's very good point. I did not take `Admin` API into account, and I am not aware that consumer / producer would have config object classes? Seems we are in a tricky situation here, because "consistent API" to me means producer, consumer, admin and KS.

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Matthias J. Sax
Congrats! On 9/30/24 6:59 AM, Yash Mayya wrote: Congratulations Kamal! On Mon, 30 Sept, 2024, 18:07 Luke Chen, wrote: Hi all, The PMC of Apache Kafka is pleased to announce a new Kafka committer, Kamal Chandraprakash. Kamal has been a Kafka contributor since May 2017. He has made significa

[jira] [Created] (KAFKA-17668) Rewrite LogCleaner#maxOverCleanerThreads and LogCleanerManager#maintainUncleanablePartitions

2024-09-30 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-17668: -- Summary: Rewrite LogCleaner#maxOverCleanerThreads and LogCleanerManager#maintainUncleanablePartitions Key: KAFKA-17668 URL: https://issues.apache.org/jira/browse/KAFKA-17668

Re: [DISCUSS] KIP-1090 Flaky Tests 👻

2024-09-30 Thread Jun Rao
Hi, David, Thanks for the report. 1. Where could we see the reports (quarantined tests, etc) mentioned in the KIP? 2. The quarantine process seems to only apply to integration tests. What's our recommendation for flaky unit tests? Jun On Thu, Sep 26, 2024 at 1:34 PM David Arthur wrote: > If t

[jira] [Resolved] (KAFKA-15862) Remove SecurityManager Support

2024-09-30 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-15862. - Fix Version/s: 4.0.0 Resolution: Fixed > Remove SecurityManager Support > ---

[jira] [Reopened] (KAFKA-15862) Remove SecurityManager Support

2024-09-30 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris reopened KAFKA-15862: - > Remove SecurityManager Support > -- > > Key: KAFKA-158

[jira] [Resolved] (KAFKA-17634) Tighten up wakeup handling for share consumer

2024-09-30 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-17634. Fix Version/s: 4.0.0 Resolution: Fixed > Tighten up wakeup handling for share consu

Re: [DISCUSS] KIP-1094 Add a new constructor method with nextOffsets to ConsumerRecords

2024-09-30 Thread Alieh Saeedi
Hi all, thanks for the feedbacks. Since there is consensus in deprecating the current constructor, we can deprecate it for now and remove it later on (in 5.0 maybe). The KIP is updated. Thanks, Alieh On Sun, Sep 29, 2024 at 6:22 AM Andrew Schofield < andrew_schofield_j...@outlook.com> wrote: >

[jira] [Resolved] (KAFKA-17669) Ivan_Test

2024-09-30 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-17669. Resolution: Invalid > Ivan_Test > - > > Key: KAFKA-17669 >

[ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Luke Chen
Hi all, The PMC of Apache Kafka is pleased to announce a new Kafka committer, Kamal Chandraprakash. Kamal has been a Kafka contributor since May 2017. He has made significant contributions to the tiered storage feature (KIP-405). He authored KIP-1018 and KIP-1075 which improved tiered storage ope

[jira] [Created] (KAFKA-17666) Kafka doesn't monitor disk space after it detects it is full

2024-09-30 Thread Kevin Fletcher (Jira)
Kevin Fletcher created KAFKA-17666: -- Summary: Kafka doesn't monitor disk space after it detects it is full Key: KAFKA-17666 URL: https://issues.apache.org/jira/browse/KAFKA-17666 Project: Kafka

Re: [VOTE] KIP-1076: Metrics for client applications a KIP-714 extension

2024-09-30 Thread Matthias J. Sax
Thanks Bill. Given that Admin client does currently not implement sending metric, the change to disable it by default for Admin client does make sense to me. It might be too much overhead to open/close connection if there are not metrics to be sent, and for developers using the new API to send

[jira] [Created] (KAFKA-17664) Generate ShareAcknowledgeCommitCallbackEvent only when the callback is configured.

2024-09-30 Thread Shivsundar R (Jira)
Shivsundar R created KAFKA-17664: Summary: Generate ShareAcknowledgeCommitCallbackEvent only when the callback is configured. Key: KAFKA-17664 URL: https://issues.apache.org/jira/browse/KAFKA-17664 Pr

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Satish Duggana
Congratulations Kamal! On Mon, 30 Sept 2024 at 18:18, Josep Prat wrote: > Congrats Kamal! > > On Mon, Sep 30, 2024 at 2:46 PM wrote: > > > Congrats Kamal! > > > > - > > Gaurav > > > > > On 30 Sep 2024, at 18:10, Mickael Maison > > wrote: > > > > > > Congratulations Kamal! > > > > > > On Mon, S

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Andrew Schofield
Congratulations, Kamal. From: Bruno Cadonna Sent: 30 September 2024 14:30 To: dev@kafka.apache.org Subject: Re: [ANNOUNCE] New committer: Kamal Chandraprakash Congrats! Best, Bruno On 9/30/24 3:23 PM, Murali Basani wrote: > Congratulations Kamal. > > O

[jira] [Created] (KAFKA-17669) Ivan_Test

2024-09-30 Thread Arseny (Jira)
Arseny created KAFKA-17669: -- Summary: Ivan_Test Key: KAFKA-17669 URL: https://issues.apache.org/jira/browse/KAFKA-17669 Project: Kafka Issue Type: Task Reporter: Arseny -- This mes

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Kamal Chandraprakash
Thank you all for your warm wishes! -- Kamal On Tue, Oct 1, 2024 at 12:11 AM Viktor Somogyi-Vass wrote: > Congrats Kamal! :) > > On Mon, Sep 30, 2024, 19:21 Matthias J. Sax wrote: > > > Congrats! > > > > On 9/30/24 6:59 AM, Yash Mayya wrote: > > > Congratulations Kamal! > > > > > > On Mon, 30

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Chris Egerton
Congrats! -- Forwarded message - From: Satish Duggana Date: Mon, Sep 30, 2024, 09:01 Subject: Re: [ANNOUNCE] New committer: Kamal Chandraprakash To: Congratulations Kamal! On Mon, 30 Sept 2024 at 18:18, Josep Prat wrote: > Congrats Kamal! > > On Mon, Sep 30, 2024 at 2:46 PM

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Chia-Ping Tsai
Congratulations Kamal!!! Satish Duggana 於 2024年9月30日 週一 下午9:01寫道: > Congratulations Kamal! > > On Mon, 30 Sept 2024 at 18:18, Josep Prat > wrote: > > > Congrats Kamal! > > > > On Mon, Sep 30, 2024 at 2:46 PM wrote: > > > > > Congrats Kamal! > > > > > > - > > > Gaurav > > > > > > > On 30 Sep 20

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Lianet M.
Congratulations Kamal!!! On Mon, Sep 30, 2024, 9:22 a.m. Chia-Ping Tsai wrote: > Congratulations Kamal!!! > > Satish Duggana 於 2024年9月30日 週一 下午9:01寫道: > > > Congratulations Kamal! > > > > On Mon, 30 Sept 2024 at 18:18, Josep Prat > > wrote: > > > > > Congrats Kamal! > > > > > > On Mon, Sep 3

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Yash Mayya
Congratulations Kamal! On Mon, 30 Sept, 2024, 18:07 Luke Chen, wrote: > Hi all, > > The PMC of Apache Kafka is pleased to announce a new Kafka committer, Kamal > Chandraprakash. > > Kamal has been a Kafka contributor since May 2017. He has made significant > contributions to the tiered storage f

Re: [DISCUSS] KIP-1092: Extend Consumer#close with an option to leave the group or not

2024-09-30 Thread TengYao Chi
Hello everyone, I'm also +1 on using the fluent API and having the `with` prefix in setter method names. Regarding Matthias' point, I agree with Sophie that we should keep the `CloseOptions` classes separate. These two `CloseOptions` serve different purposes, and while they may occasionally share

Re: [DISCUSS] KIP-1092: Extend Consumer#close with an option to leave the group or not

2024-09-30 Thread Chia-Ping Tsai
> Overall, the "static method builder" pattern seems better to me, and thus I would prefer to make it the "gold standard" and we can see what we can do for `Admin API` mid/long term? Since we want to avoid complicated compatibility issues, adding a static method builder to Admin options seems mor

[jira] [Resolved] (KAFKA-17670) stray partition deletion

2024-09-30 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-17670. -- Resolution: Duplicate > stray partition deletion > > >

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Viktor Somogyi-Vass
Congrats Kamal! :) On Mon, Sep 30, 2024, 19:21 Matthias J. Sax wrote: > Congrats! > > On 9/30/24 6:59 AM, Yash Mayya wrote: > > Congratulations Kamal! > > > > On Mon, 30 Sept, 2024, 18:07 Luke Chen, wrote: > > > >> Hi all, > >> > >> The PMC of Apache Kafka is pleased to announce a new Kafka com

Re: [DISCUSS] KIP-1092: Extend Consumer#close with an option to leave the group or not

2024-09-30 Thread Kirk True
Hi TengYao, Thanks for all the updates. I took another look at the KIP and had some more questions: KT4: Can we change the timeout() method to timeoutMs() to be more consistent? KT5: Can we add the behavior when a null CloseOptions is provided? KT6: Can we use "direct" values in CloseOptions vs.

Re: [DISCUSS] KIP-1094 Add a new constructor method with nextOffsets to ConsumerRecords

2024-09-30 Thread Kirk True
Hi Alieh, Thanks for the KIP! Questions: KT1: In the second paragraph of the Motivation section, it mentions that in some cases the consumer does not return the correct next offset. Can you add a link to the bug here? I'm curious if that still happens with the "new" consumer. KT2: There's a c

[jira] [Created] (KAFKA-17670) enable.stray.partition.deletion should be true by default on CP

2024-09-30 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-17670: Summary: enable.stray.partition.deletion should be true by default on CP Key: KAFKA-17670 URL: https://issues.apache.org/jira/browse/KAFKA-17670 Project: Kafka

Re: [DISCUSS] KIP-1092: Extend Consumer#close with an option to leave the group or not

2024-09-30 Thread Kirk True
Hi all, On Sun, Sep 29, 2024, at 12:08 PM, Chia-Ping Tsai wrote: > > From an API POV, I think the new `CloseOptions` class should not have > any "getters" and thus, it's irrelevant how we represent the different > cases in code internally (even if I believe using `Optional` might be a > good way t

Re: [DISCUSS] KIP-1090 Flaky Tests 👻

2024-09-30 Thread David Arthur
Jun, 1) The reports mentioned in the KIP will need to be built. As a start, I think we can use a cron-based GitHub Action that produces a markdown report. Longer term, we can maybe look into some static site generator like GitHub Pages for hosting weekly reports. 2) In my opinion, any test which

Re: [DISCUSS] KIP-1092: Extend Consumer#close with an option to leave the group or not

2024-09-30 Thread Matthias J. Sax
Kirk, I think good API design principle is to expose the minimum require API to users, and users don't need getters, that's why we don't have any getters in the KS config object classes. Getters are only needed internally. From an impl POV, the internal member can be either (1) package-privat

Re: [DISCUSS] KIP-1092: Extend Consumer#close with an option to leave the group or not

2024-09-30 Thread Matthias J. Sax
Sophie, yes, that a fair summary, and yes, it was only an alternative idea for the case that people think, allowing to disable leave-group request for the plain consumer is not desirable. Seems we are actually on the same page. (And yes, it was meant for this thread, not KIP-1094...) On 9/3

Re: [DISCUSS] KIP-1094 Add a new constructor method with nextOffsets to ConsumerRecords

2024-09-30 Thread Sophie Blee-Goldman
@Matthias I suspect that last email was meant for KIP-1092? But glad we are on the same page :P Seems we're on the same page here as well. Not sure if there are any other open questions, otherwise we can maybe move to a vote? On Mon, Sep 30, 2024 at 5:11 AM Alieh Saeedi wrote: > Hi all, > > th

[jira] [Created] (KAFKA-17672) Run quarantined tests separately

2024-09-30 Thread David Arthur (Jira)
David Arthur created KAFKA-17672: Summary: Run quarantined tests separately Key: KAFKA-17672 URL: https://issues.apache.org/jira/browse/KAFKA-17672 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-17671) Create better documentation for transactions

2024-09-30 Thread Justine Olshan (Jira)
Justine Olshan created KAFKA-17671: -- Summary: Create better documentation for transactions Key: KAFKA-17671 URL: https://issues.apache.org/jira/browse/KAFKA-17671 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-1092: Extend Consumer#close with an option to leave the group or not

2024-09-30 Thread Sophie Blee-Goldman
+1 to using the fluent API and including "with" in the setter names. I also think Matthias raised a good point, that maybe it would be a good time to "fix" this issue in the CloseOptions for the KafkaStreams#close method, to conform to this API format. As for his other "random idea" about combinin

[jira] [Created] (KAFKA-17673) Gradle Build Scan PR check not showing

2024-09-30 Thread David Arthur (Jira)
David Arthur created KAFKA-17673: Summary: Gradle Build Scan PR check not showing Key: KAFKA-17673 URL: https://issues.apache.org/jira/browse/KAFKA-17673 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-1092: Extend Consumer#close with an option to leave the group or not

2024-09-30 Thread Andrew Schofield
+1 from me on the fluent API using the `with` prefix too. Thanks, Andrew From: TengYao Chi Sent: 01 October 2024 05:23 To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-1092: Extend Consumer#close with an option to leave the group or not Hello everyo