Re: [DISCUSS] KIP-1187: Support to retrieve remote log size via DescribeLogDirs RPC

2025-08-04 Thread Satish Duggana
Hi PoAn, Thanks for the KIP, this is a valuable feature for operators to get better visibility into partition sizes across both local and remote storage. While RemoteLogSegmentMetadata provides segment-level details to help developers build custom utilities, it's beneficial to enhance the existing

Re: [VOTE] KIP-1123: Rack-aware partitioning for Kafka Producer

2025-05-14 Thread Satish Duggana
Thanks Ivan for the KIP, +1 (binding) ~Satish. On Wed, 14 May 2025 at 16:09, Christo Lolov wrote: > > Hello! > > Apologies for coming late to the party. I think this is a great idea and > don't see why we can't try it out in practice! > > +1 (binding) > > Best, > Christo > > On Tue, 29 Apr 2025

Re: [DISCUSS] KIP-1183: Unified Shared Storage

2025-05-14 Thread Satish Duggana
Thanks Xinyu for accepting our request to initiate a KIP on AutoMq's proposal. SD-1: It is a good start towards building log storage layer abstractions, including LogSegment and Log, for both local and shared storage. However, more clarity is needed on how these abstractions are defined. The class

[jira] [Created] (KAFKA-19265) Use snapshotting of remote log metedata topic that is used when a broker is restarted.

2025-05-11 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-19265: -- Summary: Use snapshotting of remote log metedata topic that is used when a broker is restarted. Key: KAFKA-19265 URL: https://issues.apache.org/jira/browse/KAFKA-19265

Re: [kafka-clients] [ANNOUNCE] Apache Kafka 4.0.0

2025-04-05 Thread Satish Duggana
ari Beyene, > Mickael Maison, Ming-Yen Chung, mingdaoy, msureshpro, Murali Basani, > Nancy, Nick Guo, Nick Telford, Oleg Bonar, Oleksandr K., Omnia > Ibrahim, Parker Chang, Patrik Marton, Paul R. Mellor, Peter Lee, > Philip Nee, PoAn Yang, Rajini Sivaram, Ramin Gharib, Ritika Reddy, > Rober

Re: [ANNOUNCE] New committer: Andrew Schofield

2024-12-03 Thread Satish Duggana
Congratulations Andrew! On Tue, 3 Dec 2024 at 07:55, Luke Chen wrote: > > Congratulations! Well deserved! > > Luke > > On Tue, Dec 3, 2024 at 5:55 AM Matthias J. Sax wrote: > > > Congrats! > > > > On 12/2/24 12:02 PM, Kirk True wrote: > > > Congratulations Andrew Well-deserved. > > > > > > O

Re: [VOTE] KIP-1030: Change constraints and default values for various configurations

2024-11-28 Thread Satish Duggana
+1 (binding) ~Satish. On Thu, 28 Nov 2024 at 15:46, Federico Valeri wrote: > > +1 non binding. > > Thanks > > On Thu, Nov 28, 2024 at 3:16 AM Kamal Chandraprakash > wrote: > > > > Hi Divij, > > > > +1 (Binding). Thanks for the KIP! > > > > Kamal > > > > On Thu, Nov 28, 2024, 05:37 Luke Chen wr

Re: [DISCUSS] KIP-1030: Change constraints and default values for various configurations

2024-11-27 Thread Satish Duggana
Hi Kamal, Thanks for updating the KIP with the offline discussion we had, especially on remotelog.manager.* thread pools with the new behaviour. Hi Divij, We should target having a validation tool in 4.0 so that users can run it before they run their upgrade and take necessary actions on it. I am

Re: [ANNOUNCE] Apache Kafka 3.9.0

2024-11-07 Thread Satish Duggana
et Magrans , Logan Zhu, Loïc GREFFIER, Lucas Brutschy, Luke Chen, Maciej > Moscicki, Manikumar Reddy, Mason Chen, Matthias J. Sax, Max Riedel, Mickael > Maison, Murali Basani, 120571969+nancy-ksol...@users.noreply.github.com, > NICOLAS GUYOMAR, Nikolay, Okada Haruki, Omnia Ibrahim, PaulRMello

Re: [VOTE] KIP-1105: Make remote log manager thread-pool configs dynamic

2024-11-07 Thread Satish Duggana
+1 (binding) On Thu, 7 Nov 2024 at 13:55, Federico Valeri wrote: > > +1 (non binding) > > Thanks > > On Thu, Nov 7, 2024 at 5:46 AM Chu Cheng Li wrote: > > > > +1 > > > > On Thu, Nov 7, 2024 at 12:39 PM Kamal Chandraprakash < > > kamal.chandraprak...@gmail.com> wrote: > > > > > Hi all, > > > > >

Re: [DISCUSS] KIP-1105: Make remote log manager thread-pool configs dynamic

2024-11-07 Thread Satish Duggana
ol.size config from the KIP. > > Currently, the config is not marked as deprecated which was > already proposed in > KIP-950, we can mark that config as deprecated in the source code. > > Thanks, > Kamal > > On Thu, Nov 7, 2024 at 9:23 PM Satish Duggana > wrot

Re: [DISCUSS] KIP-1105: Make remote log manager thread-pool configs dynamic

2024-11-07 Thread Satish Duggana
Thanks Kamal for the KIP. This is useful for dynamically changing the thread pool configurations, especially in production environments. We can skip remote.log.manager.thread.pool.size as it is already deprecated, and remote.log.manager.copier.thread.pool.size, remote.log.manager.expiration.thread.

Re: [VOTE] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

2024-11-05 Thread Satish Duggana
+1 Thanks Kamal for the KIP. Discussed offline with Kamal on the proposal and the changes. It improves the performance of consumers' remote reads in read_committed mode. It brings minor changes in a backward compatible manner. ~Satish. On Mon, 4 Nov 2024 at 19:33, Divij Vaidya wrote: > +1 > >

Re: [ANNOUNCE] New committer: Kamal Chandraprakash

2024-09-30 Thread Satish Duggana
Congratulations Kamal! On Mon, 30 Sept 2024 at 18:18, Josep Prat wrote: > Congrats Kamal! > > On Mon, Sep 30, 2024 at 2:46 PM wrote: > > > Congrats Kamal! > > > > - > > Gaurav > > > > > On 30 Sep 2024, at 18:10, Mickael Maison > > wrote: > > > > > > Congratulations Kamal! > > > > > > On Mon, S

Re: [DISCUSS] Apache Kafka 4.0.0 release

2024-09-23 Thread Satish Duggana
+1 Thanks David for volunteering! On Mon, 23 Sept 2024 at 19:27, Lianet M. wrote: > > +1 > > Thanks David! > > On Mon, Sep 23, 2024, 9:54 a.m. Justine Olshan > wrote: > > > +1 and thanks! > > > > On Mon, Sep 23, 2024 at 6:36 AM Chia-Ping Tsai wrote: > > > > > +1 and thanks David for volunteerin

[jira] [Resolved] (KAFKA-15434) Tiered Storage Quotas

2024-09-17 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana resolved KAFKA-15434. Resolution: Duplicate Duplicate of https://issues.apache.org/jira/browse/KAFKA-15265

Re: [ANNOUNCE] New committer: Jeff Kim

2024-09-09 Thread Satish Duggana
Congratulations Jeff! On Mon, 9 Sept 2024 at 18:37, Bruno Cadonna wrote: > > Congrats! Well deserved! > > Best, > Bruno > > > > On 9/9/24 2:28 PM, Bill Bejeck wrote: > > Congrats Jeff!! > > > > On Mon, Sep 9, 2024 at 7:50 AM Lianet M. wrote: > > > >> Congrats Jeff!!! > >> > >> On Mon, Sep 9, 202

Re: [ANNOUNCE] New committer: Lianet Magrans

2024-08-28 Thread Satish Duggana
Congratulations Lianet!! On Thu, 29 Aug 2024 at 10:08, Muralidhar Basani wrote: > > Congratulations Lianet !! > > Murali > > On Thu, 29 Aug 2024 at 05:48, Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > > > Congrats Lianet! > > > > On Thu, Aug 29, 2024 at 8:07 AM Luke Chen wrot

Re: [VOTE] KIP-1075: Introduce delayed remote list offsets purgatory to make LIST_OFFSETS async

2024-08-22 Thread Satish Duggana
Thanks Kamal for the KIP, LGTM. +1 from me. On Mon, 19 Aug 2024 at 10:41, Kamal Chandraprakash wrote: > > Hi, > > I would like to open voting for KIP-1075. I've addressed the review > comments in the discussion thread. Please vote if the proposal looks good. > > https://cwiki.apache.org/confluen

Re: [DISCUSS] KIP-1075: Introduce delayed remote list offsets purgatory to make LIST_OFFSETS async

2024-08-22 Thread Satish Duggana
Thanks Kamal for the KIP. This is an important enhancement to unblock the io threads as ListOffsetsAPI may need to fetch data from remote storage and it may take longer to process them. This feature helped in not starving other broker api requests in our environments when there are many ListOffset

[jira] [Resolved] (KAFKA-16887) document remote copy/fetch quotas metrics

2024-08-13 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana resolved KAFKA-16887. Resolution: Fixed > document remote copy/fetch quotas metr

Re: [VOTE] KIP-950: Tiered Storage Disablement

2024-08-07 Thread Satish Duggana
Thanks Kamal, and Luke for improving the earlier solution for KRaft. One minor comment I have is to change the config name from "remote.log.copy.disable" to "remote.log.copy.enable" with the default value being true. The solution summary to disable tiered storage on a topic: - When a user wants

Re: [VOTE] KIP-1057: Add remote log metadata flag to the dump log tool

2024-06-27 Thread Satish Duggana
Thanks Federico for the KIP. +1 ~Satish. On Thu, 27 Jun 2024 at 13:44, Federico Valeri wrote: > > Thanks for the votes so far, bumping this thread to get more. > > On Fri, Jun 21, 2024 at 4:23 PM Kamal Chandraprakash > wrote: > > > > Hi Federico, > > > > Thanks for the KIP! +1 from me. > > > >

Re: [DISCUSS] KIP-1057: Add remote log metadata flag to the dump log tool

2024-06-17 Thread Satish Duggana
Thanks Federico for the KIP. This feature is helpful for developers while debugging tiered storage related issues. Even though RLMM is a pluggable interface, it is still useful to have a utility that is meant for the default/inbuilt implementation based on the internal topic. We can clarify that

[jira] [Resolved] (KAFKA-15420) Kafka Tiered Storage V1

2024-06-17 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana resolved KAFKA-15420. Resolution: Fixed > Kafka Tiered Storage

[jira] [Created] (KAFKA-16977) remote log manager dynamic configs are not available after broker restart.

2024-06-17 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-16977: -- Summary: remote log manager dynamic configs are not available after broker restart. Key: KAFKA-16977 URL: https://issues.apache.org/jira/browse/KAFKA-16977

[jira] [Created] (KAFKA-16976) Improve the dynamic config handling for RemoteLogManagerConfig when a broker is restarted.

2024-06-17 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-16976: -- Summary: Improve the dynamic config handling for RemoteLogManagerConfig when a broker is restarted. Key: KAFKA-16976 URL: https://issues.apache.org/jira/browse/KAFKA-16976

Re: [DISCUSS] Apache Kafka 3.8.0 release

2024-06-14 Thread Satish Duggana
+1 on going with 3.8 release with the existing plan and targeting the required features in 3.9 timelines. 4.0 will be targeted in the usual cycle(4 months) after 3.9 is released. On Fri, 14 Jun 2024 at 15:19, Edoardo Comar wrote: > > Josep, > past the deadline sorry but I can't see reasons not t

[jira] [Resolved] (KAFKA-14877) refactor InMemoryLeaderEpochCheckpoint

2024-06-12 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana resolved KAFKA-14877. Resolution: Invalid > refactor InMemoryLeaderEpochCheckpo

[jira] [Created] (KAFKA-16947) Kafka Tiered Storage V2

2024-06-12 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-16947: -- Summary: Kafka Tiered Storage V2 Key: KAFKA-16947 URL: https://issues.apache.org/jira/browse/KAFKA-16947 Project: Kafka Issue Type: Improvement

[jira] [Resolved] (KAFKA-16890) Failing to build aux state on broker failover

2024-06-12 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana resolved KAFKA-16890. Resolution: Fixed > Failing to build aux state on broker failo

Re: [VOTE] KIP-950: Tiered Storage Disablement

2024-05-20 Thread Satish Duggana
+1 Thanks Christo for addressing the review comments. We can update the KIP for any minor comments/clarifications. On Thu, 16 May 2024 at 15:21, Luke Chen wrote: > > Thanks Chia-Ping! > Since ZK is going to be removed, I agree the KRaft part has higher priority. > But if Christo or the community

Re: [VOTE] KIP-1018: Introduce max remote fetch timeout config

2024-05-09 Thread Satish Duggana
Thanks Kamal for the KIP. +1 from me. ~Satish. On Thu, 9 May 2024 at 17:52, Christo Lolov wrote: > > Heya Kamal, > > Thanks for the KIP and the answers in the discussion! > > +1 from me :) > > Best, > Christo > > On Thu, 9 May 2024 at 11:11, Federico Valeri wrote: > > > +1 non binding > > > >

Re: [VOTE] KIP-932: Queues for Kafka

2024-05-07 Thread Satish Duggana
Hi Andrew, Thanks for the nice KIP, it will allow other messaging use cases to be onboarded to Kafka. +1 from me. Satish. On Tue, 7 May 2024 at 03:41, Jun Rao wrote: > > Hi, Andrew, > > Thanks for the KIP. +1 > > Jun > > On Mon, Mar 18, 2024 at 11:00 AM Edoardo Comar > wrote: > > > Thanks Andr

Re: [VOTE] KIP-1023: Follower fetch from tiered offset

2024-04-26 Thread Satish Duggana
Thanks Abhijeet for the KIP. +1 from me. ~Satish On Fri, 26 Apr 2024 at 8:35 PM, Jun Rao wrote: > Hi, Abhijeet, > > Thanks for the KIP. +1 > > Jun > > On Thu, Apr 25, 2024 at 10:30 PM Abhijeet Kumar < > abhijeet.cse@gmail.com> > wrote: > > > Hi All, > > > > I would like to start the vote fo

Re: [ANNOUNCE] New Kafka PMC Member: Greg Harris

2024-04-14 Thread Satish Duggana
Congratulations Greg! On Mon, 15 Apr 2024 at 11:04, Claude Warren wrote: > > Congrats Greg! All the hard work paid off. > > On Mon, Apr 15, 2024 at 6:58 AM Ivan Yurchenko wrote: > > > Congrats Greg! > > > > On Sun, Apr 14, 2024, at 22:51, Sophie Blee-Goldman wrote: > > > Congrats Greg! Happy to

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

2024-04-09 Thread Satish Duggana
when tiered storage disablement request is still being > > > processed on a topic? > > > > > > *If the disablement policy is Delete then a successive topic deletion > > > request is going to be a NOOP because RemoteLogs is already being > > deleted.*

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

2024-04-09 Thread Satish Duggana
+1 to Jun for adding the consumer fetching from a follower scenario also to the existing section that talked about the drawback when a node built with last-tiered-offset has become a leader. As Abhijeet mentioned, we plan to have a follow-up KIP that will address these by having a deprioritzation o

Re: [ANNOUNCE] New committer: Christo Lolov

2024-03-26 Thread Satish Duggana
Congratulations Christo! On Tue, 26 Mar 2024 at 19:20, Ivan Yurchenko wrote: > > Congrats! > > On Tue, Mar 26, 2024, at 14:48, Lucas Brutschy wrote: > > Congrats! > > > > On Tue, Mar 26, 2024 at 2:44 PM Federico Valeri > > wrote: > > > > > > Congrats! > > > > > > On Tue, Mar 26, 2024 at 2:27 PM

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

2024-03-26 Thread Satish Duggana
ghts on how it could be improved or should I start a >> voting thread? >> >> Best, >> Christo >> >> On Thu, 14 Mar 2024 at 06:12, Luke Chen wrote: >> >> > Hi Christo, >> > >> > Any update with this KIP? >> > If you don

Re: [DISCUSS] Apache Kafka 3.6.2 release

2024-03-13 Thread Satish Duggana
+1, Thanks Mani for volunteering. On Thu, 14 Mar 2024 at 06:01, Luke Chen wrote: > > +1, Thanks Manikumar! > > On Thu, Mar 14, 2024 at 3:40 AM Bruno Cadonna wrote: > > > Thanks Manikumar! > > > > +1 > > > > Best, > > Bruno > > > > On 3/13/24 5:56 PM, Josep Prat wrote: > > > +1 thanks for volunte

Re: [VOTE] KIP-956: Tiered Storage Quotas

2024-03-09 Thread Satish Duggana
Thanks Abhijeet for the KIP, +1 from me. On Sat, 9 Mar 2024 at 1:51 AM, Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > +1 (non-binding), Thanks for the KIP, Abhijeet! > > -- > Kamal > > On Fri, Mar 8, 2024 at 11:02 PM Jun Rao wrote: > > > Hi, Abhijeet, > > > > Thanks for the KI

Re: [DISCUSS] Apache Kafka 3.8.0 release

2024-02-28 Thread Satish Duggana
Thanks Josep, +1. On Tue, 27 Feb 2024 at 17:29, Divij Vaidya wrote: > > Thank you for volunteering Josep. +1 from me. > > -- > Divij Vaidya > > > > On Tue, Feb 27, 2024 at 9:35 AM Bruno Cadonna wrote: > > > Thanks Josep! > > > > +1 > > > > Best, > > Bruno > > > > On 2/26/24 9:53 PM, Chris Egerto

Re: [kafka-clients] Re: [ANNOUNCE] Apache Kafka 3.7.0

2024-02-28 Thread Satish Duggana
i Kokhreidze, Lianet > > Magrans, Liu Zeyu, Lucas Brutschy, Lucia Cerchie, Luke Chen, maniekes, > > Manikumar Reddy, mannoopj, Maros Orsak, Matthew de Detrich, Matthias > > J. Sax, Max Riedel, Mayank Shekhar Narula, Mehari Beyene, Michael > > Westerby, Mickael Maison, Nick Tel

[jira] [Created] (KAFKA-16161) Avoid creating remote log metadata snapshot file in partition data directory.

2024-01-17 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-16161: -- Summary: Avoid creating remote log metadata snapshot file in partition data directory. Key: KAFKA-16161 URL: https://issues.apache.org/jira/browse/KAFKA-16161

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

2024-01-17 Thread Satish Duggana
Hi Christo, Thanks for volunteering to contribute to the KIP discussion. I suggest considering this KIP for both ZK and KRaft as it will be helpful for this feature to be available in 3.8.0 running with ZK clusters. Thanks, Satish. On Wed, 17 Jan 2024 at 19:04, Christo Lolov wrote: > > Hello! >

Re: [VOTE] KIP-1005: Expose EarliestLocalOffset and TieredOffset

2024-01-11 Thread Satish Duggana
+1 (binding) Thanks, Satish. On Thu, 11 Jan 2024 at 17:52, Divij Vaidya wrote: > > +1 (binding) > > Divij Vaidya > > > > On Tue, Dec 26, 2023 at 7:05 AM Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > > > +1 (non-binding). Thanks for the KIP! > > > > -- > > Kamal > > > > On Thu

Re: [VOTE] KIP-1007: Introduce Remote Storage Not Ready Exception

2024-01-05 Thread Satish Duggana
Thanks Kamal for the KIP. +1 (binding) On Fri, 5 Jan 2024 at 17:04, Divij Vaidya wrote: > > +1 (binding) > > -- > Divij Vaidya > > > > On Thu, Dec 21, 2023 at 10:30 AM Luke Chen wrote: > > > Hi Kamal, > > > > Thanks for the KIP. > > +1 (binding) from me. > > > > Luke > > > > On Thu, Dec 21, 202

Re: [DISCUSS] KIP-1007: Introduce Remote Storage Not Ready Exception

2024-01-05 Thread Satish Duggana
Thanks for the KIP Kamal, LGTM. On Tue, 26 Dec 2023 at 10:23, Kamal Chandraprakash wrote: > > Hi Divij, > > Thanks for reviewing the KIP! I've updated the KIP with the below > documentation. Let me know if it needs to be changed: > > The consumer can read the local data as long as it knows the of

Re: [VOTE] KIP-1013: Drop broker and tools support for Java 11 in Kafka 4.0 (deprecate in 3.7)

2024-01-02 Thread Satish Duggana
Thanks Ismael for the proposal. Adopting JDK 17 enhances developer productivity and has reached a level of maturity that has led to its adoption by several other major projects, signifying its reliability and effectiveness. +1 (binding) ~Satish. On Wed, 3 Jan 2024 at 06:59, Justine Olshan wro

Re: [ANNOUNCE] New Kafka PMC Member: Divij Vaidya

2023-12-27 Thread Satish Duggana
Congratulations Divij! On Thu, 28 Dec 2023 at 07:21, Kamal Chandraprakash wrote: > > Congrats Divij! > > On Thu, Dec 28, 2023, 07:09 Kirk True wrote: > > > Congrats Divij!!! > > > > On Wed, Dec 27, 2023, at 1:44 PM, Jorge Esteban Quilcate Otoya wrote: > > > Congratulations Divij!! > > > > > > On

Re: New Release Branch 3.7

2023-12-14 Thread Satish Duggana
Hi Stanislav, +1 to what Luke mentioned in the earlier email. I have also replied to your comment on https://issues.apache.org/jira/browse/KAFKA-15147 These are minor changes and we plan to target the remaining changes to be merged in the next few days(before 20th Dec). Thanks, Satish. On Thu, 1

[jira] [Created] (KAFKA-15864) Add more tests asserting the log-start-offset, local-log-start-offset, and HW/LSO/LEO in rolling over segments with tiered storage.

2023-11-20 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15864: -- Summary: Add more tests asserting the log-start-offset, local-log-start-offset, and HW/LSO/LEO in rolling over segments with tiered storage. Key: KAFKA-15864 URL: https

Re: [DISCUSS] KIP-1005: Add EarliestLocalOffset to GetOffsetShell

2023-11-20 Thread Satish Duggana
Thanks Christo for starting the discussion on the KIP. As mentioned in KAFKA-15857[1], the goal is to add new entries for local-log-start-offset and tierd-offset in OffsetSpec. This will be used in AdminClient APIs and also to be added as part of GetOffsetShell. This was also raised by Kamal in th

Re: [VOTE] KIP-963: Additional metrics in Tiered Storage

2023-11-20 Thread Satish Duggana
+1 (binding) Thanks for the KIP and the discussion. Discussion mail thread for the KIP: https://lists.apache.org/thread/40vsyc240hyody37mf2f0pn90shkzb45 On Tue, 21 Nov 2023 at 05:21, Kamal Chandraprakash wrote: > > +1 (non-binding). Thanks for the KIP! > > On Tue, Nov 21, 2023, 03:04 Divij Vai

[jira] [Created] (KAFKA-15857) Introduce LocalLogStartOffset and TieredOffset in OffsetSpec.

2023-11-20 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15857: -- Summary: Introduce LocalLogStartOffset and TieredOffset in OffsetSpec. Key: KAFKA-15857 URL: https://issues.apache.org/jira/browse/KAFKA-15857 Project: Kafka

Re: [DISCUSS] KIP-963: Upload and delete lag metrics in Tiered Storage

2023-11-20 Thread Satish Duggana
dd the .delete > suffix we stop serving traffic from those segments, hence we will be > serving requests for those from remote. Am I wrong? > > Best, > Christo > > On Thu, 16 Nov 2023 at 08:45, Satish Duggana > wrote: > > > Thanks Christo for your reply. > > > &

Re: [DISCUSS] KIP-963: Upload and delete lag metrics in Tiered Storage

2023-11-16 Thread Satish Duggana
moteDeleteBytesPerSec as well. > > 7. Yes, this is I believe a better explanation than the one I have in the > KIP, so I will aim to update it with your one. Thank you! LocalDeleteLag > makes sense to me as well, I will aim to include it. > > *re: Kamal* > > 1. I can add

Re: [VOTE] KIP-997: Partition-Level Throughput Metrics

2023-11-15 Thread Satish Duggana
Thanks Qichao for the KIP. +1 (binding) ~Satish. On Thu, 16 Nov 2023 at 02:20, Jorge Esteban Quilcate Otoya wrote: > > Qichao, thanks again for leading this proposal! > > +1 (non-binding) > > Cheers, > Jorge. > > On Wed, 15 Nov 2023 at 19:17, Divij Vaidya wrote: > > > +1 (binding) > > > > I wa

Re: [DISCUSS] KIP-963: Upload and delete lag metrics in Tiered Storage

2023-11-10 Thread Satish Duggana
Thanks Christo for the KIP and the interesting discussion. 101. Adding metrics at partition level may increase the cardinality of these metrics. We should be cautious of that and see whether they are really needed. RLM related operations do not generally affect based on partition(s) but it is most

Re: [ANNOUNCE] New Kafka PMC Member: Satish Duggana

2023-10-31 Thread Satish Duggana
ish! > > >>>> > > >>>> On Sat, Oct 28, 2023 at 12:59 AM Luke Chen wrote: > > >>>>> > > >>>>> Congrats Satish! > > >>>>> > > >>>>> Luke > > >>>>> > >

Re: [DISCUSS] Apache Kafka 3.5.2 release

2023-10-17 Thread Satish Duggana
Thanks Luke for volunteering for 3.5.2 release. On Tue, 17 Oct 2023 at 11:58, Josep Prat wrote: > > Hi Luke, > > Thanks for taking this one! > > Best, > > On Tue, Oct 17, 2023 at 8:12 AM Luke Chen wrote: > > > Hi all, > > > > I'd like to volunteer as release manager for the Apache Kafka 3.5.2, t

[jira] [Created] (KAFKA-15612) Followup on whether the indexes need to be materialized before they are passed to RSM for writing them to tiered storage.

2023-10-16 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15612: -- Summary: Followup on whether the indexes need to be materialized before they are passed to RSM for writing them to tiered storage. Key: KAFKA-15612 URL: https

Re: [ANNOUNCE] Apache Kafka 3.6.0

2023-10-12 Thread Satish Duggana
Luke Chen wrote: > > Thanks for running the release, Satish! > > BTW, 3.6.0 should be a major release, not a minor one. :) > > Luke > > On Wed, Oct 11, 2023 at 1:39 PM Satish Duggana wrote: > > > The Apache Kafka community is pleased to announce the release for &g

[jira] [Created] (KAFKA-15594) Add 3.6.0 to streams upgrade/compatibility tests

2023-10-11 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15594: -- Summary: Add 3.6.0 to streams upgrade/compatibility tests Key: KAFKA-15594 URL: https://issues.apache.org/jira/browse/KAFKA-15594 Project: Kafka Issue

[jira] [Created] (KAFKA-15593) Add 3.6.0 to broker/client upgrade/compatibility tests

2023-10-11 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15593: -- Summary: Add 3.6.0 to broker/client upgrade/compatibility tests Key: KAFKA-15593 URL: https://issues.apache.org/jira/browse/KAFKA-15593 Project: Kafka

[jira] [Created] (KAFKA-15576) Add 3.2.0 to broker/client and streams upgrade/compatibility tests

2023-10-10 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15576: -- Summary: Add 3.2.0 to broker/client and streams upgrade/compatibility tests Key: KAFKA-15576 URL: https://issues.apache.org/jira/browse/KAFKA-15576 Project

[ANNOUNCE] Apache Kafka 3.6.0

2023-10-10 Thread Satish Duggana
, Sambhav Jain, Satish Duggana, sciclon2, Shekhar Rajak, Sungyun Hur, Sushant Mahajan, Tanay Karmarkar, tison, Tom Bentley, vamossagar12, Victoria Xia, Vincent Jiang, vveicc, Walker Carlson, Yash Mayya, Yi-Sheng Lien, Ziming Deng, 蓝士钦 We welcome your help and feedback. For more information on how to

[jira] [Resolved] (KAFKA-15535) Add documentation of "remote.log.index.file.cache.total.size.bytes" configuration property.

2023-10-10 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana resolved KAFKA-15535. Resolution: Fixed > Add documentation of "remote.log.index.file.cache.total.si

Re: Apache Kafka 3.6.0 release

2023-10-08 Thread Satish Duggana
29, 2023 at 1:42 PM Ismael Juma wrote: > > > > > Thanks Satish! > > > > > > Ismael > > > > > > On Thu, Sep 28, 2023 at 6:39 AM Satish Duggana > > > > > wrote: > > > > > > > We do not have any pending release blockers f

Re: [VOTE] 3.6.0 RC2

2023-10-03 Thread Satish Duggana
Chandraprakash * Josep Prat * Proven Provenzano * Greg Harris 0 votes * No votes -1 votes * No votes I will continue with the release process and the release announcement will follow in the next few days. Thanks, Satish. On Tue, 3 Oct 2023 at 12:35, Satish Duggana wrote: > > Hi, >

Re: [VOTE] 3.6.0 RC2

2023-10-03 Thread Satish Duggana
an all integration tests for Connect and MM2 > > - Verified that the connect-test-plugins module is present in the staging > > Maven artifacts (https://issues.apache.org/jira/browse/KAFKA-15249) > > > > Everything looks good to me! > > > > +1 (binding) > > >

[jira] [Created] (KAFKA-15535) Add documentation of "remote.log.index.file.cache.total.size.bytes" configuration property.

2023-10-03 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15535: -- Summary: Add documentation of "remote.log.index.file.cache.total.size.bytes" configuration property. Key: KAFKA-15535 URL: https://issues.apache.org/jira/browse/K

Re: [VOTE] 3.6.0 RC2

2023-10-03 Thread Satish Duggana
gt; > > > > > > > > > > > > > On Mon, Oct 2, 2023 at 9:50 AM Federico Valeri < > > fedeval...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Hi Satish, I did the following to verif

Re: [VOTE] 3.6.0 RC2

2023-10-03 Thread Satish Duggana
/a900794ace4dcf1f9dadee27fbd8b63979532a18 > > > > > > > isn't available in documentation (cc: David). I don't consider > > > this as > > > > > a > > > > > > > release blocker but we should add it as a fast follow-up. > &g

Re: [VOTE] 3.6.0 RC2

2023-10-03 Thread Satish Duggana
javadocs, protocols. > > > 4. Verified the tiered storage feature works well. > > > > > > +1 (binding). > > > > > > Thanks. > > > Luke > > > > > > > > > > > > On Mon, Oct 2, 2023 at 5:23 AM Jakub Scholz wrote: >

[jira] [Created] (KAFKA-15530) Add missing documentation of metrics introduced as part of KAFKA-15196

2023-10-03 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15530: -- Summary: Add missing documentation of metrics introduced as part of KAFKA-15196 Key: KAFKA-15530 URL: https://issues.apache.org/jira/browse/KAFKA-15530 Project

[VOTE] 3.6.0 RC2

2023-09-29 Thread Satish Duggana
Hello Kafka users, developers and client-developers, This is the third candidate for the release of Apache Kafka 3.6.0. Some of the major features include: * KIP-405 : Kafka Tiered Storage * KIP-868 : KRaft Metadata Transactions * KIP-875: First-class offsets support in Kafka Connect * KIP-898: M

Re: Apache Kafka 3.6.0 release

2023-09-28 Thread Satish Duggana
> > Found a bug while testing TS feature in 3.6 - > > > > > https://issues.apache.org/jira/browse/KAFKA-15481 > > > > > > > > > > I don't consider it as a blocker for release since it's a concurrency > > > > > bug that sh

[jira] [Created] (KAFKA-15503) CVE-2023-40167, CVE-2023-36479 - Upgrade jetty to 9.4.52, 10.0.16, 11.0.16, 12.0.1

2023-09-25 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15503: -- Summary: CVE-2023-40167, CVE-2023-36479 - Upgrade jetty to 9.4.52, 10.0.16, 11.0.16, 12.0.1 Key: KAFKA-15503 URL: https://issues.apache.org/jira/browse/KAFKA-15503

Re: [kafka-clients] [VOTE] 3.6.0 RC1

2023-09-25 Thread Satish Duggana
snappy-java/security/advisories/GHSA-55g7-9cwv-5qfv > > > > > > > that will cause OOM error in the server. > > > > Kafka is also impacted by this vulnerability since it's like > > CVE-2023-34455 > > > > <https://nvd.nist.gov/vuln/detail/CV

Re: [kafka-clients] [VOTE] 3.6.0 RC1

2023-09-24 Thread Satish Duggana
x27;t be blocker in v3.6.0. > > > > > > For KAFKA-15489 <https://issues.apache.org/jira/browse/KAFKA-15489>, I'm > > > fine if we decide to fix it in v3.6.1/v3.7.0. > > > > > > +1 (binding) from me. > > > > > > T

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

2023-09-23 Thread Satish Duggana
Congratulations Justine!! On Sat, 23 Sept 2023 at 15:46, Bill Bejeck wrote: > > Congrats Justine! > > -Bill > > On Sat, Sep 23, 2023 at 6:23 PM Greg Harris > wrote: > > > Congratulations Justine! > > > > On Sat, Sep 23, 2023 at 5:49 AM Boudjelda Mohamed Said > > wrote: > > > > > > Congrats Just

Re: [kafka-clients] [VOTE] 3.6.0 RC1

2023-09-23 Thread Satish Duggana
ses > > - Run the unit tests > > - Run integration tests > > - Run the quickstart in KRaft and Zookeeper mode > > > > > > I +1 this release (non-binding) > > > > Thanks for your efforts! > > > > On Thu, Sep 21, 2023 at 2:59 AM Satish Duggana

Re: Apache Kafka 3.6.0 release

2023-09-21 Thread Satish Duggana
splayed. With this in mind, it seems to qualify pretty clearly > as a > > regression and we should not put out a release that includes it. > > > > * - Really, these aren't duplicates; rather, they're multiple copies of > the > > same plugin that come from diffe

Re: [ANNOUNCE] New committer: Lucas Brutschy

2023-09-21 Thread Satish Duggana
Congratulations Lucas! On Thu, 21 Sept 2023 at 22:58, Viktor Somogyi-Vass wrote: > > Congrats Lucas! > > On Thu, Sep 21, 2023 at 7:12 PM Alexander Sorokoumov > wrote: > > > Congratulations, Lucas! > > > > On Thu, Sep 21, 2023 at 10:09 AM Walker Carlson > > wrote: > > > > > Congrats Lucas! > > >

Re: [ANNOUNCE] New committer: Yash Mayya

2023-09-21 Thread Satish Duggana
Congratulations Yash!! On Thu, 21 Sept 2023 at 22:57, Viktor Somogyi-Vass wrote: > > Congrats Yash! > > On Thu, Sep 21, 2023 at 7:04 PM Josep Prat > wrote: > > > Congrats Yash! > > > > ——— > > Josep Prat > > > > Aiven Deutschland GmbH > > > > Alexanderufer 3-7, 10117 Berlin > > > > Amtsgericht C

Re: [kafka-clients] [VOTE] 3.6.0 RC1

2023-09-20 Thread Satish Duggana
uldn't consider them > > as blockers to make RC1 as the final release? > > > > [1] https://lists.apache.org/thread/cokoxzd0jtgjtrlxoq7kkzmvpm75381t > > > > On Wed, Sep 20, 2023 at 4:53 PM Satish Duggana > wrote: > > > > > > Hello Kafka

Re: [kafka-clients] [VOTE] 3.6.0 RC1

2023-09-20 Thread Satish Duggana
> > On Wed, Sep 20, 2023 at 4:53 PM Satish Duggana > wrote: > > > > Hello Kafka users, developers and client-developers, > > > > This is the second candidate for the release of Apache Kafka 3.6.0. Some > of the major features include: > > > > * KIP-405 : K

[jira] [Created] (KAFKA-15483) Update metrics documentation for the new metrics implemented as part of KIP-938

2023-09-20 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15483: -- Summary: Update metrics documentation for the new metrics implemented as part of KIP-938 Key: KAFKA-15483 URL: https://issues.apache.org/jira/browse/KAFKA-15483

[jira] [Resolved] (KAFKA-15092) KafkaClusterTestKit in test jar depends on MockFaultHandler

2023-09-20 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana resolved KAFKA-15092. Resolution: Invalid > KafkaClusterTestKit in test jar depends on MockFaultHand

[jira] [Resolved] (KAFKA-15482) kafka.utils.TestUtils Depends on MockTime Which is Not in Any Jar

2023-09-20 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana resolved KAFKA-15482. Resolution: Invalid > kafka.utils.TestUtils Depends on MockTime Which is Not in Any

[VOTE] 3.6.0 RC1

2023-09-20 Thread Satish Duggana
Hello Kafka users, developers and client-developers, This is the second candidate for the release of Apache Kafka 3.6.0. Some of the major features include: * KIP-405 : Kafka Tiered Storage * KIP-868 : KRaft Metadata Transactions * KIP-875: First-class offsets support in Kafka Connect * KIP-898:

Re: [VOTE] 3.6.0 RC0

2023-09-20 Thread Satish Duggana
Thanks Luke for reviewing and running the changes https://github.com/apache/kafka/pull/14409 for merging into 3.6. On Wed, 20 Sept 2023 at 06:09, Satish Duggana wrote: > > Hi David, > Thanks for the update. Please get it reviewed sooner. I plan to create > RC1 in the next 5 hrs. As

Re: [VOTE] 3.6.0 RC0

2023-09-19 Thread Satish Duggana
e/kafka/pull/14409 > > On Tue, Sep 19, 2023 at 12:57 PM Satish Duggana > wrote: > > > Hi All, > > I will start creating a new RC with the fix and send it for a vote by > > tomorrow as https://issues.apache.org/jira/browse/KAFKA-15473 is > > determined to be a re

Re: [VOTE] 3.6.0 RC0

2023-09-19 Thread Satish Duggana
Hi All, I will start creating a new RC with the fix and send it for a vote by tomorrow as https://issues.apache.org/jira/browse/KAFKA-15473 is determined to be a release blocker. Thanks. Satish. On Tue, 19 Sept 2023 at 22:26, Satish Duggana wrote: > > Thanks Chris for your comment

Re: [VOTE] 3.6.0 RC0

2023-09-19 Thread Satish Duggana
col documentation contains the newly introduced error code as > > > part of tiered storage > > > > > > 3. verified that public keys for RM are available at > > > https://keys.openpgp.org/ > > > > > > 4. verified that public keys for RM are available

Re: [VOTE] 3.6.0 RC0

2023-09-19 Thread Satish Duggana
e > > > do as part of the release. I think we should probably take this down until > > > we're done with the release, otherwise users stumbling on this post could > > > get confused. It also contains some broken links. > > > > > > Than

Re: Apache Kafka 3.6.0 release

2023-09-19 Thread Satish Duggana
onnector-plugins` endpoint lists duplicates which may > > cause confusion for users, or poor behavior in clients. > > Using the other REST API endpoints appears unaffected. > > I'll open a PR for this later today. > > > > Thanks, > > Greg > > >

Re: [VOTE] 3.6.0 RC0

2023-09-17 Thread Satish Duggana
On Sun, 17 Sept 2023 at 23:00, Satish Duggana wrote: > Hello Kafka users, developers and client-developers, > > This is the first candidate for the release of Apache Kafka 3.6.0. Some of > the major features include: > > * KIP-405 : Kafka Tiered Storage > * KIP-868 : KRaft

  1   2   3   4   5   >