Re: [ANNOUNCE] New Committer: Manikumar Reddy

Congratulations, Manikumar! On Thu, 11 Oct 2018, 23:29 Rajini Sivaram, wrote: > Congratulations, Manikumar! > > On Thu, Oct 11, 2018 at 6:57 PM Suman B N wrote: > > > Congratulations Manikumar! > > > > On Thu, Oct 11, 2018 at 11:09 PM Jason Gustafson > > wrote: > > > > > Hi all, > > > > > > Th

Request for wiki permission to create KIP

Hi, I would like to propose a KIP for KAFKA-7499 . Please grant me permission to create KIP. ID: ckamal

[DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

https://issues.apache.org/jira/browse/KAFKA-7499 All feedbacks will be highly appreciated. Thanks, Kamal Chandraprakash

Re: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

se un-intended results. On Wed, Dec 5, 2018 at 8:41 PM Matt Farmer wrote: > Hi there, > > Thanks for this KIP. > > What’s the thinking behind doing this in ProductionExceptionHandler versus > handling these cases in your serializer implementation? > > On Mon, Dec 3, 2018 at

Re: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

ond case seems to include the first one? > > > > -Matthias > > On 12/6/18 8:35 AM, Matt Farmer wrote: > > Ah, good point. > > > > Should we consider altering the serializer interface to permit not > sending > > the record? > > > > On Wed, Dec 5, 2018 at

Re: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions

l know which topic is affected > and can provide a corresponding implementation. Was just wondering if we > should be more strict? > > > -Matthias > > On 12/6/18 10:01 AM, Kamal Chandraprakash wrote: > > Matt, > > I agree with Matthias on not to altering the ser

Re: [VOTE] KIP-322: Return new error code for DeleteTopics API when topic deletion disabled.

+1 (non-binding) On Wed, Jul 4, 2018 at 5:22 PM Magnus Edenhill wrote: > +1 (non-binding) > > 2018-07-04 13:40 GMT+02:00 Satish Duggana : > > > +1 > > > > Thanks, > > Satish. > > > > On Wed, Jul 4, 2018 at 4:11 PM, Daniele Ascione > > wrote: > > > > > +1 > > > > > > Thanks, > > > Daniele > > >

Re: [VOTE] KIP-338 Support to exclude the internal topics in kafka-topics.sh command

Chia-Ping Tsai, The voting has passed. Dong Lin (binding) is a committer. On Wed, Jul 25, 2018 at 8:24 AM Chia-Ping Tsai wrote: > Current voting result is shown below. > > +2 (binding) > 1) Jason Gustafson > 2) Gwen Shapira > > +4 (non-binding) > 1) Colin McCabe > 2) Manikumar > 3) Dong Lin > 4

Re: [VOTE] KIP-289: Improve the default group id behavior in KafkaConsumer

+1 (non-binding) Thanks for the KIP. On Wed, Aug 8, 2018 at 3:11 PM Stanislav Kozlovski wrote: > +1 (non-binding) > Thanks! > > On Tue, Aug 7, 2018 at 11:47 PM Jason Gustafson > wrote: > > > +1 Thanks Vahid. > > > > On Tue, Aug 7, 2018 at 11:14 AM, Vahid S Hashemian < > > vahidhashem...@us.ibm

Re: [VOTE] KIP-336: Consolidate ExtendedSerializer/Serializer and ExtendedDeserializer/Deserializer

+1 Thanks for the KIP! On Wed, Aug 22, 2018 at 2:48 PM Viktor Somogyi-Vass wrote: > Hi All, > > I'd like to start a vote on this KIP ( > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=87298242) > which aims to refactor ExtendedSerializer/Serializer and > ExtendedDeserializer/D

Re: [ANNOUNCE] New committer: Colin McCabe

Congratulations Colin! On Tue, Sep 25, 2018 at 7:42 PM Dhruvil Shah wrote: > Congratulations Colin! > > On Tue, Sep 25, 2018 at 6:47 AM Stephane Maarek < > steph...@simplemachines.com.au> wrote: > > > Congrats Colin ! > > > > On Tue., 25 Sep. 2018, 3:33 pm Bill Bejeck, wrote: > > > > > Congrats

Re: Re run build of a pull request

No need to push a commit to run the test case. Type "retest this please" as comment. It will again run all the test cases. On Wed, Oct 3, 2018 at 1:12 PM Suman B N wrote: > Team, > > Every once in a while, PR builds fails due to below test failure with Null > Pointer Exception. > >- org.apac

Re: [ANNOUNCE] New Committer: Bill Bejeck

Congratulations Bill! On Thu, Feb 14, 2019 at 3:55 PM Ivan Ponomarev wrote: > Congratulations, Bill! > > Your 'Kafka Streams in Action' is a great book. These months it is > always travelling with me in my backpack with my laptop )) > > Regards, > > Ivan > > 14.02.2019 3:56, Guozhang Wang пишет:

Re: [VOTE] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

Stanislav, Any updates on this KIP? We have internal users who want to skip the corrupted message while consuming the records. On Fri, Oct 19, 2018 at 11:34 PM Matthias J. Sax wrote: > I am not 100% familiar with the details of the consumer code, however I > tend to disagree with: > > > There'

Re: [VOTE] KIP-334 Include partitions in exceptions raised during consumer record deserialization/validation

mp up this discussion? I think we can have a single exception type FaultyRecordException/RecordDeserialization exception to capture both the errors. We can add an additional enum field to differentiate the errors if required. Thanks, Kamal Chandraprakash On Wed, Apr 24, 2019 at 1:49 PM

Re: [VOTE] KIP-453: Add close() method to RocksDBConfigSetter

+1 (non-binding). Thanks for the KIP. On Tue, May 7, 2019 at 3:50 AM Bill Bejeck wrote: > Thanks for the KIP Sophie. > > +1(binding) > > On Mon, May 6, 2019 at 4:51 PM John Roesler wrote: > > > Thanks, Sophie, I reviewed the KIP, and I agree this is the best / > > only-practical approach. > > >

Re: [VOTE] KIP-440: Extend Connect Converter to support headers

+1 (non-binding). Thanks for the KIP! On Sun, May 19, 2019 at 6:36 PM Dongjin Lee wrote: > +1 (non-binding). > > Binding: +2 (Randall, Gwen) > Non-binding: +2 (Andrew, Dongjin) > > We need one more +1 from the committers. Is there anyone else? > > Thanks, > Dongjin > > On Fri, May 10, 2019 at 12

Re: Contributor Apply

nit: You can *now* self-assign tickets. On Fri, May 24, 2019 at 11:30 PM Matthias J. Sax wrote: > Added you to the list on contributors. You can not self-assign tickets. > > Please read > https://cwiki.apache.org/confluence/display/KAFKA/Contributing+Code+Changes > to get started. > > > > -Matth

Re: [DISCUSS] KIP-467: Augment ProduceResponse error messaging

This KIP is inline with KIP-334 in which it's proposed that the consumer will throw FaultyRecordException with corrupt/in-operative record offset and topic-partition on encountering the invalid record. In this KIP proposed

Re: [VOTE] KIP-434: Dead replica fetcher and log cleaner metrics

+1 (non-binding). Thanks for the KIP! On Thu, Jun 6, 2019 at 8:12 PM Andrew Schofield wrote: > +1 (non-binding) > > Andrew > > On 06/06/2019, 15:15, "Ryanne Dolan" wrote: > > +1 (non-binding) > > Thanks > Ryanne > > On Wed, Jun 5, 2019, 9:31 PM Satish Duggana > wrote: > >

Re: [VOTE] 2.3.0 RC3

+1 (non-binding) * Ran unit and integration test on 2.11 and 2.12 * Verified quick start * Ran internal apps on the 3 node cluster On Thu, Jun 20, 2019 at 3:33 AM Colin McCabe wrote: > Hi all, > > We discovered some problems with the second release candidate (RC2) of > 2.3.0. Specifically, KAF

Re: [DISCUSS] KIP-490: log when consumer groups lose a message because offset has been deleted

Jose, How do you differentiate the compaction topics from the time retention topics? Deleting a message due to compaction policy is a valid case and users won't be interested in monitoring/reading those deleted messages. Thanks, Kamal On Tue, Jul 23, 2019 at 4:00 AM Jose M wrote: > Hi Coli

Re: Re: [DISCUSS] KIP-195: AdminClient.increasePartitions

Selvamalar, It's a self service. Please send a mail to ` dev-unsubscr...@kafka.apache.org` and `users-unsubscr...@kafka.apache.org` to unsubscribe from the `dev On Fri, Sep 8, 2017 at 10:02 PM, Selvamalar Arunkumar < selvamalar.arunku...@walmart.com> wrote: > Please remove my email id from t

Re: [VOTE] KIP-520: Augment Consumer.committed(partition) to allow multiple partitions

Thanks for the KIP! LGTM, +1 (non-binding). On Wed, Sep 11, 2019 at 3:23 AM Matthias J. Sax wrote: > I don't have a strong preference. So I am also fine to deprecate the > existing methods. Let's see what Jason thinks. > > Can you update the KIP to reflect the semantics of the return `Map` (ie,

Re: [VOTE] KIP-525 - Return topic metadata and configs in CreateTopics response

+1 (non-binding), Thanks for the KIP! On Mon, Sep 23, 2019 at 10:45 AM Satish Duggana wrote: > Thanks for the KIP, +1 (non binding) > > This looks to be a useful API for admin tools. When we worked on an > admin tool for Kafka, we had to explicitly call the config API for > showing the configs f

Re: [ANNOUNCE] New committer: Mickael Maison

Congrats Mickael! On Sat, Nov 9, 2019 at 4:47 AM Sean Glover wrote: > Congratulations Mickael :) > > On Fri, Nov 8, 2019 at 12:28 PM Ankit Kumar > wrote: > > > Congratulations Mickael!! > > > > *Best regards,* > > *Ankit Kumar.* > > > > > > On Fri, Nov 8, 2019 at 9:08 PM Viktor Somogyi-Vass < >

Re: [ANNOUNCE] New committer: John Roesler

Congrats John! On Wed, Nov 13, 2019 at 7:57 AM Dong Lin wrote: > Congratulations John! > > On Tue, Nov 12, 2019 at 1:56 PM Guozhang Wang wrote: > > > Hi Everyone, > > > > The PMC of Apache Kafka is pleased to announce a new Kafka committer, > John > > Roesler. > > > > John has been contributing

Re: [VOTE] KIP-635: GetOffsetShell: support for multiple topics and consumer configuration override

+1 (non-binding), thanks for the KIP! On Thu, Jul 30, 2020 at 3:31 PM Manikumar wrote: > +1 (binding) > > Thanks for the KIP! > > > > On Thu, Jul 30, 2020 at 3:07 PM Dániel Urbán > wrote: > > > Hi everyone, > > > > If you are interested in this KIP, please do not forget to vote. > > > > Thanks,

Re: [DISCUSS] KIP-930: Tiered Storage Metrics

Hi Abhijeet, Thanks for the KIP! We are changing the metric names from what was proposed in the KIP-405 and adding new metrics for RemoteIndexCache. In the KIP, it's not clear whether we are renaming the aggregate broker level metrics for remote copy/fetch/failed-copy/failed-fetch. Are these met

Re: [VOTE] KIP-930: Tiered Storage Metrics

+1 (non-binding) -- Kamal On Tue, Jul 25, 2023 at 11:30 AM Abhijeet Kumar wrote: > Hi All, > > I would like to start the vote for KIP-930 Tiered Storage Metrics. > > The KIP is here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-930%3A+Tiered+Storage+Metrics > > Regards > Abhijeet.

Re: [VOTE] KIP-959 Add BooleanConverter to Kafka Connect

+1 (non-binding). Thanks for the KIP! On Tue, Jul 25, 2023 at 11:12 PM Yash Mayya wrote: > Hi Hector, > > Thanks for the KIP! > > +1 (non-binding) > > Thanks, > Yash > > On Tue, Jul 25, 2023 at 11:01 PM Andrew Schofield < > andrew_schofield_j...@outlook.com> wrote: > > > Thanks for the KIP. As y

Re: [DISCUSS] KIP-963: Upload and delete lag metrics in Tiered Storage

Hi Christo, Thanks for the KIP! The proposed tiered storage metrics are useful. The unit mentioned in the KIP is the number of records. Each topic can have varying amounts of records in a segment depending on the record size. Do you think having the tier-lag by number of segments (or) size of se

Re: [DISCUSS] KIP-972: Add the metric of the current running version of kafka

Hi Hudeqi, Kafka already emits the version metric. Can you check whether the below metric satisfies your requirement? kafka.server:type=app-info,id=0 -- Kamal On Mon, Aug 28, 2023 at 2:29 PM hudeqi <16120...@bjtu.edu.cn> wrote: > Hi, all, I want to submit a minor kip to add a metric, which sup

Re: [ANNOUNCE] New committer: Yash Mayya

Congratulations Yash! On Thu, Sep 21, 2023, 22:03 Bill Bejeck wrote: > Congrats Yash! > > On Thu, Sep 21, 2023 at 12:26 PM Divij Vaidya > wrote: > > > Congratulations Yash! > > > > Divij Vaidya > > > > > > On Thu, Sep 21, 2023 at 6:18 PM Sagar wrote: > > > > > > Congrats Yash ! > > > On Thu, 2

Re: [ANNOUNCE] New committer: Lucas Brutschy

Congrats Lucas! On Thu, Sep 21, 2023, 22:05 Boudjelda Mohamed Said wrote: > Congratulations, Lucas!! > > On Thu 21 Sep 2023 at 18:34, Lianet M. wrote: > > > Congratulations Lucas! > > > > On Thu, Sept 21, 2023, 11:45 a.m. Bruno Cadonna > > wrote: > > > > > Hi all, > > > > > > The PMC of Apache

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

Congrats Justine! On Sat, Sep 23, 2023, 13:28 Divij Vaidya wrote: > Congratulations Justine! > > On Sat 23. Sep 2023 at 07:06, Chris Egerton > wrote: > > > Congrats Justine! > > On Fri, Sep 22, 2023, 20:47 Guozhang Wang > > wrote: > > > > > Congratulations! > > > > > > On Fri, Sep 22, 2023 at

Re: [DISCUSS] KIP-979: Allow independently stop KRaft controllers or brokers

Hi Hailey, Thanks for working on this! This is one of the long-standing open issues. Now, users have to find the PID of the respective Kafka process to stop if more than one node is being run locally for testing purposes. The updated KIP is addressing that. LGTM. Is `node.id` and `broker.id` the

Re: [VOTE] KIP-979 Allow independently stop KRaft processes

+1 (non-binding). Thanks for the KIP! On Mon, Oct 23, 2023, 12:55 Hailey Ni wrote: > Hi all, > > I'd like to call a vote on KIP-979 that will allow users to independently > stop KRaft processes. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-979%3A+Allow+independently+stop+KRaft+pro

Re: [ANNOUNCE] New Kafka PMC Member: Satish Duggana

Congratulations Satish! On Fri, Oct 27, 2023, 21:10 Jorge Esteban Quilcate Otoya < quilcate.jo...@gmail.com> wrote: > Congratulations Satish!! > > On Fri, 27 Oct 2023 at 18:38, Mickael Maison > wrote: > > > Congratulations Satish! > > > > On Fri, Oct 27, 2023 at 5:18 PM Lucas Brutschy > > wrote

Re: [DISCUSS] KIP-963: Upload and delete lag metrics in Tiered Storage

> > > > On Mon, 25 Sept 2023 at 10:49, Gantigmaa Selenge < > > gsele...@redhat.com> > > > > > wrote: > > > > > > > > > >> Hi Christo, > > > > >> > > > > >> Thank you for writing the KIP.

Re: [VOTE] KIP-997: Partition-Level Throughput Metrics

+1 (non-binding). Thanks for the KIP! On Thu, Nov 16, 2023 at 9:00 AM Satish Duggana wrote: > Thanks Qichao for the KIP. > > +1 (binding) > > ~Satish. > > On Thu, 16 Nov 2023 at 02:20, Jorge Esteban Quilcate Otoya > wrote: > > > > Qichao, thanks again for leading this proposal! > > > > +1 (non-

Re: RemoteStorageManager.fetchLogSegment - how to deal with InterruptedException?

Hi Ivan, I've opened a relevant patch to increase the timeout to fetch data from remote storage. This will reduce the error occurrence rate: https://github.com/apache/kafka/pull/14778 I think it's better to distinguish the cases: 1) Any error that happens while reading the remote data -- this s

Re: [DISCUSS] KIP-1005: Add EarliestLocalOffset to GetOffsetShell

Hi Christo, Thanks for the KIP! Similar to the earliest-local-log offset, can we also expose the highest-copied-remote-offset via GetOffsetShell tool? This will be useful during the debugging session. On Mon, Nov 20, 2023 at 5:38 PM Christo Lolov wrote: > Hello all! > > I would like to start

Re: [VOTE] KIP-963: Additional metrics in Tiered Storage

+1 (non-binding). Thanks for the KIP! On Tue, Nov 21, 2023, 03:04 Divij Vaidya wrote: > + 1 (binding) > > This Kip will greatly improve Tiered Storage troubleshooting. Thank you > Christo. > > On Mon 20. Nov 2023 at 17:21, Christo Lolov > wrote: > > > Hello all! > > > > Now that the discussion

[DISCUSS] KIP-1007: Introduce Remote Storage Not Ready Exception

Hi, I would like to start a discussion to introduce a new error code for retriable remote storage errors. Please take a look at the proposal: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1007%3A+Introduce+Remote+Storage+Not+Ready+Exception

Re: [VOTE] 3.6.1 RC0

+1 (non-binding) 1. Built the source from 3.6.1-rc0 tag in scala 2.12 and 2.13 2. Ran all the unit and integration tests. 3. Ran quickstart and verified the produce-consume on a 3 node cluster. 4. Verified the tiered storage functionality with local-tiered storage. On Tue, Nov 28, 2023 at 12:55 A

Re: [DISCUSS] KIP-1018: Introduce max remote fetch timeout config

to not see the metric in the > RequestMetrics. > > Your thoughts? > > Best, > Christo > > On Sun, 28 Apr 2024 at 10:52, Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > > > Christo, > > > > Updated the KIP with the remote fetch lat

Re: [DISCUSS] KIP-1018: Introduce max remote fetch timeout config

ere, is it > > possible > > > to add 2 metrics, one at the RequestMetrics level (even if it is > > > total-time-ms - (all other times)) and one at what you are proposing? > As > > an > > > operator I would find it strange to not see the metric in the > > &g

Re: [DISCUSS] KIP-1018: Introduce max remote fetch timeout config

Hi all, If there are no more comments, I'll open a VOTE thread. -- Kamal On Sat, May 4, 2024 at 8:39 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Luke, > > Thanks for the review! > > DelayedFetch and DelayedRemoteFetch are orthogonal calls &g

[VOTE] KIP-1018: Introduce max remote fetch timeout config

Hi all, We would like to start a voting thread for KIP-1018: Introduce max remote fetch timeout config for DelayedRemoteFetch requests. The KIP is available on https://cwiki.apache.org/confluence/display/KAFKA/KIP-1018%3A+Introduce+max+remote+fetch+timeout+config+for+DelayedRemoteFetch+requests

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

ents to delete is huge. We can further extend the same approach for the topic deletion requests. Also, Could you please update the state diagram about the transitions? It is not clear when to transit from DISABLING to DISABLED state? -- Kamal On Mon, May 6, 2024 at 6:55 PM Kamal Chandraprakash &

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

Deletion when the policy is retain happens asynchronously due to > expiration. I think that deletion when the policy is delete ought to (at > least for the initial implementation) happen synchronously. Should people > run into timeout problems we can always then have a follow-up KIP wher

Re: [VOTE] KIP-1018: Introduce max remote fetch timeout config

; wrote: > > > > > +1 non binding > > > > > > Thanks > > > > > > On Thu, May 9, 2024 at 12:05 PM Luke Chen wrote: > > > > > > > > Hi Kamal, > > > > > > > > Thanks for the KIP! > >

Re: [VOTE] KIP-950: Tiered Storage Disablement

Hi Christo, Thanks for the KIP and incorporating the review comments. Please update the KIP with the latest details. +1 (non-binding). Thanks, Kamal On Wed, May 15, 2024 at 3:18 PM Luke Chen wrote: > Hi Christo, > > In addition to the minor comments left in the discussion thread, it LGTM. >

[DISCUSS] RemoteLogSegmentMetadataRecord Schema Evolution

Hi, We have defined the remote log segment metadata schemas with type as "metadata" (controller records) in #10271 : 1. RemoteLogSegmentMetadataRecord.json

Re: [DISCUSS] RemoteLogSegmentMetadataRecord Schema Evolution

Ignore the above message. Went over the Kafka message definition README <https://github.com/apache/kafka/blob/trunk/clients/src/main/resources/common/message/README.md> guide and learnt about the tagged fields. On Tue, Jun 11, 2024 at 10:32 PM Kamal Chandraprakash < kamal.chandraprak...@

[DISCUSS] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

Hi all, I have opened a KIP-1058 to reduce the pressure on remote storage when transactional consumers are reading non-txn topics from remote storage. https://

Re: [DISCUSS] KIP-1057: Add remote log metadata flag to the dump log tool

We can use the console-consumer to read the contents of the `__remote_log_metadata` topic. Why are we proposing a new tool? sh kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic __remote_log_metadata --consumer-property exclude.internal.topics=false --formatter org.apache.kafka.s

Re: [DISCUSS] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

arameter instead of > "topicIdPartition"? Suggesting because isTxnIdEmpty is not a property of a > partition, instead it's a property of a specific segment. > > Looking forward to hearing your thoughts about the alternatives. Let's get > this fixed. > > -- &g

Re: [DISCUSS] KIP-1057: Add remote log metadata flag to the dump log tool

e dump-logs tool with custom decoder > > - Running kafka-consumer.sh on the topic. > > > > ~Satish. > > > > > > ~Satish. > > > > > > > > On Mon, 17 Jun 2024 at 15:55, Federico Valeri > > wrote: > > > > > > Hi Kamal,

Re: [VOTE] KIP-1057: Add remote log metadata flag to the dump log tool

Hi Federico, Thanks for the KIP! +1 from me. On Fri, Jun 21, 2024 at 5:47 PM Luke Chen wrote: > Hi Fede, > > Thanks for the KIP! > +1 from me. > > Luke > > On Fri, Jun 21, 2024 at 6:44 PM Federico Valeri > wrote: > > > Hi all, I'd like to kick off a vote on KIP-1057. > > > > Design doc: > > >

Re: [DISCUSS] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

Bump. Please review this proposal. On Mon, Jun 17, 2024 at 6:55 PM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Divij, > > Thanks for the review! Updated the KIP with 1, 2, 3, and 4 review > comments. > > > 4. Potential alternative - Instead of h

Re: [DISCUSS] KIP-1051 Statically configured log replication throttling

Hi Harry Fallows, Thanks for the KIP! I went over both the KIP-1051 and KIP-1009. Assuming that the leader.replication.throttled.replicas and follower.replication.throttled.replicas are set to Wildcard (*) to apply for all the partitions in the broker. If we set a static value for leader and foll

Re: [DISCUSS] KIP-1066: Mechanism to cordon brokers and log directories

Hi Mickael, Thanks for the KIP! This is a useful feature which helps to decommission the nodes by essentially creating a new replica exclude broker list. To cordon a list of brokers, we have to apply the config on each of the broker nodes and similarly to see the list of cordoned brokers, we hav

Re: [DISCUSS] KIP-1066: Mechanism to cordon brokers and log directories

rectories cordoned, and > > you still need to describe a specific broker's configuration to find > > the "name" of a log directory you want to cordon. > > > > I think an easy way to get an overall view of the cordoned log > > directories/brokers will

Re: [DISCUSS] KIP-1051 Statically configured log replication throttling

about having it as a gauge for number of partitions that are > currently leader of follower throttled (similar to the URP metric)? > > Kind regards, > Harry > > On Thursday, 11 July 2024 at 19:02, Kamal Chandraprakash < > kamal.chandraprak...@gmail.co

Re: [VOTE] KIP-950: Tiered Storage Disablement

Hi Chia-Ping, Thanks for the review! > If so, what is the purpose of `remote.log.delete.on.disable=false`? IIUC, the purpose of `remote.log.delete.on.disable` is to get explicit confirmation from the user before deleting the remote log segments. The concern raised in the thread is that if the u

Re: [VOTE] KIP-950: Tiered Storage Disablement

Correction: (2): Wait for all the remote segments to be deleted async due to breach by retention time (or) size, then set the `remote.storage.enable = false` and `remote.log.delete.on.disable = true`. This step is optional. On Thu, Jul 25, 2024 at 11:13 PM Kamal Chandraprakash

Re: [VOTE] KIP-950: Tiered Storage Disablement

et `remote.copy.disabled= > true`. If you want to disable remote storage and delete all remote data, > please set `remote.storage.enable=false,remote.log.delete.on.disable=true`. > > I've updated the KIP. Please take a look when available. > > https://cwiki.apache.org/confl

Re: [kafka-clients] [ANNOUNCE] Apache Kafka 3.8.0

wxy, Dung Ha, Edoardo Comar, Eduwer > >> Camacaro, Emanuele Sabellico, Erik van Oosten, Eugene Mitskevich, Fan > >> Yang, Federico Valeri, Fiore Mario Vitale, flashmouse, Florin Akermann, > >> Frederik Rouleau, Gantigmaa Selenge, Gaurav Narula, ghostspiders, > >>

Re: [DISCUSS] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

we would not have to > iterate through the metadata for the dominant case where the partition had > no transactions at all. > > -- > Divij Vaidya > > > > On Tue, Jun 25, 2024 at 11:42 AM Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > &g

[DISCUSS] KIP-1075: Introduce delayed remote list offsets purgatory to make LIST_OFFSETS async

Hi all, I would like to start a discussion thread on KIP-1075 to make the remote LIST_OFFSETS an async operation. The KIP is here: https://cwiki.apache.org/co

Re: [DISCUSS] KIP-1075: Introduce delayed remote list offsets purgatory to make LIST_OFFSETS async

Bumping this thread. Please take a look. On Fri, Aug 2, 2024 at 12:32 PM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi all, > > I would like to start a discussion thread on KIP-1075 > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-1075%3A+Int

Re: [VOTE] KIP-950: Tiered Storage Disablement

which > > means `retention.ms` value to be used) > > > > So, basically, we don't change the original design, just want to make > sure > > users are aware of the retention policy change after disabling remote log > > copy. > > > > Let me know if you have

Re: [DISCUSS] KIP-1075: Introduce delayed remote list offsets purgatory to make LIST_OFFSETS async

Hi Luke, Thanks for the review! LC1: When the consumer starts to read data, then it might need the below offsets: earliest, latest, and last-committed-offset based on the "auto.offset.reset" config. The earliest and latest offsets have special timestamps -2 and -1, those timestamp corresponding

Re: [DISCUSS] KIP-1075: Introduce delayed remote list offsets purgatory to make LIST_OFFSETS async

receives the "response" from > server? > > LC4: I found the different consumer and admin behavior when setting > "request.timeout" and "default.api.timeout" is confusing. Are they expected > or a bug? > > Thank you. > Luke > > > On Thu, Aug 8

Re: [DISCUSS] KIP-1058: Txn consumer exerts pressure on remote storage when reading non-txn topic

Bumping this thread for KIP review! We can go for the simplest solution that is proposed in this KIP and it can be improved in the subsequent iteration. PTAL. Thanks, Kamal On Fri, Aug 2, 2024 at 11:42 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi Divij, >

Re: [DISCUSS] KIP-1075: Introduce delayed remote list offsets purgatory to make LIST_OFFSETS async

we be consistent to use > `reader`, instead of `read`? > > LC7: We should set a default value for the newly introduced configs and > written in KIP. > > Thanks. > Luke > > On Tue, Aug 13, 2024 at 8:47 PM Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wro

Re: [DISCUSS] KIP-1075: Introduce delayed remote list offsets purgatory to make LIST_OFFSETS async

Hi all, > I saw we added some new configs/metrics. I have removed the recent changes to the public interfaces to limit the scope of the KIP to minimum. PTAL. Thanks, Kamal On Wed, Aug 14, 2024 at 9:58 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi Luk

Re: [DISCUSS] KIP-1075: Introduce delayed remote list offsets purgatory to make LIST_OFFSETS async

Hi all, If there are no more comments, I'll start a vote soon. Thanks, Kamal On Wed, Aug 14, 2024 at 5:05 PM Luke Chen wrote: > Hi Kamal, > > Thanks for the update. > LGTM. > > Luke > > On Wed, Aug 14, 2024 at 7:25 PM Kamal Chandraprakash < > kamal.chandr

[VOTE] KIP-1075: Introduce delayed remote list offsets purgatory to make LIST_OFFSETS async

Hi, I would like to open voting for KIP-1075. I've addressed the review comments in the discussion thread. Please vote if the proposal looks good. https://cwiki.apache.org/confluence/plugins/servlet/mobile?contentId=315494171#content/view/315494171 Thanks, Kamal

Re: [DISCUSS] KIP-1007: Introduce Remote Storage Not Ready Exception

Thanks Luke for reviewing this KIP! If there are no more comments from others, I'll start the VOTE since this is a minor KIP. On Mon, Dec 11, 2023 at 1:01 PM Luke Chen wrote: > Hi Kamal, > > Thanks for the KIP! > LGTM. > > Thanks. > Luke > > On Wed, Nov 22, 2023

[VOTE] KIP-1007: Introduce Remote Storage Not Ready Exception

Hi, I would like to call a vote for KIP-1007 . This KIP aims to introduce a new error code for retriable remote storage errors. Thanks to everyone who reviewed the KIP! -- Kamal

Re: [DISCUSS] KIP-1007: Introduce Remote Storage Not Ready Exception

d on a partition. > > [1] > > https://github.com/apache/kafka/blob/82808873cbf6a95611243c2e7984c4aa6ff2cfff/core/src/main/scala/kafka/log/UnifiedLog.scala#L1336 > > > -- > Divij Vaidya > > > > On Tue, Dec 12, 2023 at 9:58 AM Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: &

Re: DISCUSS KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

Hi Ziming, Thanks for the KIP! The proposal LGTM. I'm also inclined towards option 2 (i.e. add an explicit --enable-incremental flag in 3.X version) to avoid any incompatible change in v3.X. As mentioned in this thread, many users might be using external tools to do the topic rebalance and apply

Re: [VOTE] KIP-1005: Expose EarliestLocalOffset and TieredOffset

+1 (non-binding). Thanks for the KIP! -- Kamal On Thu, Dec 21, 2023 at 2:23 PM Christo Lolov wrote: > Heya all! > > KIP-1005 ( > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1005%3A+Expose+EarliestLocalOffset+and+TieredOffset > ) > has been open for around a month with no further com

Re: [ANNOUNCE] New Kafka PMC Member: Divij Vaidya

Congrats Divij! On Thu, Dec 28, 2023, 07:09 Kirk True wrote: > Congrats Divij!!! > > On Wed, Dec 27, 2023, at 1:44 PM, Jorge Esteban Quilcate Otoya wrote: > > Congratulations Divij!! > > > > On Wed 27. Dec 2023 at 14.56, Tom Bentley wrote: > > > > > Congratulations! > > > > > > On Thu, 28 Dec 2

Re: [VOTE] KIP-1013: Drop broker and tools support for Java 11 in Kafka 4.0 (deprecate in 3.7)

+1 (non-binding). On Wed, Jan 3, 2024 at 8:01 AM Satish Duggana wrote: > Thanks Ismael for the proposal. > > Adopting JDK 17 enhances developer productivity and has reached a > level of maturity that has led to its adoption by several other major > projects, signifying its reliability and effect

Re: [VOTE] KIP-1007: Introduce Remote Storage Not Ready Exception

as it will be a more explicit > > > > behaviour than what Kafka does today - I am happy with it! > > > > > > > > +1 (non-binding) from me > > > > > > > > Best, > > > > Christo > > > > > > > > On Tue, 1

Re: [VOTE] KIP-1011: Use incrementalAlterConfigs when updating broker configs by kafka-configs.sh

+1 (non-binding) On Mon, Jan 22, 2024 at 8:34 AM ziming deng wrote: > Hello everyone, > I'd like to initiate a vote for KIP-1011. > This KIP is about replacing alterConfigs with incrementalAlterConfigs when > updating broker configs using kafka-configs.sh, this is similar to what we > have done

[DISCUSS] KIP-1018: Introduce max remote fetch timeout config

Hi all, I have opened a KIP-1018 to introduce dynamic max-remote-fetch-timeout broker config to give more control to the operator. https://cwiki.apache.org/con

Re: [DISCUSS] KIP-1018: Introduce max remote fetch timeout config

pache.org/jira/browse/KAFKA-15776 > [2] https://github.com/apache/kafka/pull/14778#issuecomment-1820588080 > > On Tue, 30 Jan 2024 at 01:01, Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > > > Hi all, > > > > I have opened a KIP-1018 > > &

Re: [ANNOUNCE] Apache Kafka 3.7.0

Alok Thatikunta, Alyssa Huang, Aman Singh, Andras Katona, Andrew > > > > Schofield, Anna Sophie Blee-Goldman, Anton Agestam, Apoorv Mittal, > > > > Arnout Engelen, Arpit Goyal, Artem Livshits, Ashwin Pankaj, > > > > ashwinpankaj, atu-sharm, bachmanit

Re: [VOTE] KIP-956: Tiered Storage Quotas

+1 (non-binding), Thanks for the KIP, Abhijeet! -- Kamal On Fri, Mar 8, 2024 at 11:02 PM Jun Rao wrote: > Hi, Abhijeet, > > Thanks for the KIP. +1 > > Jun > > On Fri, Mar 8, 2024 at 3:44 AM Abhijeet Kumar > wrote: > > > Hi All, > > > > I would like to start the vote for KIP-956 - Tiered Storag

Re: [DISCUSS] Minimum constraint for segment.ms

One use case I see for setting the `segment.bytes` to 1 is to delete all the records from the topic. We can mention about it in the doc to use the `kafka-delete-records` API instead. On Wed, Mar 13, 2024 at 6:59 PM Divij Vaidya wrote: > + users@kafka > > Hi users of Apache Kafka > > With the

Re: [DISCUSS] KIP-1026: Handling producer snapshot when upgrading from < v2.8.0 for Tiered Storage

Hi Arpit, Thanks for the KIP! There is an open ticket [1] to generate the empty producer snapshot for segments which lacks one. Tiered storage is supported from IBP 2.8-IV1 which mandates that all the segments should have the producer snapshots. If we make the producer snapshot optional, then we

Re: [DISCUSS] KIP-1026: Handling producer snapshot when upgrading from < v2.8.0 for Tiered Storage

required for topics migrated from < 2.8 > > version. This will not break the existing contract with the plugin. Yes, > > metrics do not make sense here as of now. > > Greg, @Kamal Chandraprakash WDYT ? > > Arpit Goyal > > 8861094754 > > > > > > On S

Re: [ANNOUNCE] New committer: Christo Lolov

Congrats Christo! On Tue, Mar 26, 2024, 17:42 Bruno Cadonna wrote: > Congrats, Christo! > > Well deserved! > > Best, > Bruno > > On 3/26/24 1:04 PM, Luke Chen wrote: > > Hi, Everyone, > > > > The PMC of Apache Kafka is pleased to announce a new Kafka committer: > > Christo Lolov. > > > > Christo

Re: [DISCUSS] KIP-950: Tiered Storage Disablement

Hi, Thanks for the KIP! Overall the KIP looks good and covered most of the items. 1. Could you explain how the brokers will handle the DisableRemoteTopic API request? 2. Who will initiate the controller interaction sequence? Does the controller listens for topic config updates and initiate the d

Re: [DISCUSS] KIP-1021: Allow to get last stable offset (LSO) in kafka-get-offsets.sh

Hi Ahmed, Thanks for the KIP! It will be a useful addition to know the last-stable-offset (LSO) for a partition. Motivation section is not clear to me. Can you document the scenarios on how the exposed LSO can be used for txn support/debug? -- Kamal On Thu, Mar 21, 2024 at 11:30 PM Justine Olsha

  1   2   3   >