Kafka point in time recovery?

Is there any possibility of recovering kafka topic and brokers point in time?. I want to recover kafka topics and brokers as of yesterday 5 PM. I dont want any data arrived in kafka yesterday after 5 PM. I read about mirroring of data using kafka replicator, ureplicator..etc..All the mirroring too

Kafka JIRAs Waiting for Review

KafkaSpout not consuming the first uncommitted offset data from kafka

Hi Kafka, I have a trident topology in storm which consumes data from kafka. Now i am seeing an issue in KafkaSpout. This is not consuming the very first tthe first uncommitted offset data from kafka. My storm version is 1.1.1 and kafka version is 0.11.0.0. I have a topic say X and partition of t

Kafka ERROR after upgrade to Kafka 1.0.0 version : java.lang.OutOfMemoryError: Java heap space

ack$1(KafkaApis.sca la:569) at kafka.server.KafkaApis.$anonfun$handleFetchRequest$15(KafkaA pis.scala:588) at kafka.server.KafkaApis$$Lambda$662/1703788464.apply$mcVI$sp(Unknown Source) Thanks, Vinay Kumar

Re: [ANNOUNCE] Apache Kafka 2.2.0

Congratulations for the 2.2.0 release! Regards, Sanjeev On Wed, Mar 27, 2019 at 6:43 AM Guozhang Wang wrote: > Thanks Matthias for the release! > > On Tue, Mar 26, 2019 at 9:31 PM Dongjin Lee wrote: > > > Congratulations and thanks for your great work, Matthias!! > > > > Best, > > Dongjin > >

JIRA Contributor permissions

Hi, I want to start contributing to Apache Kafka.Can you please add me into JIRA contributor list? Thanks, Vikash

Re: JIRA Contributor permissions

Sorry I forgot to mention my user ID. My User ID : krvikash On Fri, Jun 7, 2019 at 10:12 PM Vikash Kumar wrote: > Hi, > > I want to start contributing to Apache Kafka.Can you please add me into > JIRA contributor list? > > Thanks, > Vikash >

Partition Reassignment in Cloud

Hi I have been trying a small experiment with partition reassignment in cloud. where instead of copying data between brokers using network, I moved the disk between the 2 brokers and ran the partition reassignment. This actually increased the speed of partition reassignment significantly. (As i

Partition Reassignment in Cloud

Hi I have been trying a small experiment with partition reassignment in cloud. where instead of copying data between brokers using network, I moved the disk between the 2 brokers and ran the partition reassignment. This actually increased the speed of partition reassignment significantly. (As i

Re: Partition Reassignment in Cloud

time running reassignments to change old_broker_id => new_broker_id ? On Wednesday, June 19, 2019, 9:19:58 PM PDT, Varun Kumar wrote: Hi I have been trying a small experiment with partition reassignment in cloud. where instead of copying data between brokers using network, I moved the d

[jira] [Created] (KAFKA-8737) TaskMigrated Exception while rebalancing kafka streams

KUMAR created KAFKA-8737: Summary: TaskMigrated Exception while rebalancing kafka streams Key: KAFKA-8737 URL: https://issues.apache.org/jira/browse/KAFKA-8737 Project: Kafka Issue Type: Bug

Re: Kafka HDFS Connector

I think you can use flume also. Thanks, Mudit On 6/22/16, 12:29 PM, "Pariksheet Barapatre" wrote: >Anybody have any idea on this? > >Thanks >Pari > >On 20 June 2016 at 14:36, Pariksheet Barapatre >wrote: > >> Hello All, >> >> I have data coming from sensors into kafka cluster in text format

SSL: Broker works but consumer/producer fail

Hi I followed Apache Kafka SSL instructions verbatim but my producer and consumer both hang or error out as follows. openssl s_client BTW does work fine with the server below yielding certificates etc thereby confirming that the server can talk back SSL. *Producer and Consumer*

Remote kafka - Connection refused.

Dear all, I am trying to connect my python consumer to a remote kafka server. But in kafka/conn.py#reinit, the socket call socket.create_connection throws "connection refused" error. to confirm, it is not a firewall issue, i tried connecting to other servers (in the same machine as kafka server

Newbie. Want to get started

Hi, I am new to open source contribution. I want to get started with apache kafka. Can anyone help me find some bugs to work on and a mentor I can ask for help? Thanks in advance. Regards, Yaswanth Kumar

Re: Newbie. Want to get started

discussed in the ticket comments, too. > > Hope this helps to get started. Welcome to the community! > > > > -Matthias > > On 4/17/17 1:58 AM, Yaswanth Kumar wrote: > > Hi, > > > > I am new to open source contribution. I want to get started with apache > >

Re: Kafka Multiple Consumer Group for Same Topic

ible. For Kafka the performance should be similar or close to as having multiple consumers using a single group. _ From: Senthil Kumar Sent: Tuesday, January 24, 2017 10:38 PM Subject: Kafka Multiple Consumer Group for Same Topic To: Cc: Hi Team

Listen interface

hi all, On my ubuntu 14.04 machine, i have a virt-manager/libvirtd private network running. And that has made a virtual network interface, virbr0 with ip 192.168.122.1. in my kafka broker settings, i have set the host.name property to the host name of the machine. When i do netstat -lntp |

Re: Listen interface

solved it by removing host.name setting in server.properties.. now kafka is listening on all interfaces, as i expected... thank you all On Mon, Jul 27, 2015 at 8:03 AM, madhavan kumar wrote: > hi all, >On my ubuntu 14.04 machine, i have a virt-manager/libvirtd private > network run

kafka - tcp ports

hi all, i am monitoring the ports that are used by kafka server on my machine. When i do, sudo netstat -ntp | grep 9092 it shows, tcp0 0 192.168.122.1:9092 192.168.122.6:59158 ESTABLISHED 27413/java tcp0 0 192.168.122.1:9092 192.168.122.1:58338 ESTABLISHED

[Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL

Hello, Starting this thread to discuss KIP-532: wiki link : https://cwiki.apache.org/confluence/display/KAFKA/KIP-523:+Add+KStream%23toTable+to+the+Streams+DSL jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658 There has been some discussion around the use-case of this KIP in the Jira

Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL

ream%23toTable+to+the+Streams+DSL jira ticket : https://issues.apache.org/jira/browse/KAFKA-7658 Best, Aishwarya On Fri, Sep 13, 2019 at 10:49 AM aishwarya kumar wrote: > Hello, > > Starting this thread to discuss KIP-532: > wiki link : > https://cwiki.apache.org/confluence/display/KAFKA/K

Re: [DISCUSS] KIP-479: Add Materialized to Join

Will this be applicable to Kstream-Ktable joins as well? Or do users always materialize these joins explicitly? I'm not sure if its even necessary (or makes sense), just trying to understand why the change is applicable to Kstream joins only? Best, Aishwarya On Tue, Sep 17, 2019 at 4:05 PM Bill B

Re: [DISCUSS] KIP-479: Add Materialized to Join

Hi Bill, Thanks for clarifying, and the KIP looks great!! Best regards, Aishwarya On Tue, Sep 17, 2019, 6:16 PM Bill Bejeck wrote: > Hi Aishwarya, > > On Tue, Sep 17, 2019 at 1:41 PM aishwarya kumar > wrote: > > > Will this be applicable to Kstream-Ktable joins as well?

Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL

and a `storeName` or > `StoreSupplier` is specified, the store will always be materialized and > also be queryable. Otherwise, case (1) or (2) applies. > > > > -Matthias > > > On 9/17/19 6:42 AM, aishwarya kumar wrote: > > Hi All, > > > > Keeping this thread

Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL

50 AM aishwarya kumar wrote: > Thanks Matthias, > > That does make sense, let me update the KIP to reflect the Materialization > scenario. > > Best, > Aishwarya > > On Tue, Sep 17, 2019, 2:49 PM Matthias J. Sax > wrote: > >> Aishwarya, >> >> than

Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL

> > Thoughts? > > Thanks, > -John > > On Tue, Sep 24, 2019 at 9:54 AM aishwarya kumar > wrote: > > > > Thank you for the suggestion Matthais, i've made the necessary changes in > > the KIP. > > > > Keeping this thread open for further in

Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL

r the KIP, Aishwarya! > -John > > On Fri, Sep 27, 2019 at 10:11 AM aishwarya kumar > wrote: > > > > Hello John, > > > > Thank you for pointing this out to me, to maintain consistency across > API's > > it does make sense to allow users to define c

Re: [Discuss] - KIP-532 - Add KStream#toTable to the Streams DSL

Apologies for the delay. I have made the changes in the KIP, i'll be starting the voting process shortly. Best regards, Aishwarya Kumar On Mon, Oct 7, 2019 at 6:06 PM Matthias J. Sax wrote: > Aishwarya, > > Why is bullet point (2) formatted as "strike through"? If yo

[VOTE] KIP-523 Add KStream#toTable to the Streams DSL

Hello All, After concluding discussions for this KIP, I would like to go forward with the voting process. Jira Ticket: https://issues.apache.org/jira/browse/KAFKA-7658 KIP : https://cwiki.apache.org/confluence/display/KAFKA/KIP-523%3A+Add+KStream%23toTable+to+the+Streams+DSL Thank you, Aishwarya

Re: [VOTE] KIP-523 Add KStream#toTable to the Streams DSL

t; On Thu, Oct 24, 2019 at 12:54 PM Matthias J. Sax > wrote: > > > Thanks for the KIP! > > > > +1 (binding) > > > > > > -Matthias > > > > On 10/24/19 6:19 AM, aishwarya kumar wrote: > > > Hello All, > > > > >

Re: [VOTE] KIP-523 Add KStream#toTable to the Streams DSL

Mon, Oct 28, 2019 at 11:58 AM aishwarya kumar > wrote: > > > > Thank you, > > > > Two binding votes so far. > > > > I'll keep this thread open for a couple of days. > > > > Best, > > Aishwarya > > > > On Thu, Oct 24, 2019,

Re: [VOTE] KIP-523 Add KStream#toTable to the Streams DSL

v 3, 2019 at 11:46 AM aishwarya kumar > wrote: > > > This thread has been open for more than 72 hours. So far there are 2 > > binding and 1 non-binding votes, looking to conclude this quickly!! > > > > Best, > > Aishwarya > > > > On Mon, Oct 28, 2019 at

Re: [ANNOUNCE] New committer: Mickael Maison

Congratulations Mickael!! *Best regards,* *Ankit Kumar.* On Fri, Nov 8, 2019 at 9:08 PM Viktor Somogyi-Vass wrote: > Congrats Mickael!! :) > > On Fri, Nov 8, 2019 at 1:24 PM Satish Duggana > wrote: > > > Congratulations Mickael!! > > > > On Fri, Nov

Re: [ANNOUNCE] New committer: John Roesler

Congratulations John!! On Wed, Nov 13, 2019 at 8:17 AM Rajini Sivaram wrote: > Congratulations, John! > > Regards, > > Rajini > > On Wed, Nov 13, 2019 at 8:49 AM Mickael Maison > wrote: > > > Congratulations John! > > > > On Wed, Nov 13, 2019 at 8:29 AM Tom Bentley wrote: > > > > > > Congratul

NODE JS INTERACTION WITH KAFKA

you, -- Ankit Kumar NodeJs Developer Appinventiv

Requesting permission to create KIP

Hi kafka-devs, I am planning to add support for sources which can't give out changed records in kafka connect. Please, provide me permission to create KIP. Wiki id - nikhil578 Email id - nikhilkumar...@gmail.com Thanks, Nikhil

Need contributor access to Jira

Hi Team , Please guide me how I can request for the contributor access for jira so that I can assign some jira tickets to myself and contribute to the kafka community. Username: sndp2693 Email: sndp2...@gmail.com Full Name: Sandeep Kumar Thanks and Regards Sandeep

Need contributor access to Kafka Improvement Proposals

HI, Can you please grant me access to create KIP? Thanks, Sandeep

Re: Need contributor access to Kafka Improvement Proposals

Hi Matthias, Can you please grant me contributor access to create KIP ? UserId : sndp2693 EmailId : sndp2...@gmail.com Regards, Sandeep On Thu, Sep 3, 2020 at 1:26 PM Sandeep Kumar wrote: > HI, > > Can you please grant me access to create KIP? > > Thanks, > Sandeep >

Discussion on KIP-670 : Add ConsumerGroupCommand to delete static members

Hi All, I am new to the Kafka contribution community. I have picked up a jira ticket https://issues.apache.org/jira/browse/KAFKA-9440 which requires KIP. I have submitted KIP for it https://cwiki.apache.org/confluence/display/KAFKA/KIP-670%3A+Add+ConsumerGroupCommand+to+delete+static+members I a

Re: Build failed in Jenkins: Kafka » kafka-trunk-jdk8 #70

unsubscribe On Sat, Sep 19, 2020 at 1:07 AM Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See < > https://ci-builds.apache.org/job/Kafka/job/kafka-trunk-jdk8/70/display/redirect?page=changes > > > > Changes: > > [github] MINOR: Remove unused variable (#9303) > > > --

[VOTE] KIP-930: Tiered Storage Metrics

Hi All, I would like to start the vote for KIP-930 Tiered Storage Metrics. The KIP is here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-930%3A+Tiered+Storage+Metrics Regards Abhijeet.

Re: [DISCUSS] KIP-930: Tiered Storage Metrics

t-offset and log-end-offset metrics. Should we > expose local-log-start-offset and highest-offset-uploaded-to-remote-storage > as metric? > > Thanks, > Kamal > > On Mon, Jul 24, 2023 at 2:08 PM Abhijeet Kumar > > wrote: > > > Hi All, > > > > I created

Re: [VOTE] KIP-930: Tiered Storage Metrics

Please find the updated link to the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-930%3A+Rename+ambiguous+Tiered+Storage+Metrics Updated the KIP as per our conversation on the discussion thread. On Tue, Jul 25, 2023 at 11:29 AM Abhijeet Kumar wrote: > Hi All, > > I woul

Re: [VOTE] KIP-930: Tiered Storage Metrics

to it yet. > > > > -- > > Divij Vaidya > > > > > > > > On Tue, Jul 25, 2023 at 6:45 PM Satish Duggana > wrote: > > > > > > +1 for the KIP. > > > > > > Thanks, > > > Satish. > > > > > > On Tue

Requesting permissions to contribute to Apache Kafka

Hi Team, Please provide access to contribute to Apache Kafka JIRA id -- akanimesh7 Wiki Id -- akanimesh7 -- Animesh Kumar 8120004556

[DISCUSS] KIP-956: Tiered Storage Quotas

Hi All, I have created KIP-956 for defining read and write quota for tiered storage. https://cwiki.apache.org/confluence/display/KAFKA/KIP-956+Tiered+Storage+Quotas Feedback and suggestions are welcome. Regards, Abhijeet.

Re: [VOTE] KIP-1023: Follower fetch from tiered offset

gt; On Fri, 26 Apr 2024 at 8:35 PM, Jun Rao wrote: > > > Hi, Abhijeet, > > > > Thanks for the KIP. +1 > > > > Jun > > > > On Thu, Apr 25, 2024 at 10:30 PM Abhijeet Kumar < > > abhijeet.cse@gmail.com> > > wrote: > > > > >

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

and now the new > last-tiered-offset is 15 > 5. Go back to 2 > > This could cause a cycle where the replica will be stuck trying to > reconcile with the leader. > > -- > Divij Vaidya > > > > On Fri, Apr 26, 2024 at 7:28 AM Abhijeet Kumar > > wrote: > >

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

gt; documented as an acceptable risk in current design. > > -- > Divij Vaidya > > > > On Tue, Jul 23, 2024 at 11:55 AM Abhijeet Kumar < > abhijeet.cse@gmail.com> > wrote: > > > Hi Divij, > > > > Seems like there is some confusion about the ne

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

r this scenario due to increased probability of having > an aggressive local cleanup with Tiered Storage. > > -- > Divij Vaidya > > > > On Wed, Jul 24, 2024 at 1:22 PM Abhijeet Kumar > > wrote: > > > Hi Divij, > > > > The rare scenario we are discu

Re: [DISCUSS] KIP-956: Tiered Storage Quotas

the situation when a segment is still waiting for > > > offloading while this segment is expired and eligible to be deleted? > > > Maybe it'll be easier to not block the RLMTask when quota exceeded, and > > > just check it each time the RLMTask runs? > > >

Re: [DISCUSS] KIP-956: Tiered Storage Quotas

RLMTask into smaller tasks is already being proposed in KIP-950. The two tasks will need some coordination at a topic-partition level to handle such cases as pointed out. For this particular case, the upload task could skip uploading the segment if it is already eligible for deletion. > Thank

Re: [DISCUSS] KIP-956: Tiered Storage Quotas

andle the situation when a segment is still waiting for > > offloading while this segment is expired and eligible to be deleted? > > Maybe it'll be easier to not block the RLMTask when quota exceeded, and > > just check it each time the RLMTask runs? > > >

Re: [DISCUSS] KIP-956: Tiered Storage Quotas

tive Kafka topics.". However, this > can happen with or without tier storage and the current quota system for > consumers is designed for solving this exact problem. Could you explain the > usage of this additional quota? > > Thanks, > > Jun > > On Mon, Feb 12, 202

Re: [DISCUSS] KIP-956: Tiered Storage Quotas

ger, name=RemoteFetchThrottleTime > kafka.log.remote:type=RemoteLogManager, name=RemoteCopyThrottleTime > > Jun > > On Tue, Feb 27, 2024 at 1:39 AM Abhijeet Kumar > > wrote: > > > Hi Jun, > > > > The existing quota system for consumers is designed

Re: [DISCUSS] KIP-956: Tiered Storage Quotas

s use read/write whereas the metrics use fetch/copy. Could > we make them consistent? > > Jun > > On Wed, Feb 28, 2024 at 6:49 AM Abhijeet Kumar > > wrote: > > > Hi Jun, > > > > Clarified the meaning of the two metrics. Also updated the KIP. > > > >

Re: [DISCUSS] KIP-956: Tiered Storage Quotas

look, and it LGTM now! > > Thanks. > Luke > > On Tue, Mar 5, 2024 at 2:50 AM Jun Rao wrote: > > > Hi, Abhijeet, > > > > Thanks for the reply. Sounds good to me. > > > > Jun > > > > > > On Sat, Mar 2, 2024 at 7:40 PM Abhijeet Kumar <

[VOTE] KIP-956: Tiered Storage Quotas

Hi All, I would like to start the vote for KIP-956 - Tiered Storage Quotas The KIP is here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-956+Tiered+Storage+Quotas Regards. Abhijeet.

Re: [DISCUSS] KIP-956: Tiered Storage Quotas

the configs: > - RLMCopyQuotaManager > - RLMFetchQuotaManager > > > On Fri, 8 Mar 2024 at 13:43, Abhijeet Kumar > wrote: > > > Thank you all for your comments. As all the comments in the thread are > > addressed, I am starting a Vote thread for the KIP. Please have a look. >

Re: [VOTE] KIP-956: Tiered Storage Quotas

t; > > > > -- > > > > Kamal > > > > > > > > On Fri, Mar 8, 2024 at 11:02 PM Jun Rao > > > wrote: > > > > > > > > > Hi, Abhijeet, > > > > > > > > > > Thanks for the KIP.

[DISCUSS] KIP-1023: Follower fetch from tiered offset

Hi All, I have created KIP-1023 to introduce follower fetch from tiered offset. This feature will be helpful in significantly reducing Kafka rebalance/rebuild times when the cluster is enabled with tiered storage. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1023%3A+Follower+fetch+from+t

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

> back to a follower - earliest-pending-upload-offset or the > latest-tiered-offset? If it is the second, then I believe KIP-1005 ought to > already be exposing that offset as part of the ListOffsets API, no? > > Best, > Christo > > On Wed, 27 Mar 2024 at 18:23, Abhijeet Kumar >

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

e follower know Local-Log-Start-Offset? > Missed this detail. The follower will need to call the leader APi to fetch the EarliestLocal offset for this. > Jun > > On Sat, Mar 30, 2024 at 5:51 AM Abhijeet Kumar > > wrote: > > > Hi Christo, > > > >

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

the offset and the corresponding epoch. > > > > 4. "Check if the follower replica is empty and if the feature to use > > last-tiered-offset is enabled." > > Why do we need to check if the follower replica is empty? > > > > 5. It can be confirmed by ch

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

we add "Rejected Alternatives" section to the end of the KIP to > add some of them? > Like the "ListOffsetRequest" approach VS "Earliest-Pending-Upload-Offset" > approach, or getting the "Earliest-Pending-Upload-Offset" from remote log > metadata

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

follower? > > > > > > 3. Could we add "Rejected Alternatives" section to the end of the KIP > to > > > add some of them? > > > Like the "ListOffsetRequest" approach VS > "Earliest-Pending-Upload-Offset" > > > approach,

Re: [DISCUSS] KIP-1023: Follower fetch from tiered offset

ments. > > Luke > > On Thu, Apr 25, 2024 at 4:21 AM Jun Rao wrote: > > > Hi, Abhijeet, > > > > Thanks for the updated KIP. It looks good to me. > > > > Jun > > > > On Mon, Apr 22, 2024 at 12:08 PM Abhijeet Kumar < > > abhijeet.cs

[VOTE] KIP-1023: Follower fetch from tiered offset

Hi All, I would like to start the vote for KIP-1023 - Follower fetch from tiered offset The KIP is here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1023%3A+Follower+fetch+from+tiered+offset Regards. Abhijeet.

Supported Kafka/Zookeeper Version with ELK 8.4.3

) consuming the data. We have an integration of ELK 7.16 with Kafka V2.5.1 and zookeeper 3.5.8. Please suggest if we upgrade on ELK 8.4.3 version which Kafka and Zookeeper version will be supported? Provide us handful documents. Let me know if you any further questions. Thanks Sudip Kumar

RE: Supported Kafka/Zookeeper Version with ELK 8.4.3

Hi Team, We are still waiting for the reply. Please update we must know what version of Kafka is compatible with ELK 8.4 version. Still, I can see no one replied on user and Dev community portal [cid:image001.png@01D8EAFE.9B1F9280] Thanks Sudip From: Kumar, Sudip Sent: Monday, October 17

Re: [ANNOUNCE] Apache Kafka 2.2.2

Hi, May I know how would this version be different from the later versions 2.3.0, 2.3.1. I'm looking to upgrade my current Kafka 2.1.0, so can you please let me know what differences it would make for upgrading to 2.2.2 or 2.3.1? Thanks, Vinay On Monday, December 2, 2019, Vahid Hashemian wrote:

Re: [DISCUSS] KIP-553: Enable TLSv1.3 by default and disable all protocols except [TLSV1.2, TLSV1.3]

, > > >>> > > >>> I'd like to start a discussion of KIP. > > >>> Its goal is to enable TLSv1.3 and disable obsolete versions by > default. > > >>> > > >>> > > >> > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=142641956 > > >>> > > >>> Your comments and suggestions are welcome. > > >>> > > >> > > >> > > > > > -- Regards, Brajesh Kumar

[DISCUSS] KIP-930: Tiered Storage Metrics

Hi All, I created KIP-930 for adding RemoteIndexCache stats and also to rename some tiered storage metrics added as part of KIP-405 to remove ambiguity. https://cwiki.apache.or

[DISCUSS] KIP-930: Tiered Storage Metrics

Hi All, I created KIP-930 for adding RemoteIndexCache stats and also to rename some tiered storage metrics added as part of KIP-405 to remove ambiguity. https://cwiki.apache.or

Re: [ANNOUNCE] New Kafka PMC Member: Randall Hauch

Congratulations, Randall! Best regards, Rankesh Kumar Partner Solutions Engineer +91 (701)913-0147 Follow us: Blog • Slack • Twitter • YouTube > On 17-Apr-2021, at 1:41 PM, Tom Bentley wrote: > > Congratulations Randall! > > > > On Sat, Apr 17, 2021 at 7:36 A

Re: [ANNOUNCE] New Kafka PMC Member: Konstantine Karantasis

Yay! Congratulations, KK! Best regards, Rankesh Kumar Partner Solutions Engineer +91 (701)913-0147 Follow us: Blog • Slack • Twitter • YouTube > On 21-Jun-2021, at 8:58 PM, Mickael Maison wrote: > > Hi, > > It's my pleasure to announce that Konstantine Karantasis is n

Re: [ANNOUNCE] New Kafka PMC member: David Jacot

Congratulations, David! Best regards, Rankesh Kumar Partner Solutions Engineer +91 (701)913-0147 Follow us: Blog • Slack • Twitter • YouTube > On 18-Dec-2021, at 4:38 AM, Gwen Shapira wrote: > > Hi everyone, > > David Jacot has been an Apache Kafka committer since Oct 20

kafka_2.9.2-0.8.2.0 - Topic marked for deletion

Hi, I am trying to delete the kafka topic, verison I am using is 2.9.2-0.8.2. I am using dockerized kafka to delete the topic already created and didn’t set the “delete.topic.enable=true”. When I am listing the topics, it is giving as “ marked for deletion”. Is this a bug? However when I am s

Re: [ANNOUNCE] New committer: David Jacot

Many congratulations, David. It is awesome. Best regards, Rankesh > On 16-Oct-2020, at 9:51 PM, Mickael Maison wrote: > > Congratulations David! > > On Fri, Oct 16, 2020 at 6:05 PM Bill Bejeck wrote: >> >> Congrats David! Well deserved. >> >> -Bill >> >> On Fri, Oct 16, 2020 at 12:01 PM G

Re: [ANNOUNCE] New committer: A. Sophie Blee-Goldman

Many congratulations, Sophie. Best regards, Rankesh > On 20-Oct-2020, at 12:34 AM, Gwen Shapira wrote: > > Congratulations, Sophie! > > On Mon, Oct 19, 2020 at 9:41 AM Matthias J. Sax wrote: >> >> Hi all, >> >> I am excited to announce that A. Sophie Blee-Goldman has accepted her >> invita

Re: [ANNOUNCE] New committer: Chia-Ping Tsai

Many congratulations, Chia-Ping! Best regards, Rankesh > On 20-Oct-2020, at 6:45 AM, Luke Chen wrote: > > Congratulations! Chia-Ping大大! > Well deserved! > > Luke > > On Tue, Oct 20, 2020 at 2:30 AM Mickael Maison > wrote: > >> Congrats Chia-Ping! >> >> On Mon, Oct 19, 2020 at 8:29 PM Ismae

Will Cache will get invalidated if i read lots of messages using random offsets and partitions

Hi Team, I understand that Kafka is meant for sequential access, but I have use case of accessing the messages from Kafka based on random offsets and partitions. For Example There is topic called “topic-A” some consumers are listening from this topic, it may happen that processing of these mes

All Lead Replica are on a Single Broker

Kafka Dev, I am not sure if we are missing something over here but after cluster restart I see all the existing topics choose one broker as the Lead replica which is probably not a right behavior as we want to distribute the load across the cluster opic: credit.cache.smc.debt.queue parti

How to Rate limit number of messages while writing Spark DataFrame to kafka topic

0 I have a spark dataframe which I am writing to Kafka topic using the below script. val ProducerConfig: Map[String, String] = Map( s"kafka.${ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG}" -> Integer.MAX_VALUE.toString, s"kafka.${ProducerCon

Re: [DISCUSS] KIP-1153: Extending support for Microsecond Precision for Kafka Connect

Hi Kafka Community, Sorry due to some changes I had to change the link to the kip. Here is the updated KIP link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1154%3A+Extending+support+for+Microsecond+Precision+for+Kafka+Connect On Sat, Apr 5, 2025 at 12:14 AM pritam kumar wrote: >

Coordinating Cross-Cluster Kafka Transactions with Exactly-Once Semantics

suggestions, or shared pain would be incredibly helpful! Thanks in advance, Pritam Kumar

Re: Re: [DISCUSS] KIP-1153: Extending support for Microsecond Precision for Kafka Connect

Hi Sudesh, Sorry for the earlier comment, I just checked that avro in 1.12 has timestamp-nanos support. I will update the KIP to have nanosecond support also and correspondingly I will make changes for nanosecond. Thanks Pritam. On Wed, Apr 16, 2025 at 7:13 PM pritam kumar wrote: > Also j

Re: Re: [DISCUSS] KIP-1153: Extending support for Microsecond Precision for Kafka Connect

Thank you Sudesh, for reviewing the KIP and the valuable feedback. Thanks, Pritam On Thu, Apr 17, 2025 at 8:59 PM Sudesh Wasnik wrote: > Thanks Pritam! > No more feedback from my end. Nice addition ! > > Thanks, > Sudesh > On 17 Apr 2025 at 4:59 PM +0530, pritam kumar , &

Re: Re: [DISCUSS] KIP-1153: Extending support for Microsecond Precision for Kafka Connect

Also just to add I did not add this in the first place as I think Avro itself does not have the Nanosecond logical type. On Wed, Apr 16, 2025 at 7:01 PM pritam kumar wrote: > Thanks Sudesh, for taking a look at this. I am already working on > extending this for nanosecond precision a

Re: Re: [DISCUSS] KIP-1153: Extending support for Microsecond Precision for Kafka Connect

he KIP to also add support for Nanosecond precision! > > Thanks > Sudesh > > On 2025/04/05 01:30:49 pritam kumar wrote: > > Hi Kafka Community, > > Sorry due to some changes I had to change the link to the kip. > > Here is the updated KIP link: > > > https

Re: Re: [DISCUSS] KIP-1153: Extending support for Microsecond Precision for Kafka Connect

I have made Nanosecond related changes in the KIP. Please have a look. Thanks Pritam. On Thu, Apr 17, 2025 at 4:25 PM pritam kumar wrote: > Hi Sudesh, > Sorry for the earlier comment, I just checked that avro in 1.12 has > timestamp-nanos support. I will update the KIP to have n

[DISCUSS] KIP-1173: Connect Storage Topics Sharing Across Clusters

Hi Kafka Community, I'd like to start a discussion on KIP-1173: Connect Storage Topics Sharing Across Clusters . The primary motivation for writing this KIP and proposing this enhancemen

Re: [DISCUSS] KIP-1153: Extending support for Microsecond Precision for Kafka Connect

Hi Chris, Can you please review this one too? On Sat, Apr 5, 2025 at 7:00 AM pritam kumar wrote: > Hi Kafka Community, > Sorry due to some changes I had to change the link to the kip. > Here is the updated KIP link: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1154%

Re: [DISCUSS] KIP-1173: Connect Storage Topics Sharing Across Clusters

ilures: The unavailability/corruption of one topic now > affects multiple Connect clusters instead of just one > * Complications with Exactly Once mode: transactional writes to internal > topics may cause unavailability in other clusters from hanging > transactions. > > Thanks! >

Re: [DISCUSS] KIP-1173: Connect Storage Topics Sharing Across Clusters

; > Finally, I think we may want to use a more structured format for record > keys. IIRC there's essentially no limits on what characters can be used for > group IDs (please correct me if I'm wrong), so we can't use any kind of > delimiter that won't be included in the group

[DISCUSS] KIP-1153: Extending support for Microsecond Precision for Kafka Connect

Hi Kafka Community, I’d like to start a discussion on KIP-1153: Extending Support for Microsecond Precision for Kafka Connect . The primary motivation behind this KIP is t

Re: [DISCUSS] KIP-1153: Extending support for Microsecond Precision for Kafka Connect

Hi Kafka Community, If there is no other feedback. I would like to start a VOTE. On Mon, May 19, 2025 at 6:51 PM pritam kumar wrote: > Hi Kafka Community, > Here is the KIP LInk -> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=347933282 > > On Mon, May 1

Re: [DISCUSS] KIP-1153: Extending support for Microsecond Precision for Kafka Connect

Hi Kafka Community, Could you please review this KIP proposal when you have a chance? Thank you, Pritam On Sat, May 3, 2025 at 10:53 PM pritam kumar wrote: > Hi Chris, > Can you please review this one too? > > On Sat, Apr 5, 2025 at 7:00 AM pritam kumar > wrote: > >

  1   2   3   4   >