Re: [ANNOUNCE] New Kafka PMC member: Dong Lin

Congrats Dong! -James > On Aug 20, 2018, at 3:54 AM, Ismael Juma wrote: > > Hi everyone, > > Dong Lin became a committer in March 2018. Since then, he has remained > active in the community and contributed a number of patches, reviewed > several pull requests and participated in numerous KIP d

Re: [ANNOUNCE] New committer: Colin McCabe

Congrats, Colin! -James > On Sep 25, 2018, at 1:39 AM, Ismael Juma wrote: > > Hi all, > > The PMC for Apache Kafka has invited Colin McCabe as a committer and we are > pleased to announce that he has accepted! > > Colin has contributed 101 commits and 8 KIPs including significant > improvemen

Re: [ANNOUNCE] Apache Kafka 2.1.0

Thanks Dong for running the release, and congrats to everyone in the community! -James Sent from my iPhone > On Nov 21, 2018, at 10:09 AM, Dong Lin wrote: > > The Apache Kafka community is pleased to announce the release for Apache > Kafka 2.1.0 > > > This is a major release and includes sig

Re: [ANNOUNCE] New Committer: Vahid Hashemian

Congrats, Vahid!! -James > On Jan 15, 2019, at 2:44 PM, Jason Gustafson wrote: > > Hi All, > > The PMC for Apache Kafka has invited Vahid Hashemian as a project committer > and > we are > pleased to announce that he has accepted! > > Vahid has made numerous contributions to the Kafka communi

Re: [ANNOUNCE] New Committer: Randall Hauch

Congrats, Randall! Well deserved! -James Sent from my iPhone > On Feb 14, 2019, at 6:16 PM, Guozhang Wang wrote: > > Hello all, > > The PMC of Apache Kafka is happy to announce another new committer joining > the project today: we have invited Randall Hauch as a project committer and > he has

Re: [ANNOUNCE] New Kafka PMC member: Matthias J. Sax

Congrats!! -James Sent from my iPhone > On Apr 18, 2019, at 2:35 PM, Guozhang Wang wrote: > > Hello Everyone, > > I'm glad to announce that Matthias J. Sax is now a member of Kafka PMC. > > Matthias has been a committer since Jan. 2018, and since then he continued > to be active in the commu

Re: [ANNOUNCE] New committer: A. Sophie Blee-Goldman

Congratulations, Sophie! -James > On Oct 19, 2020, at 9:40 AM, Matthias J. Sax wrote: > > Hi all, > > I am excited to announce that A. Sophie Blee-Goldman has accepted her > invitation to become an Apache Kafka committer. > > Sophie is actively contributing to Kafka since Feb 2019 and has > a

Re: [ANNOUNCE] New committer: Chia-Ping Tsai

Congratulations Chia-Ping! -James > On Oct 19, 2020, at 10:24 AM, Guozhang Wang wrote: > > Hello all, > > I'm happy to announce that Chia-Ping Tsai has accepted his invitation to > become an Apache Kafka committer. > > Chia-Ping has been contributing to Kafka since March 2018 and has made 74

Re: [ANNOUNCE] New committer: David Jacot

Congratulations, David! -James > On Oct 16, 2020, at 9:01 AM, Gwen Shapira wrote: > > The PMC for Apache Kafka has invited David Jacot as a committer, and > we are excited to say that he accepted! > > David Jacot has been contributing to Apache Kafka since July 2015 (!) > and has been very act

Re: [ANNOUNCE] Apache Kafka 2.6.1

Thank you Mickael for running the release. Good job everyone! -James Sent from my iPhone > On Jan 11, 2021, at 2:17 PM, Mickael Maison wrote: > > The Apache Kafka community is pleased to announce the release for > Apache Kafka 2.6.1. > > This is a bug fix release and it includes fixes and im

Re: [ANNOUNCE] Apache Kafka 3.2.0

Bruno, Congrats on the release! There is a small typo on the page. > KIP-791 > > adds method recordMetada() to the StateStoreContext, Should be > KIP-791 >

Can I tell mirrormaker2 to start (re)consuming from a specific offset?

Hi everyone, I am using Mirrormaker 2. From what I understand, mirrormaker2 does not store its offsets in the source cluster's __consumer_offsets topic. Rather, it stores its offsets in the destination cluster in the value defined by offset.storage.topic In the __consumer_offsets world, I coul

When using mirrormaker, how are people creating topics?

Hi, We are using mirrormaker to mirror topics from one cluster to another, and I wanted to get some advice from the community on how people are doing mirroring. In particular, how are people dealing with topic creation? Do you turn on auto-topic creation in your destination clusters (auto.crea

Why does consumer.subscribe(Pattern) require a ConsumerRebalanceListener?

Hi, I was looking at the docs for the consumer, and noticed that when calling subscribe() with a regex Pattern, that you are required to pass in a ConsumerRebalanceListener. On the other hand, when you use a fixed set of topic names (Collection), the ConsumerRebalanceListener is optional (that

Re: Is this a bug or just unintuitive behavior?

Jeff, Your analysis is correct. I would say that it is known but unintuitive behavior. As an example of a problem caused by this behavior, it's possible for mirrormaker to miss messages on newly created topics, even thought it was subscribed to them before topics were creted. See the following

Re: Lost message with Kafka configuration

kafka-console-producer.sh defaults to acks=0, which means that the producer essentially throws messages at the broker and doesn't wait/retry to make sure they are properly received. In the kafka-console-producer.sh usage text: --request-required-acksrequests (default: 0) Try

Re: Is this a bug or just unintuitive behavior?

ting consumer group via the join API " I'm guessing that this would affect any of those scenarios. -James > > > > On Thu, Jan 5, 2017 at 12:40 AM, James Cheng wrote: > >> Jeff, >> >> Your analysis is correct. I would say that it is known but unintu

Re: Lost message with Kafka configuration

> On Jan 5, 2017, at 8:23 AM, Hoang Bao Thien wrote: > > Yes, the problem is from producer configuration. And James Cheng has told > me how to fix it. > However I still get other poblem with a large file: > > org.apache.kafka.common.errors.TimeoutException: Batch con

Re: Under-replicated Partitions while rolling Kafka nodes in AWS

> On Jan 5, 2017, at 7:55 AM, Jack Lund wrote: > > Hello, all. > > We're running multiple Kafka clusters in AWS, and thus multiple Zookeeper > clusters as well. When we roll out changes to our zookeeper nodes (which > involves changes to the AMI, which means terminating the zookeeper instance >

Re: [ANNOUNCE] New committer: Grant Henke

Congrats, Grant!! -James > On Jan 11, 2017, at 11:51 AM, Gwen Shapira wrote: > > The PMC for Apache Kafka has invited Grant Henke to join as a > committer and we are pleased to announce that he has accepted! > > Grant contributed 88 patches, 90 code reviews, countless great > comments on discu

Re: Kafka docs for current trunk

+1 In particular, this would help when there is a doc change submitted to trunk which is applicable to the currently released version. It would help the change get out there faster. -James > On Feb 1, 2017, at 9:03 AM, Guozhang Wang wrote: > > +1 > > Guozhang > > > On Wed, Feb 1, 2017 at

Is anyone running Kafka on CoreOS?

Hi, (This question is kinda Kafka related, but mostly CoreOS related, so sorry if this is the wrong place to ask this.) Is anyone running Kafka on CoreOS? We run Kafka in docker containers on CoreOS. CoreOS has an OS-update policy where they will automatically install new OS updates, during wh

Re: KIP-122: Add a tool to Reset Consumer Group Offsets

Yeah, that's a good point. Some of the operations might make sense on multiple partitions at once. Moving to a timestamp might apply to all partitions, moving backwards and forwards by N offsets might apply to all partitions. However, moving to a specific offset ("set to offset 43") would most

Re: [ANNOUNCE] Apache Kafka 0.10.2.0 Released

Woohoo! Thanks for running the release, Ewen! -James > On Feb 22, 2017, at 12:33 AM, Ewen Cheslack-Postava wrote: > > The Apache Kafka community is pleased to announce the release for Apache > Kafka 0.10.2.0. This is a feature release which includes the completion > of 15 KIPs, over 200 bug fix

Re: Question about messages in __consumer_offsets topic

Yup, this got fixed in 0.10.2 https://issues.apache.org/jira/browse/KAFKA-2000 -James > On Feb 23, 2017, at 11:10 AM, Jeff Widman wrote: > > The topic deletion only triggers tombstone on brokers >= 0.10.2, correct? I > thought there was an ou

Re: Clarification on min.insync.replicas​

> On Mar 7, 2017, at 7:44 AM, Shrikant Patel wrote: > > Thanks for clarification. I am seeing strange behavior in that case, > > When I set min.insync.replicas=2 in my server.properties (restart the server) > and set the acks=all on producer, I am still able to publish to topic even > when on

Re: Clarification on min.insync.replicas​

> On Mar 7, 2017, at 12:18 PM, James Cheng wrote: > > >> On Mar 7, 2017, at 7:44 AM, Shrikant Patel wrote: >> >> Thanks for clarification. I am seeing strange behavior in that case, >> >> When I set min.insync.replicas=2 in my server.properties (resta

Re: How does offsets.retention.minutes work

Yes, that is correct. I filed a JIRA about that issue here: https://issues.apache.org/jira/browse/KAFKA-4682 -James > On Mar 15, 2017, at 8:51 PM, tao xiao wrote: > > Hi team, > > I know that Kafka deletes offset for a consumer group after a period of > time (configured by offsets.retention.m

Re: Offset commit request failing

I think it's due to the high number of partitions and the high number of consumers in the group. The group coordination info to keep track of the assignments actually happens via a message that travels through the __consumer_offsets topic. So with so many partitions and consumers, the message g

Re: ISR churn

Marcos, Radu, Are either of you seeing messages saying "Cached zkVersion [...] not equal to that in zookeeper"? If so, you may be hitting https://issues.apache.org/jira/browse/KAFKA-3042 Not sure if that helps you, since I haven't been able i

Re: stunning error - Request of length 1550939497 is not valid, it is larger than the maximum size of 104857600 bytes

Ramya, Todd, Jiefu, David, Sorry to drag up an ancient thread. I was looking for something in my email archives, and ran across this, and I might have solved part of these mysteries. I ran across this post that talked about seeing weirdly large allocations when incorrect requests are accidental

<    1   2