Kafka Transaction "one by one", what is the bottleneck ?

Hello, I try to understand a performance problem using Transactions: Here my setup (for whatever reason, we need to create a transaction "per message") (kafka v2.3/java): --- consumer.poll().forEach( record -> { producer.beginTransaction(); producer.send(...); ... pr

Very Urgent: Error while creating ephemeral at /brokers/ids/0, node already exists and owner

Hi, Whenever I try to do rolling restart and start one of my broker, I get this error. Can Anyone help me to get rid of this? [2020-03-12 18:33:34,661] INFO Logs loading complete in 1553 ms. (kafka.log.LogManager) [2020-03-12 18:33:34,677] INFO Starting log cleanup with a period of 30 ms. (

Re: Very Urgent: Error while creating ephemeral at /brokers/ids/0, node already exists and owner

Hello Sunil, are you gracefully shutting down the broker? can you verify there is no other broker within the cluster with the same id 0? can you check in the logs that the connection to ZooKeeper is getting close/disconnected when shutting down the Broker? Cheers! On Thu, Mar 12, 2020 at 10

RE: [External] Re: Very Urgent: Error while creating ephemeral at /brokers/ids/0, node already exists and owner

Hi Jonathan, "are you gracefully shutting down the broker?" above question solved my prob gracefully 😊. Hahaha Thanks a lot! Regards, Sunil. -Original Message- From: Jonathan Santilli Sent: Thursday, March 12, 2020 5:17 PM To: Kafka Users Subject: [External] Re: Very Urgent

Re: what happened in case of single disk failure

Please see the following link from Confluent. Also, if you register with Confluent Technical Talks, they are running quite a lot of nice and simplified webinar this month on Fundamentals of Kafka. https://www.youtube.com/watch?v=ibozaujze9k I thought the 2 part presentation was quite good (but I

Re: [kafka-clients] Re: [VOTE] 2.5.0 RC1

Hi David, 1. Scanned the Javadoc, looks good 2. Downloaded kafka_2.12-2.5.0 and ran the quickstart and streams quickstart 3. Verified the signatures +1 (non-binding) Thanks for running the release David! -Bill On Tue, Mar 10, 2020 at 4:01 PM David Arthur wrote: > Thanks for the

Re: Incremental topic subscription

Hello, Does anyone have insight into this? Does Kafka support incremental topic subscription and unsubscription? This question is in the context of the latest Kafka version (2.4.0). Thank you. Pedro Cardoso Research Data Engineer pedro.card...@feedzai.com [image: Follow Feedzai on Facebook

Re: Kafka Transaction "one by one", what is the bottleneck ?

When doing transaction, replication must finish broker side what implies increase latency. And the `commitTransaction` call is blocking (including a producer.flush()), the latency directly translates into decreased throughput. -Matthias On 3/12/20 2:40 AM, JALADY Christophe wrote: > Hello, > >

[ANNOUNCE] Apache Kafka 2.4.1

The Apache Kafka community is pleased to announce the release for Apache Kafka 2.4.1 This is a bug fix release and it includes fixes and improvements from 39 JIRAs, including a few critical bugs. All of the changes in this release can be found in the release notes: https://www.apache.org/dist/kaf

Re: [ANNOUNCE] Apache Kafka 2.4.1

Thanks for driving the release Bill! -Matthias On 3/12/20 1:22 PM, Bill Bejeck wrote: > The Apache Kafka community is pleased to announce the release for Apache > Kafka 2.4.1 > > This is a bug fix release and it includes fixes and improvements from 39 > JIRAs, including a few critical bugs. > >

Re: what happened in case of single disk failure

Thanks, it helps a lot, for a long time I am used to read documentation on Kafka official site, you made me realize that there are also a lot of resources on Confluent. M. Manna 于2020年3月12日周四 下午9:06写道： > Please see the following link from Confluent. Also, if you register with > Confluent Technic