Re: Client session timed out

2020-10-20 Thread Sabina Marx
Hi Praveen, it seems to be the same problem, your log looks quite similar to mine. But I have no solution until now. Regards Sabina Von: Praveen Kumar K S Antworten an: "users@kafka.apache.org" Datum: Dienstag, 20. Oktober 2020 um 20:07 An: "users@kafka.apache.org" Betreff: Re: Client sessio

RE: Kafka Naming

2020-10-20 Thread Jonathan Goings
Example convo. 😊 Me: how will you pull data from Kafka to Elastic? Non-kafka: we're going to us a consumer Me: Ok, is that a standalone app? Non-kafka: no, we're just going to stream it through Kafka and use there out of the box solution Me: Do you need me to configure connect? Non-kafka: Well,

Re: Kafka Naming

2020-10-20 Thread Liam Clarke-Hutchinson
Hi Jonathon, Kafka Connect is designed to, well, connect things to Kafka. Kafka Streams is for stream processing using Kafka. They're rather descriptive and apt names, don't you think? If it helps, we abbreviate Connect as KC and Streams as KStreams in our organisation. Kind regards, Liam Clark

Re: Client session timed out

2020-10-20 Thread Praveen Kumar K S
Hello, I'm not sure if I can add my issue in this thread. But it seems like I'm facing the same problem. KAFKA_VERSION=2.5.1 ZK_VERSION=3.5.8 I run 3 node zookeeper cluster and 3 node kafka cluster as docker containers in docker swarm environment. When I install it for first time, everything goe

Re: Kafka Streams : Rack Aware deployment in EC2

2020-10-20 Thread Guozhang Wang
Hello Tony, I think you already know the consumer-client side fetch-from-follower feature: https://cwiki.apache.org/confluence/display/KAFKA/KIP-392%3A+Allow+consumers+to+fetch+from+closest+replica Say your Kafka deployment is cross-AZs, and your Streams is also deployed cross-AZ, you can, by set

Re: kafka-streams: do not output anything while state is not stable

2020-10-20 Thread Matthias J. Sax
It's highly use-case dependent, but applying a filter at the end does sound like a good solution to me. -Matthias On 10/19/20 12:40 PM, Mathieu D wrote: > Hello there, > > Let's say I need to restart my streams app from a blank state (whether by > changing app.id or using application-reset-tool)

Re: Subscribe to Kafka user mailing list

2020-10-20 Thread Matthias J. Sax
If you want to subscribe, please follow the instructions from the web page: https://kafka.apache.org/contact -Matthias On 10/19/20 6:32 PM, Peng Lei wrote: > >

Re: toombstones, kafkastreams, Avro & NPE

2020-10-20 Thread Matthias J. Sax
Well, the exception is throw fomr `magnolia.Magnolia$$anon$5.dereference` and I have no idea what this is... Maybe the used serde does not hangle `null` correctly? As `null` should be transalted to `null`, it might be possible to workaround the issue by "wrapping" the serde with you own impl that

toombstones, kafkastreams, Avro & NPE

2020-10-20 Thread Dumitru-Nicolae Marasoui
Hi, I am trying to delete (tombstone) some records in a compacted topic, by means of emitting a ("key", null) key-value pair, which by compaction would get removed after some time. However, I am getting the exception below: Exception in thread "SV-6c606e52-46eb-4a59-a006-27ce6ce1a603-StreamThread-

Subscribe to Kafka user mailing list

2020-10-20 Thread Peng Lei

Kafka Naming

2020-10-20 Thread Jonathan Goings
I've been working through product redesign and development with product groups in our organization. Has anyone ever tried to explain data flow through Kafka to non-Kafka people and notice the inconveniently named Kafka product offerings? Please rename Connect and Streams. Jonathan Goings NOTI

Re: LeaderElectionNotNeededException

2020-10-20 Thread Liam Clarke-Hutchinson
Hi Tom, You're right, I had had a complete brain fade on this. Turns out that the script I wrote had a logical error - while trying to ensure every zone had a replica, it went through zones in this same order, thus unintentionally ensuring that the two brokers in zone A were quite often the first

Re: LeaderElectionNotNeededException

2020-10-20 Thread Tom Bentley
Hi Liam, I think you have a misunderstanding of what preferred leader election does. All it does is ensure that the "preferred leader" (the first in the list of replicas for a partition) becomes the actual leader if it can (that is, if it's in the ISR) and if the current leader is already the pref

Re: Client session timed out

2020-10-20 Thread Sabina Marx
Yes, it's the same problem. Am 19.10.20, 19:50 schrieb "Mich Talebzadeh" : can you try to disable automatic start and on the node just booted, start zookeeper first, check the log that it is connected and then start Kafka? I assume everything is set-up OK including in $KAFKA_HOM

Re: [ANNOUNCE] New committer: David Jacot

2020-10-20 Thread David Jacot
Thanks! On Mon, Oct 19, 2020 at 8:01 PM James Cheng wrote: > Congratulations, David! > > -James > > > On Oct 16, 2020, at 9:01 AM, Gwen Shapira wrote: > > > > The PMC for Apache Kafka has invited David Jacot as a committer, and > > we are excited to say that he accepted! > > > > David Jacot has