Re: Kafka Connect ignores the bootstrap.servers parameter

Hi Goran, Thanks for your reply. We've tried this and got interesting results. First, we found out that there is a port forwarding between the client and the server. So we put the local IP (192.168.x.x) into the 'listeners' parameter, and the external IP (98.1.96.147) into the 'advertised.listene

Re: Apache Kafka Streams with Sprinng Boot

Hi Gary, Thank you very much for the help. Thanks C Suresh On Wednesday, April 29, 2020, Gary Russell wrote: > There's a simple example in this answer on Stack Overflow [1]. > > I am the project lead for the Spring for Apache Kafka project [2]. > > If you have further questions; ask on Stack O

Re: Apache Kafka Streams with Sprinng Boot

Hi ivan, Thank you for the help. Thanks C Suresh On Wednesday, April 29, 2020, Ivan Ponomarev wrote: > Hi everyone, > > take a look at https://github.com/inponomarev/kstreams-examples > > These are small examples for my training course about Kafka Streams. All > the examples utilize Spring Boo

A general question regarding Kefka clients version

The latest version of Kafka as of now is 2.5.0. How ever when I see the clients that support this version are none. Clients for languages like c# (0.11x), python(1.0.x), swift (0.9x) which is too far from current release. Actually I was comparing with RabbitMQ which seems to be more updated. But

Re: Kafka Streams support for connecting to different Kafka clusters

Not sure if there is a JIRA already. However, even if, nobody is working on this atm. It's also quite complex... especially for exactly-once processing. -Matthias On 4/28/20 2:10 PM, Daniel Germain wrote: > As per > https://kafka.apache.org/25/documentation/streams/developer-guide/config-strea

Kafka Streams support for connecting to different Kafka clusters

As per https://kafka.apache.org/25/documentation/streams/developer-guide/config-streams Tip: Kafka Streams applications can only communicate with a single Kafka cluster specified by this config value. Future versions of Kafka Streams will support connecting to different Kafka clusters for reading i

Re: Apache Kafka Streams with Sprinng Boot

Hi everyone, take a look at https://github.com/inponomarev/kstreams-examples These are small examples for my training course about Kafka Streams. All the examples utilize Spring Boot, and they use quite recent versions of everything. Supporting notes and slides are in Russian, unfortunately.

Re: How to handle RebalanceInProgressException?

Thanks for the explanation Ben. They are very helpful. Just to clarify on the context here: 1) Before Kafka 2.0 the poll(long) call make sure that the rebalance would be completed when the call returns, no matter how long it takes. Note this also includes the time for refreshing the metadata for

Re: Apache Kafka Streams with Sprinng Boot

There's a simple example in this answer on Stack Overflow [1]. I am the project lead for the Spring for Apache Kafka project [2]. If you have further questions; ask on Stack Overflow tagged with spring-kafka (I don't monitor this group closely) [1]: https://stackoverflow.com/questions/58193901/u

Re: Kafka Connect ignores the bootstrap.servers parameter

Client connects to IP address defined by bootstrap.servers and gets metada that contain IP address where kafka is available. That is configured in server.properties: # Hostname and port the broker will advertise to producers and consumers. If not set, # it uses the value for "listeners" if conf

Re: Kafka Connect ignores the bootstrap.servers parameter

Hi Suresh, Thanks for your quick response. All the configuration files, except for the connect-standalone.properties are unmodified and use all the default parameters. Attaching them anyway just in case. This setup works perfectly on my laptop with VirtualBox (ZooKeeper and the broker on VM, the

Re: updates on Kafka Summit in 2020

Hi, Everyone, Given the current global health crisis and ongoing uncertainty, we have decided to go virtual for Kafka Summit "Austin." The CFP deadline has been extended to May 17, 2020. More details can be found at https://events.kafka-summit.org/2020-faq. Thanks, Jun On Wed, Apr 15, 2020 at

Re: Apache Kafka Streams with Sprinng Boot

Sure Khaja. Thanks C Suresh On Tuesday, April 28, 2020, KhajaAsmath Mohammed wrote: > Hello Suresh, > > I am also looking for the same. Let me know if you find anything > > Sent from my iPhone > > > On Apr 28, 2020, at 8:25 PM, Suresh Chidambaram > wrote: > > > > Hi Team, > > > > Greetings. >

Re: Apache Kafka Streams with Sprinng Boot

Hello Suresh, I am also looking for the same. Let me know if you find anything Sent from my iPhone > On Apr 28, 2020, at 8:25 PM, Suresh Chidambaram > wrote: > > Hi Team, > > Greetings. > > I have been looking for an example application which uses Kafka Streams > with Spring Boot, but I'm

Apache Kafka Streams with Sprinng Boot

Hi Team, Greetings. I have been looking for an example application which uses Kafka Streams with Spring Boot, but I'm unable to find one in the internet. Could someone help me by providing the code? Thanks C Suresh

Kafka: Messages disappearing from topics, largestTime=0

Hi, We have messages disappearing from topics on Apache Kafka with versions 2.3, 2.4.0, 2.4.1 and 2.5.0. We noticed this when we make a rolling deployment of our clusters and unfortunately it doesn't happen every time, so it's very inconsistent. Sometimes we lose all messages inside a topic, other

Can kafka state stores be used without streams and accessed globally through multiple application instances?

Hi All, I have a kafka application which consums events from multiple topics, stores certain fields from the events in local in-memory cache, based on the cached data it enriches some of the events consumed from another topic e.g. say for agent id 10, the first name and last name are cached then i

Re: MirrorMaker2 ordering guarantees

Hey, so I was just about to set `max.in.flight.requests.per.connection=1` when I found in the logs that it is already set for the "destination" producers :) Cool. Duplicates are fine for me, we have dedup checks or immutable operations on the consumers. Have a nice day! Peter On Mon, 27 Apr 202

Re: Getting NotLeaderForPartitionException for a very long time

Thanks Manoj for the comment. When i say it is taking long time, it is really long, i.e more than a day!!! thats why i am not able to understand the behavior... Regards, Gopal On Tue, Apr 28, 2020 at 2:22 PM wrote: > Follower take some time to become the leader in case leader is down . you > c

Re: Getting NotLeaderForPartitionException for a very long time

Follower take some time to become the leader in case leader is down . you can build retry logic to around this to handle this situation . On 4/28/20, 1:08 AM, "M.Gopala Krishnan" wrote: [External] Hi, I have a 3 node kafka cluster (replication-factor : 3), suddenly one of the

Re: How to handle RebalanceInProgressException?

Hi Guozhang, thanks for your reply On 27.04.20 19:17, Guozhang Wang wrote: > Hello Ben, > > First of all, just to clarify your versioning should be (4.5.0 -> 2.5.0) > and (4.3.1 -> 2.3.1) etc, right? Currently Apache Kafka's latest release is > 2.5.0. sorry, you are right I meant 2.3.1 and I upgr

Getting NotLeaderForPartitionException for a very long time

Hi, I have a 3 node kafka cluster (replication-factor : 3), suddenly one of the node in the cluster was down and i started seeing the NotLeaderForPartitionException exception in my application logs when sending the message to one of the topics, however for some of the topics i am able post and con

Kafka Connect ignores the bootstrap.servers parameter

Hi Val, Could you share the server.properties and zookeeper.properties? Thanks C Suresh On Tuesday, April 28, 2020, Valentin Kulichenko < valentin.kuliche...@gmail.com> wrote: > Greetings to the Kafka Community! > > I'm a newbie in Kafka and only recently went beyond a local installation > des