Request/Response correlation

Hello, I have a web application that uses Kafka in the back-end. When a user makes a request via the browser, the application sends a message to a Kafka topic and then listens for a response sent to another topic. I'm struggling to see how it is possible to route the response to the user/browser t

Re: Request/Response correlation

I have done this in the past using the message key. The reason I used the key was so I did not have to unbundle the entire message to find who the request response needed to be routed to. The way it worked was that the request service would create a key and then submit the information to be proc

Re: Request/Response correlation

Thanks Steven On 28 August 2017 at 18:03, Steven Elliott wrote: > I have done this in the past using the message key. The reason I used the > key was so I did not have to unbundle the entire message to find who the > request response needed to be routed to. > > The way it worked was that the r

Re: Is it possible to disable caching for some kafka topics?

What I mean by caching is that the messages are being stored in memory and consumers are also reading from the memory (instead of the disk). In this scenario, I would want messages related to topic A to be cached in memory. And messages related to topic B don't have to be cached in memory. I want t

RE: Is it possible to disable caching for some kafka topics?

I don't think that is possible since Kafka uses the file system cache for this. -Dave -Original Message- From: Archie [mailto:anubhavnidhi1...@gmail.com] Sent: Monday, August 28, 2017 4:14 PM To: users@kafka.apache.org Subject: Re: Is it possible to disable caching for some kafka topics?

Strange Kafka throughput issues

Hello, First here's what I have: - 5 node (m4.2xlarge, 8 vCPU, 32G RAM) Kafka cluster - running version: 0.11.0 - each broker has 4 dedicated ebs storage - a single topic: 40 partitions, repl-factor=3 I'm using the kafka-producer-perf-test to benchmark it. - Test 1: - rec

Offsets across brokers

Hi, If a topic partition is replicated and leader switches from broker 1 to broker 2 , are the offsets for messages in broker 2 same as broker1 ? If not, how can applications that store offsets outside of Kafka handle the difference? Thanks, Vignesh.

Re: Offsets across brokers

Yes the offsets are the same. -hans > On Aug 28, 2017, at 8:32 PM, Vignesh wrote: > > Hi, > > If a topic partition is replicated and leader switches from broker 1 to > broker 2 , are the offsets for messages in broker 2 same as broker1 ? If > not, how can applications that store offsets out