Re: Spread primary and replica of a partition across different zones in AWS

them over and see if going forward, they remain spread across the zones. Thanks for the pointers Alex. On Wed, Nov 20, 2024 at 4:11 PM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > On Wed, Nov 20, 2024 at 11:11 AM Soham Chakraborty > wrote: > > > Fair enough.

Re: Spread primary and replica of a partition across different zones in AWS

is thrown out and gets reassigned, the leader/replica will be in different zones. Thanks, On Wed, Nov 20, 2024 at 3:22 PM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > On Wed, Nov 20, 2024 at 9:15 AM Soham Chakraborty > wrote: > > > > > So my goal is

Spread primary and replica of a partition across different zones in AWS

Hi, I have an even number of kafka brokers running on AWS, across two different AZs. What I am observing is that for all topics, sometimes both the leader and replica of a partition are present in the same AZ. I am looking to spread that across two different AZs. The ReplicationFactor for all topi

Re: Logging configuration to see from which AZ consumers are fetching data in AWS environment

And kafka client version is 2.8.0 as well. I realized that I didn't answer in last message. Thanks, On Fri, Nov 1, 2024 at 2:52 PM Soham Chakraborty wrote: > Hi Artem, > > Yes, everything is in the same AWS account and same region. The goal is to > minimize the intra-AZ d

Re: Logging configuration to see from which AZ consumers are fetching data in AWS environment

; client.rack=us-east-1a > > > > Are your clients in the same AWS account as your Kafka cluster is? > > On Fri, Nov 1, 2024 at 10:46 AM Soham Chakraborty > wrote: > > > Hi, > > > > Thank you for replying. > > > > We are using Java as the client library

Re: Logging configuration to see from which AZ consumers are fetching data in AWS environment

consumers? > > Regards, > OSB > > On Wed, Oct 30, 2024, 13:51 Soham Chakraborty wrote: > > > Hi, > > > > We are trying to implement the feature from KIP 392 to allow consumers to > > fetch from the closest replica. We are using AWS and Kubernetes (kOps) >

Logging configuration to see from which AZ consumers are fetching data in AWS environment

Hi, We are trying to implement the feature from KIP 392 to allow consumers to fetch from the closest replica. We are using AWS and Kubernetes (kOps) and the consumers do not always fetch from the broker in the same AZ, resulting in cross AZ traffic cost. The entire purpose of this mail/exercise is

Question about moving kafka brokers between AZs in AWS

Hello, Here is the scenario. We have a kafka/zookeeper ensemble, where kafka and zookeeper are installed on the same node (instance). There are three zk nodes and two kafka nodes. In other words, out of the three zk nodes, kafka is present in two. Now, these two kafka brokers are in different AZ