RE: Sudden imbalance between partitions

2023-03-27 Thread Margaret Figura
rg Subject: Re: Sudden imbalance between partitions CAUTION: External Email : Be wary of clicking links or if this claims to be internal. Meg, What version are your clients, and what partitioner are you using for these records? If you're using the DefaultPartitioner from 2.4.0+, it has a

Re: Sudden imbalance between partitions

2023-03-24 Thread Greg Harris
Meg, What version are your clients, and what partitioner are you using for these records? If you're using the DefaultPartitioner from 2.4.0+, it has a known imbalance flaw that is described and addressed by this KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+St

Sudden imbalance between partitions

2023-03-24 Thread Margaret Figura
Hi, We have a 22-node Kafka 3.3.1 cluster on K8s. All data is sent with null partitionId and null key from 20 Java producers, so it should be distributed evenly across partitions. All was good for days, but a couple hours ago, broker 21 started receiving about 2x the data of the other brokers f