Hi, We have some improvement for the preferred read replica configured case. Ex: https://github.com/apache/kafka/pull/11942 https://github.com/apache/kafka/pull/11965
I know one improvement will be included in the v3.2.0 release, which will be released soon. Maybe you can give it a try to see if it improves the throughput. Thank you. Luke On Wed, May 11, 2022 at 2:56 PM benitocm <benit...@gmail.com> wrote: > Hi, > > We are using the functionality provided by KIP-392 (a consumer can fetch > the data from a ISR replica instead of the partition leader) in a Kafka > cluster stretched between two very close DCs (average round-trip latency > about 2 milliseconds). > > What we have seen is that, on average, when the consumer is in the same DC > (configured by rack.id) as the partition leader (i.e. the consumer will > consume from the leader), the time that takes the message to get to the > consumer is close to 20 milliseconds. However, when the consumer is in a > different DC than the partition leader (the consumer will consume from a > replica that is in the same DC as the consumer) that latency goes to around > 400 milliseconds. > > We have also checked that if we dont configure the rack.id in a consumer > to force it to consume from the leader although the partition leader is a > different DC (i.e. the consumer is in DC1 and the partition leader is in > DC2 so the consumer goes from a DC to the other DC) , the latency is > reduced to the 20 milliseconds. > > From those tests, we have concluded that consuming from a ISR replica > implies to have higher latencies. > > Please does anybody share any thoughts on this? > > Thanks in advance >
