Re: [DISCUSS] KIP-1178: Introduce remote.max.partition.fetch.bytes in Consumer

returned for a partition than expected. > > Thanks, > Andrew > > ________ > From: Kamal Chandraprakash > Sent: 08 May 2025 19:24 > To: dev@kafka.apache.org > Subject: Re: [DISCUSS] KIP-1178: Introduce > remote.max.partition.fetch.

Re: [DISCUSS] KIP-1178: Introduce remote.max.partition.fetch.bytes in Consumer

FetchRequest and how sensitive it is to more data being returned for a partition than expected. Thanks, Andrew From: Kamal Chandraprakash Sent: 08 May 2025 19:24 To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-1178: Introduce remote.max.partition.fetch.by

Re: [DISCUSS] KIP-1178: Introduce remote.max.partition.fetch.bytes in Consumer

Hi Andrew, Thanks for the review! The initial idea was to introduce the configuration on the broker side, similar to how remote.fetch.max.wait.ms complements fetch.max.wait.ms: - fetch.max.wait.ms — configured via ConsumerConfig

Re: [DISCUSS] KIP-1178: Introduce remote.max.partition.fetch.bytes in Consumer

Hi Kamal, Thanks for the KIP. While it makes a lot of sense to me to be able to control the fetching from remote storage to make sure it's sympathetic to the characteristics of the storage provider, it seems to me that extending this concept all the way to the individual consumers is not a good