Hi Kamal, Thanks for the KIP. While it makes a lot of sense to me to be able to control the fetching from remote storage to make sure it's sympathetic to the characteristics of the storage provider, it seems to me that extending this concept all the way to the individual consumers is not a good idea. You might have different consumers specifying their own wildly different values, when really you want a consistent configuration which applies whenever data is fetched from remote storage. Could a broker or topic config be used to achieve this more effectively? It worries me whenever we have a configuration which would ideally be used by all consumers setting the same value. It suggests that they shouldn't be able to provide their own values at all.
Thanks, Andrew ________________________________________ From: Kamal Chandraprakash <kamal.chandraprak...@gmail.com> Sent: 08 May 2025 12:07 To: dev@kafka.apache.org <dev@kafka.apache.org> Subject: [DISCUSS] KIP-1178: Introduce remote.max.partition.fetch.bytes in Consumer Hi all, I've opened the KIP-1178 to add a new config 'remote.max.partition.fetch.bytes' in the consumer. This config allows it to read from remote storage faster. https://cwiki.apache.org/confluence/display/KAFKA/KIP-1178%3A+Introduce+remote.max.partition.fetch.bytes+config+in+Consumer Please take a look and suggest your thoughts. Thanks, Kamal
