Thanks, Luke. > But if the producers and consumers all existed in the same organization, which means upgrading producers/consumers for the org's cost saving, should be a reasonable motivation.
Yeah, that works in this case. However, Kafka is often used as a service (on premise or in cloud) nowadays and in this case the producers/consumers versions are completely out of control thus my concern. BR, David On Fri, May 12, 2023 at 10:47 AM Luke Chen <show...@gmail.com> wrote: > Hi David, > > Yes, you're right. I've bumped the version of record batch, and describe > the down-conversion will happen like what we do for message format v1 now > when old consumers consuming records. > > > Overall, I wonder if the bandwidth saving is worth this change given that > it will put more pressure on the brokers. > Actually, I'm not 100% sure. So I'd also like to hear what the community > thought about it. > But if the producers and consumers all existed in the same organization, > which means upgrading producers/consumers for the org's cost saving, should > be a reasonable motivation. > > Thanks. > Luke > > > On Fri, May 12, 2023 at 3:43 PM David Jacot <dja...@confluent.io.invalid> > wrote: > > > Hi Luke, > > > > Thanks for the KIP. > > > > What do we do in the case where a batch is written with > > `ignoreMessageAttributes` set to 1, which means that messages won't have > > the `attributes`, and is consumed by a consumer which does not understand > > this new format? I suppose that we would need to introduce a new version > > for the message format (v3) and that we will have to downconvert records > > from the new format version to v2 in this case. This is not clear in the > > KIP. Could you elaborate a bit more on this? Overall, I wonder if the > > bandwidth saving is worth this change given that it will put more > pressure > > on the brokers. > > > > Best, > > David > > > > On Fri, May 12, 2023 at 9:04 AM Luke Chen <show...@gmail.com> wrote: > > > > > Hi all, > > > > > > I'd like to start a discussion for the KIP-931: Flag to ignore unused > > > message attribute field. This KIP is to add a flag in the batch header > to > > > indicate if messages inside the batch have attribute field or not, to > > > reduce the message size, thus, save network traffic and storage size > (and > > > money, of course). > > > > > > Please check the link for more detail: > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-931%3A+Flag+to+ignore+unused+message+attribute+field > > > > > > Any feedback is welcome. > > > > > > Thank you. > > > Luke > > > > > >
