Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-09-30 Thread Chia-Ping Tsai
> - Original Message - > From: "Chia-Ping Tsai" > To: dev@kafka.apache.org > Sent: Fri, 05 Sep 2025 18:28:51 -0000 > Subject: Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes > config > > hi Lan > > chia_05: should we add metrics f

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-09-26 Thread Lan Ding
00 Subject: Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config hi Lan chia_05: should we add metrics for this buffer cache? For instance, coordinator-append-buffer-size-bytes and coordinator-append-buffer-skip-cache-count. Those information will be useful to kafka

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-09-05 Thread Chia-Ping Tsai
hi Lan chia_05: should we add metrics for this buffer cache? For instance, coordinator-append-buffer-size-bytes and coordinator-append-buffer-skip-cache-count. Those information will be useful to kafka maintainer. chia_06: typo: `CoordinatorRunntime` -> `CoordinatorRuntime` Best, Chia-Ping O

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-28 Thread Lan Ding
Hi Chia-Ping, Thanks for the suggestions. chia_04: Updated the default value of "max.buffer.size". Best, Lan At 2025-08-25 04:34:27, "Chia-Ping Tsai" wrote: >hi Lan > >chia_04: > >The default value of maximum message size is `1024 * 1024 + >Records.LOG_OVERHEAD`. Should the new configurations be a

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-24 Thread Chia-Ping Tsai
hi Lan chia_04: The default value of maximum message size is `1024 * 1024 + Records.LOG_OVERHEAD`. Should the new configurations be aligned with it? Best, Chia-Ping Lan Ding 於 2025年8月22日 週五 上午9:12寫道: > Hi Sean, > > You're absolutely right. Thanks for summarizing it so clearly! > > That's als

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-21 Thread Lan Ding
Hi Sean, You're absolutely right. Thanks for summarizing it so clearly! That's also the reason I renamed the config the original name could indeed be misleading. Best, Lan At 2025-08-22 01:42:09, "Sean Quah" wrote: >Hi Lan, >Thanks for your response. I misunderstood the purpose of the p

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-21 Thread Lan Ding
Hi Chia-Ping, Thanks for review. chia_03: As the number of consumers in a cluster dynamically increases, the size of records may also change dynamically. If a large number of consumers join the cluster while the `max.buffer.size` remains set to a relatively small value, performance could poten

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-21 Thread Sean Quah
Hi Lan, Thanks for your response. I misunderstood the purpose of the proposed `max.buffer.size` configs earlier. I thought we were going to reject messages larger than the `max.buffer.size`, but that's not the case. The new configs only control the buffer caching behavior and we will continue to al

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-21 Thread Chia-Ping Tsai
hi Lan chia_03: Since the hard limit between `message.max.bytes` and this new config was removed, is it still necessary to make `max.buffer.size` dynamic? Users won’t encounter errors when decreasing `message.max.bytes` anymore. Best, Chia-Ping On 2025/08/17 16:37:40 Lan Ding wrote: > Hello

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-21 Thread Lan Ding
gt;> >> AS1: This is a change to the coordinator runtime and there >> >> are two concrete implementations, the group coordinator and >> >> the share coordinator. I think it would be equally valuable >> >> to have `share

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-21 Thread Sean Quah
uld be equally valuable > >> to have `share.coordinator.append.max.buffer.size`. > >> The same default of 1*1024*1024 seems appropriate to me. > >> > >> Thanks, > >> Andrew > >> > >> Fr

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-20 Thread Lan Ding
.coordinator.append.max.buffer.size`. >> The same default of 1*1024*1024 seems appropriate to me. >> >> Thanks, >> Andrew >> ________ >> From: Lan Ding >> Sent: 20 August 2025 14:57 >> To: dev@kafka.apache.org >> Subject: Re: [DISCUSS] KIP-119

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-20 Thread Sushant Mahajan
ually valuable > to have `share.coordinator.append.max.buffer.size`. > The same default of 1*1024*1024 seems appropriate to me. > > Thanks, > Andrew > > From: Lan Ding > Sent: 20 August 2025 14:57 > To: dev@kafka.apache.org > Subject: Re: [DISCUSS] KIP-1196:

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-20 Thread Andrew Schofield
same default of 1*1024*1024 seems appropriate to me. Thanks, Andrew From: Lan Ding Sent: 20 August 2025 14:57 To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config Hi Chia-Ping, Thanks for the

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-20 Thread Lan Ding
Hi Chia-Ping, Thanks for the feedback. The original configuration name was potentially ambiguous; it has therefore been renamed to `group.coordinator.append.max.buffer.size`. chia_00: Update the vaild values of group.coordinator.append.max.buffer.size to be less than message.max.byt

Re: [DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-17 Thread Chia-Ping Tsai
hi Lan chia_00: The `group.coordinator.append.max.bytes` must be smaller than the max message size; Otherwise, it could generate an invalid record chia_01: Since the max message size is a dynamic configuration, should we also make `group.coordinator.append.max.bytes` dynamic? chia_02: Is comp

[DISCUSS] KIP-1196: Introduce group.coordinator.append.max.bytes config

2025-08-17 Thread Lan Ding
Hello everyone, I'd like to discuss a KIP regarding introducing a new configuration, group.coordinator.append.max.bytes Thank you! KIP link: https://cwiki.apache.org/confluence/x/hA5JFg Best, Lan Ding