Hi Jun, I updated the KIP to document the points you have raised.
On Thu, Apr 24, 2025 at 1:34 PM Jun Rao <j...@confluent.io.invalid> wrote: > The follower of regular topic could have fetched to the end of the log from > the leader, but need to wait 500ms for the true HWM to be propagated. This > will delay the consumption of the consumer on the follower by 500ms since > it's gated by HWM, right? Yes. Good catch, I forgot that Kafka supports fetch-from-frollower. I updated the KIP and FETCH RPC section to provide some detail. > Also, in FetchResponse, we use -1 as the default for various offsets. > Should we follow that convention for the new field? The new HighWatermark field can have 3 types of value: 1. The sending replica supports version 18 and doesn't know the HWM 2. The sending replica supports version 18 and knows the HWM 3. The sending replica doesn't support version 18 or doesn't send the HighWatermark field The default value is meant to address case number 3. For this case, Kafka (KRaft and replica manager) should behave the same way it does today. If we set the default value to the maximum value for int64, the predicate to determine if the HWM has changed and the FETCH request should not be parked is just "local HWM > remote HWM." If we use a different default value (-1 or -2), the predicate is a bit more complicated: "remote HWM != -2 and local HWM > remote HWM". Thanks, -- -José
