Re: [DISCUSS] KIP-1187: Support to retrieve remote log size via DescribeLogDirs RPC

2025-07-30 Thread PoAn Yang
Hi Chia-Ping, Thanks for the review. It looks like there is another thread which don’t match to the discussion thread on the KIP, so I change to response here. chia_00: Renamed `setIncludeRemoteInfo(boolean)` to `includeRemoteInfo(boolean)`. chia_01: Yes, like Kamal mentioned, the `PartitionSiz

Re: [DISCUSS] KIP-1187: Support to retrieve remote log size via DescribeLogDirs RPC

2025-07-28 Thread Kamal Chandraprakash
Hi Chia, Thanks for the review! > the `PartitionSize` does not include the `RemoteLogSize`, right? Yes, it is the local-log size. > Should we add similar fields for `TotalBytes` and `UsableBytes`? TotalBytes and UsableBytes may not be applicable for remote storage. As the storage implementer ca

Re: [DISCUSS] KIP-1187: Support to retrieve remote log size via DescribeLogDirs RPC

2025-07-20 Thread Chia-Ping Tsai
hi PoAn chia_00: should `setIncludeRemoteInfo(boolean)` be renamed to ` includeRemoteInfo(boolean)`? chia_01: the `PartitionSize` does not include the `RemoteLogSize`, right? chia_02: Should we add similar fields for `TotalBytes` and `UsableBytes`? Best, Chia-Ping ally heev 於 2025年7月17日 週四

Re: [DISCUSS] KIP-1187: Support to retrieve remote log size via DescribeLogDirs RPC

2025-07-20 Thread Kamal Chandraprakash
Hi PoAn, > Regarding `onlyLocalLogSize`, does it refer to local log size minus overlapping retention part? Yes, correct. https://sourcegraph.com/github.com/apache/kafka/-/blob/storage/src/main/java/org/apache/kafka/storage/internals/log/UnifiedLog.java?L2005 Thanks, Kamal On Mon, Jul 21, 2025 a

Re: [DISCUSS] KIP-1187: Support to retrieve remote log size via DescribeLogDirs RPC

2025-07-20 Thread PoAn Yang
Hi Kamal, Thanks for the feedback. It’s helpful to understand the overlapping parts between local and remote logs. Regarding `onlyLocalLogSize`, does it refer to local log size minus overlapping retention part? If so, I prefer to set the field when IncludeRemoteInfo is true. Renamed the `RemoteP

RE: [DISCUSS] KIP-1187: Support to retrieve remote log size via DescribeLogDirs RPC

2025-07-17 Thread ally heev
On 2025/06/16 14:32:34 PoAn Yang wrote: > Hi all, > > I would like to start a discussion thread about KIP-1187. > > Please take a look and feel free to share any thought. > > https://cwiki.apache.org/confluence/x/sYkhFg > > Thanks, > PoAn Minor comment: *describeLogDirs *literally means describing

Re: [DISCUSS] KIP-1187: Support to retrieve remote log size via DescribeLogDirs RPC

2025-06-27 Thread Kamal Chandraprakash
Hi PoAn, Thanks for the KIP! Having the remoteLogSize as part of the DescribeLogDirs response will be useful to determine the exact cost of the topic. Kafka uploads the segments eagerly. Assume a topic is configured with 48 hrs of retention time and 12 hrs of local-retention time. Then, the remot

[DISCUSS] KIP-1187: Support to retrieve remote log size via DescribeLogDirs RPC

2025-06-16 Thread PoAn Yang
Hi all, I would like to start a discussion thread about KIP-1187. Please take a look and feel free to share any thought. https://cwiki.apache.org/confluence/x/sYkhFg Thanks, PoAn