Re: [DISCUSS] KIP-1183: Unified Shared Storage

2025-08-06 Thread Jun Rao
Hi, Xinyu, Thanks for the KIP. A few high level comments. JR1. One potential downside of using RF=1 is availability. Since there is no active standby replica, on a failover, the new leader needs to recover the log and rebuild/reload the state before it can serve writes. So, the window of unavaila

Re: [DISCUSS] KIP-1183: Unified Shared Storage

2025-05-15 Thread Xinyu Zhou
Hi Colin, Thank you for taking the time to read this KIP, and no worries, negative feedback is a catalyst for improvement. Sorry for the inappropriate description in the Motivation section; my background influenced my writing, but I didn’t mean it. I will remove it. Thanks for the reminder. I co

Re: [DISCUSS] KIP-1183: Unified Shared Storage

2025-05-15 Thread Colin McCabe
Hi Xinyu Zhou, Thanks for the KIP. It's good to see more people contributing to the community. I think this is your first KIP, so please forgive me for giving some negative feedback. KIPs need to be written in a vendor-neutral manner, for the whole community. So please do not do things like be

Re: [DISCUSS] KIP-1183: Unified Shared Storage

2025-05-14 Thread Xinyu Zhou
Hi Satish, Thanks for your incisive comments! For SD-1: I am now confident that we have a better chance of reaching a consensus on the abstract layer. I will complete this part and address all the questions from you and Luke. Once finished, I will ping you again. For SD-2: In this KIP, a partiti

Re: [DISCUSS] KIP-1183: Unified Shared Storage

2025-05-14 Thread Satish Duggana
Thanks Xinyu for accepting our request to initiate a KIP on AutoMq's proposal. SD-1: It is a good start towards building log storage layer abstractions, including LogSegment and Log, for both local and shared storage. However, more clarity is needed on how these abstractions are defined. The class

Re: [DISCUSS] KIP-1183: Unified Shared Storage

2025-05-14 Thread Xinyu Zhou
Hi Luke, Thank you for your review and insightful questions. 1. Regarding the future plan for this KIP, once we establish an abstract log layer, we might implement a default version on object storage for Kafka. For other types of shared storage, we'll decide based on community needs. 2. In my vi

Re: [DISCUSS] KIP-1183: Unified Shared Storage

2025-05-14 Thread Luke Chen
Hi Xinyu, Thanks for the proposal! I agree abstracting the log and logSegment layers can allow users to implement any kinds of storage mediums. Some high-level comments: 1. I expect to see the future plan of this KIP, but can't find it. What do you expect to achieve the "shared storage" goal afte