Hi Anton and Keith, Thank you very much for the detailed and thoughtful feedback. These are all valid points that deserve clearer treatment in the FIP, particularly around the plan/read consistency contract, descriptor compatibility, retry and liveness semantics, resource bounds, PK merge semantics, split parallelism, and object-storage security.
I’m considering these points and revising the FIP accordingly. I’ll follow up with a revised version. Best, Jim > 2026年8月6日 上午10:47，Jim Hu <[email protected]> 写道： > > Hi, all > > Thanks Anton and Keith for > > Keith Lee <[email protected] <mailto:[email protected]>> 于2026年7月31日周五 > 18:59写道： >> Hello Jim, >> >> Thank you for the FIP, the general direction looks solid. I have the >> following questions: >> >> 1. Have we considered timeout, liveness or retry semantics? A hung storage >> request or dead server would stall the stream indefinitely without timeout. >> This also apply for PK splits where merge must first materialise the whole >> changeling tail before emitting anything. It might be worth adding >> configurable timeouts. Without this, user will have to resort to wrapping >> the kernel with watchdog. >> 2. On PK hash overlay, the proposal relies on the premise that tail is >> bounded and small. There's no defined behaviour when that premise fails. >> Could we specify a failure mode? e.g. configurable memory / tail-size cap >> surfacing an error such as FlakeLakeError("snapshot too stale") rather than >> risking OOMing the host process? >> 3. Can we include in the proposal on the credential and security story for >> object storage? I appreciate that the proposal might be following Java side >> implementation, calling it out in the proposal would make it clear. >> >> It's a great FIP, looking forward to your response. >> >> Best regards >> Keith >> >> On Thu, Jul 30, 2026 at 4:15 PM Jim Hu <[email protected] >> <mailto:[email protected]>> wrote: >> >> > Hi all, >> > >> > I'd like to start a discussion on FIP-48: Introduce a Union Read Kernel >> > for fluss-rust. >> > >> > The proposal is available here: >> > >> > https://cwiki.apache.org/confluence/spaces/FLUSS/pages/444334625/FIP-48+Introduce+a+Union+Read+Kernel+for+fluss-rust >> > >> > Union read is already available to Java engines through the Flink >> > connector (FIP-6). FIP-48 brings the same capability to fluss-rust, so that >> > non-JVM engines (StarRocks, DataFusion, Python) can read lake-enabled >> > tables without embedding a JVM. This FIP was drafted together with beryllw. >> > >> > In short, it proposes a new fluss-lake crate (Paimon-first) providing: >> > >> > - A bounded batch read model: one lake snapshot plus its bounded log tail, >> > stitched at the readable-snapshot seam with no overlap and no gap >> > - A table -> scan -> plan -> read_split contract with distributable, >> > serializable, versioned splits; engines schedule splits and consume Arrow >> > streams without reimplementing seam or merge logic >> > - Three read semantics: append/log tables, primary-key tables (merged >> > current view), and lake-only >> > - Predicate pushdown with automatic partition/bucket pruning, plan-level >> > statistics, and a typed error surface >> > >> > Looking forward to your feedback and suggestions. Please feel free to >> > raise questions in this thread, and I will keep updating FIP-48 based on >> > the discussion. If there are no further concerns, I plan to start a vote >> > next week. >> > >> > Best regards, >> > Jim
