Hello Anton, Thank you for the FIP. It's a great proposal as it extensively thought out the implementation (crate boundaries, responsibilities, implementation challenges) and trade offs, with following questions and comments.
1. I echo Jim's question on implementation boundary with FIP-48. If there's necessity that drove the overlap, it's worth calling out and discuss potential changes to either FIP to reduce code maintenance. 2. Currently the unqualified table name implicitly means bounded union read as a side effect of planning, but I think this deserves to be an explicit semantic decision since it is a one-way door. When we later add lake-only, changelog or unbounded reads, the query text itself cannot express the distinction (boundedness in DataFusion is a source property, not a query property), so the choice has to live somewhere deliberate like a table name convention, a registration option or session config. Should we reserve that surface (or at very least for bounded vs unbounded) now even if implementation is deferred, e.g. plain name = bounded union read, $lake = lake-only, $changelog = the eventual unbounded surface, and snapshot reads via the pinned catalog provider? Again great proposal and overall +1 from me once the above are addressed. Best regards Keith On Fri, 14 Aug 2026 at 04:57, Jim Hu <[email protected]> wrote: > > > > Hi Anton, > > > > Thanks for putting together this detailed FIP. The motivation for an > > embeddable query engine on top of the Rust client makes sense to me, and > I > > also like that DataFusion is kept outside the core client. The discussion > > around correctness and resource constraints is very helpful. > > > > I may have missed some details, but I have a question about how this > > proposal is intended to work with FIP-48. > > > > I see that tiered reads are listed as depending on FIP-48, and that lake > > data > > enters DataFusion through the lake reader. What I could not tell is > whether > > fluss-datafusion consumes the complete Union Read plans, opaque splits > and > > readers defined by FIP-48, or only uses the lake-format reader while > owning > > the boundary handling and reconciliation itself. > > > > My understanding of FIP-48 is that snapshot/log boundary freezing, > lake/log > > stitching and PK current-view reconciliation belong to the engine-neutral > > fluss-lake layer. If FIP-50 intends to place some of these > responsibilities > > in DataFusion instead, should the boundary defined by FIP-48 be adjusted? > > Otherwise, could we end up with two implementations of seam handling, > retry > > semantics and PK reconciliation? > > > > Thanks again for this great FIP! > > > > Best, > > Jim > > >
