>
> Hi Anton,
>
> Thanks for putting together this detailed FIP. The motivation for an
> embeddable query engine on top of the Rust client makes sense to me, and I
> also like that DataFusion is kept outside the core client. The discussion
> around correctness and resource constraints is very helpful.
>
> I may have missed some details, but I have a question about how this
> proposal is intended to work with FIP-48.
>
> I see that tiered reads are listed as depending on FIP-48, and that lake
> data
> enters DataFusion through the lake reader. What I could not tell is whether
> fluss-datafusion consumes the complete Union Read plans, opaque splits and
> readers defined by FIP-48, or only uses the lake-format reader while owning
> the boundary handling and reconciliation itself.
>
> My understanding of FIP-48 is that snapshot/log boundary freezing, lake/log
> stitching and PK current-view reconciliation belong to the engine-neutral
> fluss-lake layer. If FIP-50 intends to place some of these responsibilities
> in DataFusion instead, should the boundary defined by FIP-48 be adjusted?
> Otherwise, could we end up with two implementations of seam handling, retry
> semantics and PK reconciliation?
>
> Thanks again for this great FIP!
>
> Best,
> Jim
>

Reply via email to