Hi all, I would like to start a discussion for FIP-50: https://cwiki.apache.org/confluence/spaces/FLUSS/pages/446071500/FIP-50+Native+DataFusion+Query+Engine+for+the+Fluss+Rust+Client
Querying Fluss today means Flink or Spark, and both bring their own cluster runtime, so FIP-50 proposes a new crate, fluss-datafusion, that registers a Fluss cluster as a DataFusion catalog so that SELECT * FROM fluss.db.t runs in the calling process. It is a library and a query gateway would sit above it and make it usable for a larger subset of tools/clients. Most of what the crate needs from the core client already exists in the Java client, so the work is porting rather than design: filter pushdown, KvBatchScanner for the FIP-17 KV scan, LookupBatch, FlussBucketingFunction.bucketForRowKey, and SortMergeReader for the primary key merge. FIP-50 sits directly above FIP-48: the union read kernel supplies the lake side and this proposal supplies the catalog, planner and pushdown above it, which means tiered reads in FIP-50 depend on FIP-48. Looking forward to your feedback. -- Anton
