I created a new repo in the datafusion-contrib GitHub org over the weekend
with a starting point for supporting DataFusion as both a producer and
consumer of Substrait plans.

https://github.com/datafusion-contrib/datafusion-substrait

I am hopeful that we can eventually use Substrait in Ballista as a
replacement for the current query plan protobuf format, meaning that the
Ballista scheduler could potentially be used with engines other than
DataFusion.

I also think it could be helpful with in-memory language interoperability,
such as passing query plans between Python and Rust.

I plan on continuing to merge my own PRs here as I flesh out more of this,
at least until there are other contributors.

Thanks,

Andy.

Reply via email to