There is a proposal [1] to bring the Ray SQL project [2] into the DataFusion Python project.
This would mean that DataFusion Python can be run both in-process and also scale out on Ray clusters. Please feel free to comment on the proposal if you have any feedback. Thanks, Andy. [1] https://github.com/apache/datafusion-python/issues/872 [2] https://github.com/datafusion-contrib/ray-sql
