robtandy commented on issue #59: URL: https://github.com/apache/datafusion-ray/issues/59#issuecomment-2634885083
👋 Hi @sairamkrish ! I've been working on a revised version of datafusion ray for the past month or so. I submitted a PR today in fact. It replaces the disk based shuffle with ArrowFlight streaming shuffle, and is able to scale beyond one node. It should be in a place where it can build and work for you. You can read the PR here https://github.com/apache/datafusion-ray/pull/60 And access the branch from here: https://github.com/robtandy/datafusion-ray/tree/main/src If you have any trouble installing or using, you can open an issue here and I'll try to help. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
