andygrove opened a new pull request, #47: URL: https://github.com/apache/datafusion-ray/pull/47
This PR brings back the original disk-backed shuffle. There is a limitation that all workers must have the same local disk (so limited to single node) but we can follow up to add a mechanism for reading shuffle files from other nodes via Arrow Flight to remove this limitation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
