andygrove commented on issue #1944: URL: https://github.com/apache/datafusion-ballista/issues/1944#issuecomment-4895997862
Superseded by #1959, which adds the join-strategy dimension (SMJ vs ShuffledHashJoin across all three engines). Key update: enabling hash join on Ballista (`prefer_hash_join=true`) cuts the SF100 suite −29% (560s → 398s) with no OOM, bringing it within ~15% of Comet and ~1.5× faster than vanilla Spark — most of the gap in this snapshot was the SMJ-vs-SHJ default mismatch, not an engine difference. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
