asolimando commented on issue #27600:
URL: https://github.com/apache/datafusion/issues/27600#issuecomment-7057438877

   Thanks @stuhood, keeping `RangePartitioning` minimal and carrying emptiness 
in `Statistics` makes sense to me. I did a quick pass on your prototype to 
validate your proposal, and I noticed two points worth sharing beyond my 
validation above:
   
   1. I have verified that `estimate_join_statistics` reports a provably empty 
join as `Inexact(0)` instead of `Exact(0)`, so exchanges that skip `Exact(0)` 
inputs cannot skip it. This fix does not depend on the partitioning change, I 
have filed https://github.com/apache/datafusion/issues/27629 to track it.
   
   2. `ElementwiseFromChildren` relies on operators not reading the partition 
index, as stated in the doc. A test that compares the batch result with the 
per-partition walk for each operator that opts in would catch a later change 
that breaks this.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to