avantgardnerio commented on PR #24035: URL: https://github.com/apache/datafusion/pull/24035#issuecomment-5246515024
@timsaucer thank you for the thorough review. These are all legitimate concerns: 1. I was not able to reproduce this, but I added a test anyway, as well as your suggested mitigation 2. Following the belt-and-suspenders theme, I made this case hard-error in a central place 3. I've updated all the construction sites to include the callback during clones 4. It now prevents installation of a callback if any frames are sliding. I think all of the above stem from this trying to fire per-partition-key, rather than once, at the end, when the motivating case is prefix scanning. What would you think about a getter shape (#24031 https://github.com/apache/datafusion/pull/24031)) where a downstream node reads Some(state) / None from the BWAG it already holds via input()? Scope is single-group-per-output-partition (matches distributed prefix-scan; wouldn't cover per-key PARTITION BY windows). Concerns 1/2/3 collapse to not-applicable, and 4 is a one-line gate at the write site. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
