asolimando commented on PR #25098: URL: https://github.com/apache/datafusion/pull/25098#issuecomment-5665863437
> On the cache reset: agreed it is a footgun to leave to consumers, but since it is tangential to this PR I have filed it as a follow-up (#25141) so the robustness can be tracked separately, and kept the safe reset-on-change here. Between your two ideas I lean toward (b) holding node `Arc`s in the cache, though as noted there it would widen the `compute` API a bit; captured both options on the issue. Happy to take that on. @zhuqi-lucas thanks for filing #25141, and I agree that (b) looks like the best option at the moment, as the least intrusive (the memory cost from the plan copy is not worrisome, and it's bounded in "time", after the rule application we can free the copy). I will start tackling https://github.com/apache/datafusion/issues/21120 for the next weeks, in case you have bandwidth to file a PR I'd be glad to review, otherwise I will probably get to this after https://github.com/apache/datafusion/issues/21120 gets finalized. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
