aditanase commented on PR #15201: URL: https://github.com/apache/datafusion/pull/15201#issuecomment-2736011205
Hi @jayzhan211 @alamb, I'm a bit concerned with this patch, I was just in the process of submitting an enhancement to `InlineTableScan` that was dealing with multiple levels of views registered with `into_view`. https://github.com/hstack/arrow-datafusion/commit/0f79c370373cae04c541c91862cb9bf0ab2c737f The proposal on the original implementation was that we should be transforming down instead of up, to allow the child plans to be expanded all the way down. The downside with not doing inlining all the way to the original scan is that the various optimizer stages don't have full visibility and will run in separate stages. For example from the 2nd level of nesting, logical planning will only be triggered in the physical planning of the 2nd `DataFrameTableProvider`. What are your thoughts around this? How are you dealing with a query plan that is built incrementally from 2-3 levels of using `into_view`? (resulting in chained `DataFrameTableProvider` sources along the way). I'm not seeing any tests handling this use case yet, and my first attempt of cherry-picking your commit breaks in other places for our fork. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
