logan-keede commented on issue #14444: URL: https://github.com/apache/datafusion/issues/14444#issuecomment-2688791206
>Seems like that creates a dependency problem I'm not sure how to untangle. catalog::Session will need datasource::FiltFormatFactory, which needs FileFormat, which needs FileSource, which needs FileScanConfig I remember contemplating over that, I think I had a loose sketch but I don't recall anything directly useful at the moment. > I can merge datafusion-datasource into datafusion-catalog, but also open to other ideas, and I'll also keep at it later today. Hopefully something will come up that won't require a huge breaking change. 1. I think we should try to avoid that mostly because it seems like a regression in term of modularity and code quality. (maybe we can only move some part of either or both crates up and down the dependency tree instead, not sure about this.) 2. It is my understanding that we do eventually want to move `SessionState` and everything else out of core, right? Following on that, I think the recent refactor has allowed us to move more stuff out of core (like `CteWorktable and `catalog-common`), which may also be direct or indirect dependency of `SessionState`. I suggest that we take an Incremental approach and move those out first, that should help us identify what are the actual blockers* and then we can target them separately. does that make sense? *Highly coupled code that is hindering refactor. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
