alamb opened a new issue, #13714: URL: https://github.com/apache/datafusion/issues/13714
### Is your feature request related to a problem or challenge? Quoting from @westonpace on https://github.com/apache/datafusion/pull/13582 > Many catalogs are remote (and/or disk based) and offer only asynchronous APIs. For example, [Polaris](https://github.com/apache/polaris), [Unity](https://github.com/unitycatalog/unitycatalog), and [Hive](https://hive.apache.org/). Integrating with this catalogs is impossible since something like ctx.sql("SELECT * FROM db.schm.tbl") first enters an async context (sql) then a synchronous context (calling the catalog provider to resolve db) and then we need to go into an asynchronous context to interact with the catalog and this async -> sync -> async path is generally forbidden. This also came up in - https://github.com/apache/datafusion/issues/13525 I believe it is possible to interact with remote catalogs with DataFusion's non async CatalogAPIs but it is not obvious how to do so ### Describe the solution you'd like I would like a clear well documented example of a DataFusion catalog that interacts with a remote catalog ### Describe alternatives you've considered Another approach that is taken by the [SessionContext::sql](https://docs.rs/datafusion/latest/datafusion/execution/context/struct.SessionContext.html#method.sql) Is: Does an initial pass through the parse tree to find all references (non async) Then fetch all references (can be async) Then does the planning (non async) with all the relevant references I don't think this is particularly well documented ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
