pepijnve commented on PR #27210: URL: https://github.com/apache/datafusion/pull/27210#issuecomment-6978288834
@alexanderbianchi > Effectively that means removing the Enum and making AsOfVersion / AsOfTimestamp an implementation of TableReadSelectorExtension. I've been thinking about exactly that since for the most part DataFusion core treats that value as opaque anyway. Why bother with these two special cases? The only complexity that introduces is handling formatting in explain output and in the unparser. If we want to support that we would need to delegate the formatting/unparsing work to the extension point as well which would then have to deal with the dialect differences. Doesn't seem ideal either. @alamb > I thought the model was that (effectively) there were different Arc<dyn TableProvider>s for each union input Yes, the table providers are indeed distinct, but logical plan equality doesn't and can't take that into account since `TableSource` is not required to have a `PartialEq` implementation. The only real discriminator is the `TableReference` and that's the same for both. That's why it's being augmented here with an identity representing value that is `PartialEq`. > For example, I would love to help ensure that https://github.com/apache/datafusion-iceberg ends up supporting time travel queries too > So as a practical step forward, what do you think about making an example, in the spirit of https://github.com/apache/datafusion/blob/b7b014e731/datafusion-examples/examples/data_io/remote_catalog.rs#L38-L96 (or maybe extend it) to show how to connect in AS OF type queries? Our end goals are the same. I'm implementing this in function of our internal (I know, yet another one) fork of the alternative iceberg implementation. I'll see about extending the remote catalog example. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
