pepijnve commented on PR #27210:
URL: https://github.com/apache/datafusion/pull/27210#issuecomment-6978288834

   @alexanderbianchi 
   
   > Effectively that means removing the Enum and making AsOfVersion / 
AsOfTimestamp an implementation of TableReadSelectorExtension.
   
   I've been thinking about exactly that since for the most part DataFusion 
core treats that value as opaque anyway. Why bother with these two special 
cases? The only complexity that introduces is handling formatting in explain 
output and in the unparser. If we want to support that we would need to 
delegate the formatting/unparsing work to the extension point as well which 
would then have to deal with the dialect differences. Doesn't seem ideal either.
   
   @alamb 
   
   > I thought the model was that (effectively) there were different Arc<dyn 
TableProvider>s for each union input
   
   Yes, the table providers are indeed distinct, but logical plan equality 
doesn't and can't take that into account since `TableSource` is not required to 
have a `PartialEq` implementation. The only real discriminator is the 
`TableReference` and that's the same for both. That's why it's being augmented 
here with an identity representing value that is `PartialEq`.
   
   > For example, I would love to help ensure that 
https://github.com/apache/datafusion-iceberg ends up supporting time travel 
queries too
   > So as a practical step forward, what do you think about making an example, 
in the spirit of 
https://github.com/apache/datafusion/blob/b7b014e731/datafusion-examples/examples/data_io/remote_catalog.rs#L38-L96
 (or maybe extend it) to show how to connect in AS OF type queries?
   
   Our end goals are the same. I'm implementing this in function of our 
internal (I know, yet another one) fork of the alternative iceberg 
implementation. I'll see about extending the remote catalog example.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to