DoumanAsh opened a new issue, #23341: URL: https://github.com/apache/datafusion/issues/23341
### Describe the bug Current implementation of caching will result in performing full listing of the table despite user explicit request for pruning partitions This means that user MUST disable cache if he wants to perform efficient data lookup with partition pruning This should be documented if it is not intended to be fixed https://github.com/apache/datafusion/blob/main/datafusion/datasource/src/url.rs#L418 ### To Reproduce _No response_ ### Expected behavior Only cache paths by prefix, not full table ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
