cisaacson commented on issue #13994: URL: https://github.com/apache/datafusion/issues/13994#issuecomment-2574327502
Thanks @jonahgao , this is very helpful. The documentation does not fully reflect this, I will try and update it. The way I have things now I am not dependent on the DataFusion `filters` but I want it to know what my custom data source can do so that it does not do the work itself. I see now if it knew that a `filter` was labeled `Exact` and later we return all `Unsupported` from subsequent calls, the first `Exact` should be respected. But then the `filter` may not be pushed down as the DataFusion plan does not require it. I will still return the right data regardless the way I have it now. This did require an external (to DataFusion) implementation for my custom data source. Unless there is an important reason to keep a non-mutable `&self` in the `supports_filters_pushdown` I would recommend change it to `&mut self`. This should be OK as you can only modify the custom data source struct which the implementor owns. Do you agree? If so I will file an enhancement issue. @alamb you have been watching this issue I believe, let me know if you have an opinion on making mutable references for some of these functions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
