adriangb commented on issue #15780: URL: https://github.com/apache/datafusion/issues/15780#issuecomment-2898896876
As discussed a bit in https://github.com/apache/datafusion/pull/16086#discussion_r2100826502 there is a fundamental problem that all of the predicates are planned at the table level. So for example the predicate `func(col1)` where at the table / logical level the types are `col1: Utf8View`. If the file types are `col1: Uf8View` then how do we know that `func(Utf8View)` won't error? There's other cases... basically it's not clear to me that it's trivial to get there / I worry we have to re-invent casting rules at the physical layer. I remember @findepi said somewhere something along the lines of "casting should not be changed after planning" (please correct me or feel free to chime in) My feeling is that we eventually have to do this for performance (unnecessary casts / conversions) and [correctness reasons](https://github.com/apache/datafusion/pull/15821), but evidently my initial attempt was too naive so [we had to revert it](https://github.com/apache/datafusion/pull/16086). I am willing to try again later, especially that we now have some tests in place (we need more, e.g. for the example with a function above). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
