adragomir opened a new pull request, #25193: URL: https://github.com/apache/datafusion/pull/25193
- ATM, the Parquet file schema type coercion applies view transforms for example only on top-level fields, but not on nested fields. - For some Parquet schemas, where we request a view transform for a nested field, this translated to an unmodified schema passed to the reader. Then, when we try to apply predicates on the actual data, the actual predicate execution needs a cast, because the schemas are different. ## Which issue does this PR close? - Closes #25192 ## Rationale for this change - Rationale is explained in #25192 - Also, I don't see any reason (explicit comments) why this function would not work on deeply nested schemas ## What changes are included in this PR? - Modification of the `apply_file_schema_type_coercions` function to work on nested fields - Respective tests for the functionality ## What is the testing strategy for this PR? - All tests pass - Newly added unit tests pass ## Are there any user-facing changes? - Unsure ? Probably not ? The documentation in the function does NOT explicitly say the current behavior. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
