dwsmith1983 opened a new pull request, #25895: URL: https://github.com/apache/datafusion/pull/25895
## Which issue does this PR close? - Backport of #24790 to `branch-55`, for the 55.2.0 patch release (#25758). - Closes #24786 on `branch-55`. ## Rationale for this change With `datafusion.execution.parquet.coerce_int96` set, a file with an INT96 column comes back with the metadata of every struct, list and map field emptied, while leaf fields keep theirs. Readers that match fields by Parquet field id lose the id of any container whose id sits only on the container. Apache DataFusion Comet always enables the coercion, so with Spark's field id reads on it null fills such a container where Spark reads it (apache/datafusion-comet#6131). ## What changes are included in this PR? A clean cherry-pick of #24790 (with `-x`): the three container constructors in `schema_coercion.rs` keep the original field's metadata, plus that PR's tests. ## What is the testing strategy for this PR? The tests from #24790 come with the cherry-pick. The `datafusion-datasource-parquet` crate's tests, `cargo fmt` and `cargo clippy` pass on this branch. ## Are there any user-facing changes? Struct, list and map fields read from files with INT96 columns keep their field metadata when `coerce_int96` is set. No API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
