mbutrovich commented on code in PR #1756:
URL: https://github.com/apache/datafusion-comet/pull/1756#discussion_r2098530199
##########
native/core/src/parquet/schema_adapter.rs:
##########
@@ -196,15 +198,43 @@ impl SchemaMapper for SchemaMapping {
// go through each field in the projected schema
.fields()
.iter()
+ .enumerate()
// and zip it with the index that maps fields from the projected
table schema to the
// projected file schema in `batch`
.zip(&self.field_mappings)
// and for each one...
- .map(|(field, file_idx)| {
+ .map(|((field_idx, field), file_idx)| {
file_idx.map_or_else(
- // If this field only exists in the table, and not in the
file, then we know
- // that it's null, so just return that.
- || Ok(new_null_array(field.data_type(), batch_rows)),
Review Comment:
Got rid of instantiating an entire null array in favor of a single null
value for column.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
