github-actions[bot] commented on code in PR #68611:
URL: https://github.com/apache/doris/pull/68611#discussion_r4129809382
##########
be/src/format/arrow/arrow_row_batch.cpp:
##########
@@ -146,10 +146,17 @@ Status convert_to_arrow_type(const DataTypePtr&
origin_type,
std::shared_ptr<arrow::DataType> key_type;
std::shared_ptr<arrow::DataType> val_type;
RETURN_IF_ERROR(convert_to_arrow_type(type_map->get_key_type(),
&key_type, timezone,
- datetime_naive));
+ datetime_naive, map_as_list));
RETURN_IF_ERROR(convert_to_arrow_type(type_map->get_value_type(),
&val_type, timezone,
- datetime_naive));
- *result = std::make_shared<arrow::MapType>(key_type, val_type);
+ datetime_naive, map_as_list));
+ // Arrow MAP forbids null keys. A session-selected LIST representation
must be
+ // fixed before reading any batches, including batches without null
keys.
+ if (map_as_list) {
Review Comment:
[P2] Keep LIST pair-field nullability consistent with the MAP child types.
For a MAP with a nonnullable key or value child (as Lance can describe),
`maps_as_lists` retains that child type, but these `arrow::field` calls mark
both pair fields nullable. The exact check in `write_plain_arrow_column` then
rejects the transformed struct before writing a batch. `GetTables` can also
report a required value as nonnullable while this BE schema reports it
nullable. Derive both BE pair-field flags from the MAP child types, preserve
those flags in FE list metadata, and test nonnullable children in both paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
