HuaHuaY opened a new issue, #55124:
URL: https://github.com/apache/arrow/issues/55124
### Describe the bug, including details regarding any error messages,
version, and platform.
```cpp
auto type =
::arrow::struct_({::arrow::field("inner", ::arrow::int32(),
/*nullable=*/false)});
auto values = ::arrow::ArrayFromJSON(type, R"([{"inner": 42}])");
auto indices = ::arrow::ArrayFromJSON(::arrow::int32(), "[0, null]");
ASSERT_OK_AND_ASSIGN(auto array, ::arrow::compute::Take(*values, *indices));
const auto& structs = checked_cast<const ::arrow::StructArray&>(*array);
ASSERT_EQ(structs.null_count(), 1);
ASSERT_EQ(structs.field(0)->null_count(), 1);
auto table = ::arrow::Table::Make(
::arrow::schema({::arrow::field("outer", type, /*nullable=*/true)}),
{std::move(array)});
ASSERT_OK(table->ValidateFull());
CheckSimpleRoundtrip(table, /*row_group_size=*/2);
```
When checking whether a non-nullable leaf node contains null values, Parquet
fails to account for the nullability of the parent column; this causes valid
Arrow RecordBatches to fail during writing.
```cpp
// cpp/src/parquet/column_writer.cc
......
Status WriteArrow(const int16_t* def_levels, const int16_t* rep_levels,
int64_t num_levels, const ::arrow::Array& leaf_array,
ArrowWriteContext* ctx, bool leaf_field_nullable)
override {
BEGIN_PARQUET_CATCH_EXCEPTIONS
// Leaf nulls are canonical when there is only a single null element
after a list
// and it is at the leaf.
bool single_nullable_element =
(level_info_.def_level == level_info_.repeated_ancestor_def_level +
1) &&
leaf_field_nullable;
if (!leaf_field_nullable && leaf_array.null_count() != 0) {
return Status::Invalid("Column '", descr_->name(),
"' is declared non-nullable but contains
nulls");
}
```
### Component(s)
C++, Parquet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
