github-actions[bot] commented on code in PR #69032:
URL: https://github.com/apache/doris/pull/69032#discussion_r4608728224


##########
be/src/format/transformer/viceberg_parquet_writer.cpp:
##########
@@ -104,6 +149,9 @@ Status VIcebergParquetWriter::write(const Block& block) {
     if (block.rows() == 0) {
         return Status::OK();
     }
+    ColumnNumbers column_numbers(block.columns());
+    std::iota(column_numbers.begin(), column_numbers.end(), 0);
+    RETURN_IF_ERROR(validate_spatial_wkb_inputs(block, column_numbers));

Review Comment:
   [P1] Reject required spatial container NULLs before Arrow conversion
   
   For an imported V3 Parquet target `items` whose LIST element is required 
`STRUCT<g:GEOMETRY>`, `INSERT INTO dst(items) SELECT array(s) FROM src WHERE s 
IS NULL` (`src.s` is a nullable STRUCT with the same spatial metadata) passes 
FE validation because `s` is a runtime slot. This WKB-only check skips the NULL 
STRUCT. `DataTypeStructSerDe` then calls [Arrow 24 
StructBuilder::AppendNull](https://github.com/apache/arrow/blob/apache-arrow-24.0.0/cpp/src/arrow/array/builder_nested.h#L711-L717),
 which pads `g` with a valid empty binary value; its [Parquet level 
builder](https://github.com/apache/arrow/blob/apache-arrow-24.0.0/cpp/src/parquet/arrow/path_internal.cc#L708-L711)
 ignores the bitmap of a required STRUCT. The writer can therefore commit a 
present `g` containing zero-byte annotated WKB, and reading that column 
subsequently fails WKB validation. Validate runtime null maps against the 
complete Iceberg schema, including required intermediate containers, before 
Arrow conversion and 
 cover rejection of this write. This differs from the existing nested-literal 
validation thread: the NULL here is a runtime parent, and the invalid WKB is 
synthesized after validation. Source tracing only; no Doris test was executed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to