morningman opened a new issue, #69782: URL: https://github.com/apache/doris/issues/69782
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version branch-4.2 (checked at `9f63be85cea`) and branch-4.1 (checked at `5fc51d21256`), FileScannerV2 Iceberg reader. ### What's Wrong? `AncestorNullDefaultExpr` in `be/src/format_v2/table/iceberg_reader.cpp` never opens its children. Its `open()` is ```cpp RETURN_IF_ERROR_OR_PREPARED(VExpr::open(state, context, scope)); ``` and `RETURN_IF_ERROR_OR_PREPARED` (`be/src/exprs/vexpr.h`) returns `Status::OK()` as soon as `_prepared` is set, which `prepare()` has already done with the same macro. So `VExpr::open` is never called and none of the children is opened. The equality-delete key of a missing nested field is built in `_build_missing_equality_delete_key_expr`. When the table leaf type differs from the key type, the initial default is wrapped in a cast (`EvolutionCast` on branch-4.2, `Cast` on branch-4.1). When the data file has the key's parent struct, that cast becomes the second child of `AncestorNullDefaultExpr`, which returns NULL for a NULL parent and the default otherwise. The cast is therefore executed without having been opened, and it fails `Check failed: _open_finished || block == nullptr` in `Cast`, which aborts the BE. This follows from the code. It was not reproduced on branch-4.1 or branch-4.2. On master the same code was hit by a unit test, where the BE aborted exactly this way, and it is fixed in #69711 (commit `3c2a8e9cb87`) by opening the children the way `EqualityDeletePredicate` does. ### What You Expected? The scan applies the equality delete: rows under a NULL parent get a NULL key, other rows get the converted default, and the BE does not abort. ### How to Reproduce? Not reproduced on 4.x. From the code, the conditions are: 1. An Iceberg table with an equality delete keyed on a nested field (`s.k`). 2. A data file that has the parent struct `s` but not the field `k`, so `k` is resolved through its initial default (Iceberg v3 initial defaults). 3. The table leaf type of `k` differs from the key type the reader compares the key in (`filter.key_types`), so the default takes the `!table_leaf->type->equals(*filter.key_types[key_idx])` branch and gets the cast. 4. A non-NULL initial default, so the cast actually runs. The master unit test `IcebergMissingNestedPlaceholderKeyParsesTextDefaultInTombstoneType` in `be/test/format_v2/table/iceberg_reader_test.cpp` (in #69711) builds this shape. ### Anything Else? A second possible issue on branch-4.2, also not verified: the branch-4.2 FE may send an Iceberg TIME or `timestamptz_ns` field as `TPrimitiveType.UNSUPPORTED`. `thrift_to_type` (`be/src/core/data_type/primitive_type.cpp`) has no case for it, and its `default:` is `CHECK(false) << ", meet unknown type "`. If such a field reaches the BE as an equality-delete key type, the BE would abort while building the key. Master instead sends these types as a STRING placeholder; #69711 also handles that placeholder for TIME keys and their defaults. ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
