englefly opened a new pull request, #69328:
URL: https://github.com/apache/doris/pull/69328
### What problem does this PR solve?
Issue Number: DORIS-29733, DORIS-29734
Related PR: None
Problem Summary:
A STRUCT -> STRUCT cast converts positionally and its result carries the
target field names, but
the FE treated a cast that only renames fields as a no-op and dropped the
CAST node when
serializing expressions to thrift: `CastExpr.noOp` comes from
`Type.matchExactType`, whose struct
branch compared the fields positionally and ignored their names. The
folded-away cast also
overwrote the child expression's type with the target type, so the FE
bookkeeping and the physical
column disagreed: the value kept the source field names, while the
ELEMENT_AT binding and the
nested column pruning of the cast target were planned against the target
names.
Reproduction with the default session variables on a single FE/BE:
```sql
CREATE TABLE t (id INT, s STRUCT<a:STRING,b:STRING>) ENGINE = OLAP
DUPLICATE KEY(id) DISTRIBUTED BY HASH(id) BUCKETS 1 PROPERTIES
("replication_num" = "1");
INSERT INTO t VALUES (1, named_struct('a', 'left', 'b', 'right'));
SELECT element_at(CAST(s AS STRUCT<b:STRING,a:STRING>), 'b') FROM t;
```
Before the fix:
* with nested column pruning on (the default) the pruned slot kept only
source field `a`, and the
query failed with
`errCode = 2, detailMessage = ... [RUNTIME_ERROR]Element not found for
function element_at: name b not found in Struct(a:Nullable(String))`;
* with nested column pruning off the query silently returned `right` (the
source field `b`)
instead of `left`;
* reading both target fields returned (`right`, `left`) instead of (`left`,
`right`), while the
whole value CAST correctly returned `{b:'left', a:'right'}`.
`EXPLAIN VERBOSE` is correct in every case because the Nereids plan keeps
the cast; only the thrift
sent to BE lost it, which is what makes this easy to misdiagnose as a
pruning bug.
Root cause in one line: `noOp == true` meant "no CAST_EXPR node", and BE's
`element_at` resolves the
field name against the type attached to the argument column in the Block
(`try_get_position_by_name`
on the struct type of the block position). With the cast gone, that type was
the scan type, i.e.
the source field names. BE has an explicit hook for this FE/BE type
divergence
(`VExpr::execute_type`, "for slot-ref expressions the type of the returned
Column may not match
data_type"), which is exactly the gap the elided cast fell into.
Fix:
A struct field name is part of the value identity, so `Type.matchExactType`
now also requires the
field names to match (comparing the already normalized lower case name,
consistent with BE's case
insensitive struct field lookup). A rename-only cast is serialized instead
of being skipped, and BE
relabels the value with the target type. Since the positional field types
are unchanged, BE keeps
the identity cast wrapper (`prepare_impl` compares with
`DataTypeStruct::equals`, which ignores
names), so no data is copied. Renames nested in ARRAY/MAP are covered by the
same recursion, and
`try_cast` inherits the fixed no-op decision.
After the fix both toggle states return the positional result, b=`left` and
a=`right`, matching the
whole value CAST, and per-field pruning is preserved: the scan still reads
only the source field
behind the accessed target field (`all access paths: [s.a]` / `[s.b]`).
### Release note
Fixed a wrong result and an "Element not found for function element_at"
error when a
STRUCT -> STRUCT cast renames fields and its result is accessed by field
name.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [x] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [ ] No.
- [x] Yes. A cast that only renames STRUCT fields is no longer folded
away, so its result
carries the target field names as the type system already promised.
Casts whose field names
already match keep the existing no-op behavior.
- Does this need documentation?
- [x] No.
Test detail:
* Unit test: `mvn test -pl fe-core -am
-Dtest='CastExprHashPartitionTest,CastMonotonicityTest,CastTest,CheckCastTest,SimplifyCastRuleTest,TryCastTest,TypeTest,ExprToThriftBehaviorTest'`
passes.
`ExprToThriftBehaviorTest#testStructRenameCastIsNotNoOp` asserts `isNoOp()
== false` and that the
serialized tree is `CAST_EXPR(type=struct<b,a>)` +
`SLOT_REF(type=struct<a,b>)`;
`#testStructRenameNestedInArrayCastIsNotNoOp` covers the recursion;
`#testSameStructTypeCastIsStillNoOp` pins that the no-op optimization is
kept;
`TypeTest#testStructWithNestedFieldsExactMatch` covers the match rule
itself.
* Regression test: `./run-regression-test.sh --run -d
nereids_rules_p0/column_pruning` — 16 suites
pass, including the new `struct_cast_rename_pruning`, which asserts the
values and
`all access paths: [s.a]` / `[s.b]` for pruning on and off.
* Manual test: the snippet above plus WHERE / GROUP BY / ORDER BY / subquery
/ CTE / `INSERT INTO` a
table whose struct column is named differently / ARRAY and MAP nested
renames, all with nested
column pruning both on and off, on a local FE built from this commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
