This is an automated email from the ASF dual-hosted git repository.
englefly pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 955f401908f [fix](rbo) Fix ExprId renaming during constant propagation
(#69784)
955f401908f is described below
commit 955f401908f9bc25180753e2ac174c7958d4679c
Author: minghong <[email protected]>
AuthorDate: Thu Nov 19 16:01:42 2026 +0800
[fix](rbo) Fix ExprId renaming during constant propagation (#69784)
### What problem does this PR solve?
Issue Number: None
Related PR: #69111
Problem Summary:
Constant propagation materializes a constant above the aggregate or the
project which defines a
column. The materialized item is a literal, which is always
not-nullable, while the column it
replaces may be nullable. The item used to keep the exprId of the column
in every case, so the
rewritten plan could contain two definitions of the same exprId with
different nullability: the
not-nullable literal above, and the nullable column of the subtree
below.
The plan stays valid while a node still separates the two definitions,
but a later rule may remove
that node. In the reported queries `EliminateGroupByKeyByUniform`
replaces the uniform DISTINCT
aggregate by `limit + project`, and the not-nullable literal is then
attached directly above the
nullable set operation. `AdjustNullable` has to widen the column on the
way down, and with
`fe_debug` enabled the check aborts the query with
`AdjustNullable convert slot ... from not-nullable to nullable`.
Reproduction: with #69111 reverted, both reported queries trip the check
again; the reduced form of
the second one is
```sql
select * from ((
select 'vZRgkfajsD', 'cKToNmsxok'
except (select col_char_25__undef_signed_not_null,
col_char_40__undef_signed from by54)
union distinct
select col_varchar_25__undef_signed, col_char_25__undef_signed from by53
where col_tinyint_undef_signed_not_null = 4.4929476592789594
intersect distinct
select col_varchar_100__undef_signed, col_char_40__undef_signed from by53
where col_bigint_undef_signed_not_null < 54)
except (select col_char_25__undef_signed,
col_varchar_100__undef_signed_not_null from by5)
) as t where exists (select 1);
```
The fix keeps the exprId only when
* the materialized constant does not change the nullability of the
column, or
* the child does not produce the column any more, then the project item
is the only definition of the exprId.
When the child still produces the column and the constant has another
nullability, the constant gets
a fresh exprId and the references above are refreshed, so one exprId
keeps one definition. Not
renaming the columns whose nullability does not change keeps the
identity of the columns the
materialized view matching relies on, which #69111 had to trade for a
refresh of the whole plan
above.
### Release note
None
### Check List (For Author)
- Test: Unit Test / Regression test
- Unit: `ConstantPropagationTest`, `ConstantPropagationFreshExprIdTest`
(the reported queries and
the reduced form are rewritten with `fe_debug` enabled, and the
rewritten plan is checked to
have one nullability per exprId)
- Regression: the mv suites under `nereids_rules_p0/mv`
(`agg_optimize_when_uniform`,
`aggregate_with_roll_up`, `aggregate_without_roll_up`, `availability`,
`mv_ssb_test`,
`pre_write_strategy`)
- Behavior changed: No
- Does this need documentation: No
---
.../nereids/rules/rewrite/ConstantPropagation.java | 53 +++++--
.../ConstantPropagationFreshExprIdTest.java | 171 +++++++++++++++++----
.../rules/rewrite/ConstantPropagationTest.java | 11 +-
3 files changed, 181 insertions(+), 54 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/ConstantPropagation.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/ConstantPropagation.java
index c1531addfa1..21910bdbb2e 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/ConstantPropagation.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/ConstantPropagation.java
@@ -235,6 +235,14 @@ public class ConstantPropagation extends
DefaultPlanRewriter<CascadesContext> im
ImmutableList.Builder<NamedExpression> projectBuilder
= ImmutableList.builderWithExpectedSize(oldOutputExprs.size());
boolean containsConstantOutput = false;
+ // the columns the child still produces: when the materialized
constant has another nullability than the
+ // aggregate output, keeping the exprId would leave two definitions of
the same exprId in the plan
+ Set<ExprId> childOutputExprIds = aggregate.child().getOutput().stream()
+ .map(Slot::getExprId).collect(Collectors.toSet());
+ // the fresh exprIds of the constants materialized by the project
above the aggregate. They are only
+ // published when that project is kept: the aggregate may be returned
unchanged below, and in that case
+ // nothing defines the fresh exprId any more while the plan above
would already consume it.
+ Map<ExprId, ExprId> pendingRemaps = Maps.newHashMap();
// after normal agg, group by expressions and output expressions are
slots,
// after this rule, they may rewrite to literal, since literal are not
slot,
@@ -250,13 +258,22 @@ public class ConstantPropagation extends
DefaultPlanRewriter<CascadesContext> im
}
if (replacedExpr.isConstant() && !replacedExpr.equals(expr)) {
- // The constant is materialized above the aggregate, but it
keeps the exprId of the aggregate
- // output instead of getting a fresh one and refreshing the
plan above: when the aggregate still
- // outputs that column, the value is the same constant, and
when it removed the column, this
- // project item is the only definition of the exprId. Renaming
the exprId would break the
- // consumers which rely on the identity of the column, e.g.
the materialized view matching, and
- // would make the plan above be rewritten for nothing.
- projectBuilder.add(new Alias(expr.getExprId(), replacedExpr,
expr.getName()));
+ // The constant is materialized above the aggregate. It keeps
the exprId of the aggregate output
+ // when the nullability of the column does not change, or when
the child does not produce the
+ // column any more (then this project item is the only
definition of the exprId): the consumers
+ // which rely on the identity of the column, e.g. the
materialized view matching, keep working.
+ // When the child still produces the column and the constant
has another nullability, the exprId
+ // has to be renamed and the references above have to be
refreshed: otherwise the plan keeps two
+ // definitions of the same exprId with different nullability,
and AdjustNullable widens the
+ // column on the way down.
+ ExprId outputExprId = expr.getExprId();
+ if (childOutputExprIds.contains(outputExprId) &&
replacedExpr.nullable() != expr.nullable()) {
+ ExprId freshExprId =
pendingRemaps.computeIfAbsent(outputExprId,
+ id -> StatementScopeIdGenerator.newExprId());
+ projectBuilder.add(new Alias(freshExprId, replacedExpr,
expr.getName()));
+ } else {
+ projectBuilder.add(new Alias(outputExprId, replacedExpr,
expr.getName()));
+ }
containsConstantOutput = true;
} else {
Preconditions.checkArgument(newOutputExpr instanceof
NamedExpression, newOutputExpr);
@@ -265,9 +282,12 @@ public class ConstantPropagation extends
DefaultPlanRewriter<CascadesContext> im
}
if (newGroupByExprs.equals(oldGroupByExprs) &&
newOutputExprs.equals(oldOutputExprs)) {
- // the aggregate is not changed, the plan above keeps referencing
the output slots of the aggregate
+ // the aggregate is not changed, the plan above keeps referencing
the output slots of the aggregate,
+ // and the project above the aggregate is dropped together with
the fresh exprIds it would define
return aggregate;
}
+ // the project is kept, so the plan above has to follow the fresh
exprIds it defines
+ replacedExprIds.putAll(pendingRemaps);
aggregate = aggregate.withGroupByAndOutput(newGroupByExprs,
newOutputExprs);
if (containsConstantOutput) {
@@ -426,16 +446,19 @@ public class ConstantPropagation extends
DefaultPlanRewriter<CascadesContext> im
// Alias always yields an Alias again, so only its expression may
change
Expression newExpr = replaceConstants(expr, false, context, equalSet,
constants);
if (!(newExpr instanceof Alias) || newExpr.equals(expr)
- || !childOutputExprIds.contains(expr.getExprId())) {
- // unchanged, or the child does not produce the column any more,
then this project item is the only
- // definition of the exprId and it is kept
+ || !childOutputExprIds.contains(expr.getExprId())
+ || newExpr.nullable() == expr.nullable()) {
+ // unchanged; or the child does not produce the column any more,
then this project item is the only
+ // definition of the exprId; or the nullability of the column does
not change, then the exprId is kept
return (NamedExpression) newExpr;
}
- // the child still produces this column, so the project item would
shadow it: materialize the constant with
- // a fresh exprId and refresh the references above, see replacedExprIds
+ // the child still produces this column and the materialized constant
has another nullability than the
+ // column: keeping the exprId would leave two definitions of the same
exprId with different nullability in
+ // the plan, so materialize the constant with a fresh exprId and
refresh the references above, see
+ // replacedExprIds
Alias replacedAlias = (Alias) newExpr;
- ExprId freshExprId = StatementScopeIdGenerator.newExprId();
- replacedExprIds.put(expr.getExprId(), freshExprId);
+ ExprId freshExprId = replacedExprIds.computeIfAbsent(expr.getExprId(),
+ id -> StatementScopeIdGenerator.newExprId());
return new Alias(freshExprId, replacedAlias.child(0),
replacedAlias.getName());
}
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/ConstantPropagationFreshExprIdTest.java
b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/ConstantPropagationFreshExprIdTest.java
index 674723d15a5..cb72da612af 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/ConstantPropagationFreshExprIdTest.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/ConstantPropagationFreshExprIdTest.java
@@ -34,7 +34,9 @@ import org.apache.doris.utframe.TestWithFeService;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
+import java.util.HashMap;
import java.util.List;
+import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
@@ -66,6 +68,23 @@ public class ConstantPropagationFreshExprIdTest extends
TestWithFeService {
+ " select col_datetime_3__undef_signed_not_null,
col_date_undef_signed_not_null"
+ " from
table_100_undef_partitions2_keys3_properties4_distributed_by54 ;";
+ /** the reported query reduced to the shape which still trips the nullable
check. */
+ private static final String SQL_11000259585_SIMPLIFIED = "select * from (("
+ + " select 'vZRgkfajsD', 'cKToNmsxok'"
+ + " except (select col_char_25__undef_signed_not_null,
col_char_40__undef_signed"
+ + " from
table_100_undef_partitions2_keys3_properties4_distributed_by54)"
+ + " union distinct"
+ + " select col_varchar_25__undef_signed, col_char_25__undef_signed"
+ + " from
table_100_undef_partitions2_keys3_properties4_distributed_by53"
+ + " where col_tinyint_undef_signed_not_null = 4.4929476592789594"
+ + " intersect distinct"
+ + " select col_varchar_100__undef_signed,
col_char_40__undef_signed"
+ + " from
table_100_undef_partitions2_keys3_properties4_distributed_by53"
+ + " where col_bigint_undef_signed_not_null < 54)"
+ + " except (select col_char_25__undef_signed,
col_varchar_100__undef_signed_not_null"
+ + " from
table_100_undef_partitions2_keys3_properties4_distributed_by5)"
+ + " ) as t where exists (select 1);";
+
private static final String SQL_11000259585 = "select * from ((select
'vZRgkfajsD', 'cKToNmsxok' except (select col_char_25__undef_signed_not_null,
col_char_40__undef_signed from
table_100_undef_partitions2_keys3_properties4_distributed_by54 where
col_date_undef_signed = '2016-09-20' order by 1, 2 desc limit 1, 8) union
distinct select col_varchar_25__undef_signed, col_char_25__undef_signed from
table_100_undef_partitions2_keys3_properties4_distributed_by53 where
col_tinyint_undef_si [...]
private static final String SQL_MATERIALIZED_CTE = "WITH c AS (SELECT k,
v, count(*) AS cnt"
@@ -85,7 +104,7 @@ public class ConstantPropagationFreshExprIdTest extends
TestWithFeService {
@Override
protected void runBeforeAll() throws Exception {
createDatabase("test");
- createTable("create table test.constant_propagation_fresh_expr_id_t(k
int null, v int null)"
+ createTable("create table test.constant_propagation_fresh_expr_id_t(k
int null, v int null, n int not null)"
+ " distributed by hash(k) properties('replication_num' =
'1');");
createTable("create table
test.table_100_undef_partitions2_keys3_properties4_distributed_by52 ("
+ " col_datetime_3__undef_signed_not_null datetime(3) not
null,"
@@ -138,7 +157,7 @@ public class ConstantPropagationFreshExprIdTest extends
TestWithFeService {
}
@Test
- void testConstantAggregateProjectKeepsTheExprId() {
+ void testConstantAggregateProjectRenamesNullableKeys() {
// run this rule alone, so the project it materializes above the
aggregate is still part of the plan
PlanChecker checker = PlanChecker.from(connectContext)
.analyze("select k, v from
constant_propagation_fresh_expr_id_t where k = 5 and v = 7 group by k, v")
@@ -147,41 +166,126 @@ public class ConstantPropagationFreshExprIdTest extends
TestWithFeService {
LogicalProject<?> constantProject = findConstantProject(plan);
Assertions.assertNotNull(constantProject, plan.treeString());
List<NamedExpression> items = constantProject.getProjects();
- // the kept grouping key is still materialized as a constant above the
aggregate, with the exprId of the
- // aggregate output: no fresh exprId and no refresh of the plan above
- NamedExpression keptKey = items.get(0);
- Assertions.assertTrue(keptKey instanceof Alias, plan.treeString());
- Assertions.assertEquals(new IntegerLiteral(5), ((Alias)
keptKey).child());
+ // the columns are nullable and the materialized constants are not, so
both items get a fresh exprId and
+ // the plan above follows it: no exprId is left with two definitions
of different nullability
+ for (NamedExpression item : items) {
+ Assertions.assertTrue(item instanceof Alias, plan.treeString());
+ Assertions.assertFalse(constantProject.child().getOutput().stream()
+ .anyMatch(slot ->
slot.getExprId().equals(item.getExprId())),
+ "a nullable column materialized as a not-nullable constant
must get a fresh exprId:\n"
+ + plan.treeString());
+ }
+ Assertions.assertEquals(new IntegerLiteral(5), ((Alias)
items.get(0)).child());
+ Assertions.assertEquals(new IntegerLiteral(7), ((Alias)
items.get(1)).child());
+
Assertions.assertTrue(plan.getOutput().stream().map(Slot::getExprId).collect(Collectors.toSet())
+
.containsAll(items.stream().map(NamedExpression::getExprId).collect(Collectors.toSet())),
+ "the plan above does not reference the materialized
constants:\n" + plan.treeString());
+ }
+
+ @Test
+ void testConstantAggregateProjectKeepsNotNullableKeyExprId() {
+ // a not-nullable column is materialized with the exprId of the
aggregate output: the nullability does not
+ // change, so the consumers which rely on the identity of the column
(e.g. the mv matching) keep working
+ PlanChecker checker = PlanChecker.from(connectContext)
+ .analyze("select n, v from
constant_propagation_fresh_expr_id_t"
+ + " where n = 5 and v = 7 group by n, v")
+ .applyCustom(new ConstantPropagation());
+ Plan plan = checker.getPlan();
+ LogicalProject<?> constantProject = findConstantProject(plan);
+ Assertions.assertNotNull(constantProject, plan.treeString());
+ List<NamedExpression> items = constantProject.getProjects();
+ // the not-nullable column keeps the exprId of the aggregate output,
the nullable one gets a fresh exprId
+ Alias notNullableItem = (Alias) items.stream()
+ .filter(item ->
item.getName().equals("n")).findFirst().orElseThrow(AssertionError::new);
+ Assertions.assertEquals(new IntegerLiteral(5),
notNullableItem.child());
Assertions.assertTrue(constantProject.child().getOutput().stream()
- .anyMatch(slot ->
slot.getExprId().equals(keptKey.getExprId())),
- "the kept grouping key keeps the exprId of the aggregate
output:\n" + plan.treeString());
- // the removed grouping key is materialized with the exprId it already
had as well
- NamedExpression removedKey = items.get(1);
- Assertions.assertTrue(removedKey instanceof Alias, plan.treeString());
- Assertions.assertEquals(new IntegerLiteral(7), ((Alias)
removedKey).child());
- // the plan above references the outputs of the project, unchanged
- Set<ExprId> projectOutputIds = constantProject.getOutput().stream()
- .map(Slot::getExprId).collect(Collectors.toSet());
- Set<ExprId> rootOutputIds = plan.getOutput().stream()
- .map(Slot::getExprId).collect(Collectors.toSet());
- Assertions.assertTrue(rootOutputIds.containsAll(projectOutputIds),
- "the plan above does not reference the project outputs:\n" +
plan.treeString());
+ .anyMatch(slot ->
slot.getExprId().equals(notNullableItem.getExprId())),
+ "a not-nullable column keeps the exprId of the aggregate
output:\n" + plan.treeString());
+ Alias nullableItem = (Alias) items.stream()
+ .filter(item ->
item.getName().equals("v")).findFirst().orElseThrow(AssertionError::new);
+ Assertions.assertEquals(new IntegerLiteral(7), nullableItem.child());
+ Assertions.assertFalse(constantProject.child().getOutput().stream()
+ .anyMatch(slot ->
slot.getExprId().equals(nullableItem.getExprId())),
+ "a nullable column materialized as a not-nullable constant
gets a fresh exprId:\n"
+ + plan.treeString());
}
@Test
- void testRqgCase11000258898() {
- // the RQG case this PR was reported from: on the reported base it
tripped "AdjustNullable convert slot
- // 2004-04-28#xx from not-nullable to nullable" during the rewrite, so
the rewrite and the planning of the
- // query have to succeed
- PlanChecker.from(connectContext).analyze(SQL_11000258898).rewrite();
- PlanChecker.from(connectContext).checkPlannerResult(SQL_11000258898);
+ void testSoleNullableGroupKeyKeepsTheAggregateUnchanged() {
+ // select k, count(*) from t where k = 5 group by k: the aggregate
keeps its only group key, so it is
+ // returned unchanged and the fresh exprId recorded for the nullable
key must not be published to the plan
+ // above, otherwise nothing defines that exprId any more
+ Plan plan = PlanChecker.from(connectContext)
+ .analyze("select k, count(*) from
constant_propagation_fresh_expr_id_t"
+ + " where k = 5 group by k")
+ .applyCustom(new ConstantPropagation()).getPlan();
+ assertReferencedSlotsProducedByChildren(plan);
+ // the full rewrite has to stay plannable as well
+ PlanChecker.from(connectContext)
+ .analyze("select k, count(*) from
constant_propagation_fresh_expr_id_t"
+ + " where k = 5 group by k")
+ .rewrite();
+ }
+
+ /** Every slot a node references has to be an output slot of one of its
children. */
+ private static void assertReferencedSlotsProducedByChildren(Plan plan) {
+ plan.foreach(node -> {
+ Plan planNode = (Plan) node;
+ Set<ExprId> childOutputIds = new java.util.HashSet<>();
+ planNode.children().forEach(child -> child.getOutput()
+ .forEach(slot -> childOutputIds.add(slot.getExprId())));
+ for (org.apache.doris.nereids.trees.expressions.Expression
expression : planNode.getExpressions()) {
+
java.util.Set<org.apache.doris.nereids.trees.expressions.Expression> slots
+ = expression.collectToSet(e -> e instanceof Slot);
+ for (org.apache.doris.nereids.trees.expressions.Expression
slot : slots) {
+ ExprId exprId = ((Slot) slot).getExprId();
+ Assertions.assertTrue(childOutputIds.contains(exprId)
+ || planNode.getOutput().stream()
+ .anyMatch(output ->
output.getExprId().equals(exprId)),
+ "slot " + exprId + " is referenced but not
produced: " + planNode.treeString());
+ }
+ }
+ return false;
+ });
}
@Test
- void testRqgCase11000259585() {
- // the second case of the same RQG report, same failure mode: the
rewrite and the planning have to succeed
- PlanChecker.from(connectContext).analyze(SQL_11000259585).rewrite();
- PlanChecker.from(connectContext).checkPlannerResult(SQL_11000259585);
+ void testRqgCasesDoNotConvertNullableSlots() {
+ // the rewrite of the reported queries used to trip
+ // "AdjustNullable convert slot ... from not-nullable to nullable"
once the check is enabled: one exprId
+ // must not have two definitions with different nullability, so
rewrite them with the check enabled
+ boolean feDebug = connectContext.getSessionVariable().isFeDebug();
+ connectContext.getSessionVariable().feDebug = true;
+ try {
+ Plan reduced = PlanChecker.from(connectContext)
+ .analyze(SQL_11000259585_SIMPLIFIED).rewrite().getPlan();
+ assertSingleNullabilityPerExprId(reduced);
+
PlanChecker.from(connectContext).checkPlannerResult(SQL_11000259585_SIMPLIFIED);
+ Plan reported =
PlanChecker.from(connectContext).analyze(SQL_11000258898).rewrite().getPlan();
+ assertSingleNullabilityPerExprId(reported);
+
PlanChecker.from(connectContext).checkPlannerResult(SQL_11000258898);
+ Plan full =
PlanChecker.from(connectContext).analyze(SQL_11000259585).rewrite().getPlan();
+ assertSingleNullabilityPerExprId(full);
+
PlanChecker.from(connectContext).checkPlannerResult(SQL_11000259585);
+ } finally {
+ connectContext.getSessionVariable().feDebug = feDebug;
+ }
+ }
+
+ /** One exprId must have one nullability in the rewritten plan, otherwise
AdjustNullable widens it. */
+ private static void assertSingleNullabilityPerExprId(Plan plan) {
+ Map<ExprId, Boolean> nullabilityByExprId = new HashMap<>();
+ plan.foreach(node -> {
+ Plan planNode = (Plan) node;
+ for (Slot slot : planNode.getOutput()) {
+ Boolean previous =
nullabilityByExprId.putIfAbsent(slot.getExprId(), slot.nullable());
+ if (previous != null) {
+ Assertions.assertEquals(previous, slot.nullable(),
+ "exprId " + slot.getExprId() + " has two
nullabilities in:\n" + plan.treeString());
+ }
+ }
+ return false;
+ });
}
@Test
@@ -192,13 +296,12 @@ public class ConstantPropagationFreshExprIdTest extends
TestWithFeService {
PlanChecker.from(connectContext).analyze(SQL_MATERIALIZED_CTE).getPlan());
Plan plan =
PlanChecker.from(connectContext).analyze(SQL_MATERIALIZED_CTE).rewrite().getPlan();
assertCteConsumerMapsStayValid(plan);
- // constant propagation keeps the exprIds of the producer outputs, so
they are the same as in the analyzed
- // plan and the consumer maps still resolve; a rule which renames them
has to refresh the maps, which the
- // assertCteConsumerMapsStayValid above checks
+ // a producer output whose column is nullable is renamed when it is
materialized as a constant, and the
+ // consumer maps have to be refreshed with it, which
assertCteConsumerMapsStayValid above checks
Set<ExprId> rewrittenProducerOutputIds = cteProducerOutputIds(plan);
Assertions.assertEquals(analyzedProducerOutputIds.size(),
rewrittenProducerOutputIds.size(),
plan.treeString());
- Assertions.assertEquals(analyzedProducerOutputIds,
rewrittenProducerOutputIds, plan.treeString());
+ assertSingleNullabilityPerExprId(plan);
// the query has to stay plannable
PlanChecker.from(connectContext).checkPlannerResult(SQL_MATERIALIZED_CTE);
}
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/ConstantPropagationTest.java
b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/ConstantPropagationTest.java
index eea5de6aa85..2c328e47cac 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/ConstantPropagationTest.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/ConstantPropagationTest.java
@@ -453,9 +453,9 @@ class ConstantPropagationTest {
}
@Test
- public void testProjectItemShadowingAChildSlotGetsFreshExprId() {
- // a project item which re-defines a column which its child still
outputs describes another value, so it
- // must be materialized with a fresh exprId, and the sink above has to
follow that exprId
+ public void
testProjectItemShadowingAChildSlotWithSameNullabilityKeepsTheExprId() {
+ // a project item which re-defines a column which its child still
outputs and which does not change the
+ // nullability of the column keeps the exprId, so the sink above keeps
referencing it
LogicalFilter<LogicalOlapScan> filter = new
LogicalFilter<>(ImmutableSet.of(
new EqualTo(studentId, new IntegerLiteral(5))), student);
LogicalProject<LogicalFilter<LogicalOlapScan>> project = new
LogicalProject<>(ImmutableList.of(
@@ -466,10 +466,11 @@ class ConstantPropagationTest {
= new LogicalBlackholeSink<>(sinkOutputExprs, project);
LogicalSink<?> rewrittenSink = (LogicalSink<?>)
executor.rewriteRoot(sink, jobContext);
- // the child of the project still outputs studentId, so the constant
item got a fresh exprId
+ // the constant does not change the nullability of the column, so the
exprId is kept and the consumers
+ // which rely on the identity of the column keep working
NamedExpression constantItem = ((LogicalProject<?>)
rewrittenSink.child()).getProjects().get(0);
Assertions.assertEquals(new IntegerLiteral(5), ((Alias)
constantItem).child());
- Assertions.assertNotEquals(studentId.getExprId(),
constantItem.getExprId());
+ Assertions.assertEquals(studentId.getExprId(),
constantItem.getExprId());
// and the sink follows it, otherwise it would reference a slot which
its child no longer outputs
Assertions.assertEquals(constantItem.getExprId(),
rewrittenSink.getOutputExprs().get(0).getExprId());
List<ExprId> childOutputIds =
rewrittenSink.child().getOutput().stream()
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]