[
https://issues.apache.org/jira/browse/CALCITE-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16735127#comment-16735127
]
Vladimir Sitnikov commented on CALCITE-2223:
--------------------------------------------
{quote}Also, rule FilterSetOpTransposeRule is also affected by the same bug.
Will try to specify the test cases for this.
{quote}
[~vvysotskyi], have you had a chance to find a test case?
I've looked into MaterializationTest failure (infinite planning), and the case
there is as follows.
I think the issue is more materialization-induced rather than
FilterSetOpTransposeRule and/or UnionMergeRule.
{code:java}
@Test public void testJoinMaterializationX() {
checkMaterialize(
"select \"depts\".\"deptno\"\n"
+ "from \"depts\"\n"
+ "join \"dependents\" on (\"depts\".\"name\" =
\"dependents\".\"name\")\n"
+ "where \"depts\".\"deptno\" > 30",
"select \"depts\".\"deptno\"\n"
+ "from \"depts\"\n"
+ "join \"dependents\" on (\"depts\".\"name\" =
\"dependents\".\"name\")\n"
+ "where \"depts\".\"deptno\" > 10",
HR_FKUK_MODEL,
CalciteAssert.checkResultContains(
"EnumerableUnion(all=[true])"
));
}
{code}
Let's call the original join with X (the join without where).
Then materialization replaces X with
{code:sql}
(select * from X where deptno>10 and deptno<=30)
union all select * from m0
{code}
So far so good.
However, later it happens to apply materialization again:
{code:sql}
(select * from (
(select * from X where deptno>10 and deptno<=30)
union all select * from m0
) where deptno>10 and deptno<=30)
union all select * from m0
{code}
Then it pushes WHERE into UNION:
{code:sql}
(select * from (
(select * from X where deptno>10 and deptno<=30)
union all select * from m0 where deptno>10 and deptno<=30)
)
union all select * from m0
{code}
Then it merges unions:
{code:sql}
(select * from X where deptno>10 and deptno<=30)
union all select * from m0 where deptno>10 and deptno<=30)
union all select * from m0
{code}
The full plan after "merge union" looks like
{noformat}
LogicalUnion(all=[true])
LogicalFilter(subset=[rel#352:Subset#21.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#317:Subset#19.NONE.[]], condition=[>($0, 10)])
LogicalProject(subset=[rel#312:Subset#18.NONE.[]], deptno=[$0])
LogicalFilter(subset=[rel#310:Subset#17.NONE.[]], condition=[>=(30,
$0)])
LogicalJoin(subset=[rel#216:Subset#4.NONE.[]], condition=[=($1, $2)],
joinType=[inner])
LogicalProject(subset=[rel#211:Subset#1.NONE.[]], deptno=[$0],
name0=[CAST($1):VARCHAR CHARACTER SET "ISO-8859-1" COLLATE
"ISO-8859-1$en_US$primary"])
EnumerableTableScan(subset=[rel#209:Subset#0.ENUMERABLE.[]],
table=[[hr, depts]])
LogicalProject(subset=[rel#214:Subset#3.NONE.[]],
name0=[CAST($1):VARCHAR CHARACTER SET "ISO-8859-1" COLLATE
"ISO-8859-1$en_US$primary"])
EnumerableTableScan(subset=[rel#212:Subset#2.ENUMERABLE.[]],
table=[[hr, dependents]])
LogicalFilter(subset=[rel#353:Subset#22.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#318:Subset#20.NONE.[]], condition=[>($0, 10)])
EnumerableTableScan(subset=[rel#276:Subset#12.ENUMERABLE.[0]],
table=[[hr, m0]])
EnumerableTableScan(subset=[rel#276:Subset#12.ENUMERABLE.[0]], table=[[hr,
m0]])
{noformat}
Then the process continues, and it just keeps adding forever
{noformat}
union all select * from m0 where deptno>10 and deptno<=30){noformat}
{noformat}
LogicalUnion(all=[true])
LogicalFilter(subset=[rel#363:Subset#23.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#352:Subset#21.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#317:Subset#19.NONE.[]], condition=[>($0, 10)])
LogicalProject(subset=[rel#312:Subset#18.NONE.[]], deptno=[$0])
LogicalFilter(subset=[rel#310:Subset#17.NONE.[]], condition=[>=(30,
$0)])
LogicalJoin(subset=[rel#216:Subset#4.NONE.[]], condition=[=($1,
$2)], joinType=[inner])
LogicalProject(subset=[rel#211:Subset#1.NONE.[]], deptno=[$0],
name0=[CAST($1):VARCHAR CHARACTER SET "ISO-8859-1" COLLATE
"ISO-8859-1$en_US$primary"])
EnumerableTableScan(subset=[rel#209:Subset#0.ENUMERABLE.[]],
table=[[hr, depts]])
LogicalProject(subset=[rel#214:Subset#3.NONE.[]],
name0=[CAST($1):VARCHAR CHARACTER SET "ISO-8859-1" COLLATE
"ISO-8859-1$en_US$primary"])
EnumerableTableScan(subset=[rel#212:Subset#2.ENUMERABLE.[]],
table=[[hr, dependents]])
LogicalFilter(subset=[rel#364:Subset#24.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#353:Subset#22.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#318:Subset#20.NONE.[]], condition=[>($0, 10)])
EnumerableTableScan(subset=[rel#276:Subset#12.ENUMERABLE.[0]],
table=[[hr, m0]])
LogicalFilter(subset=[rel#365:Subset#25.NONE.[]], condition=[>=(30, $0)])
EnumerableTableScan(subset=[rel#276:Subset#12.ENUMERABLE.[0]], table=[[hr,
m0]])
EnumerableTableScan(subset=[rel#276:Subset#12.ENUMERABLE.[0]], table=[[hr,
m0]])
{noformat}
{noformat}
LogicalUnion(all=[true])
LogicalFilter(subset=[rel#409:Subset#27.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#363:Subset#23.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#352:Subset#21.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#317:Subset#19.NONE.[]], condition=[>($0, 10)])
LogicalProject(subset=[rel#312:Subset#18.NONE.[]], deptno=[$0])
LogicalFilter(subset=[rel#310:Subset#17.NONE.[]], condition=[>=(30,
$0)])
LogicalJoin(subset=[rel#216:Subset#4.NONE.[]], condition=[=($1,
$2)], joinType=[inner])
LogicalProject(subset=[rel#211:Subset#1.NONE.[]], deptno=[$0],
name0=[CAST($1):VARCHAR CHARACTER SET "ISO-8859-1" COLLATE
"ISO-8859-1$en_US$primary"])
EnumerableTableScan(subset=[rel#209:Subset#0.ENUMERABLE.[]],
table=[[hr, depts]])
LogicalProject(subset=[rel#214:Subset#3.NONE.[]],
name0=[CAST($1):VARCHAR CHARACTER SET "ISO-8859-1" COLLATE
"ISO-8859-1$en_US$primary"])
EnumerableTableScan(subset=[rel#212:Subset#2.ENUMERABLE.[]],
table=[[hr, dependents]])
LogicalFilter(subset=[rel#410:Subset#28.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#364:Subset#24.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#353:Subset#22.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#318:Subset#20.NONE.[]], condition=[>($0, 10)])
EnumerableTableScan(subset=[rel#276:Subset#12.ENUMERABLE.[0]],
table=[[hr, m0]])
LogicalFilter(subset=[rel#411:Subset#29.NONE.[]], condition=[>=(30, $0)])
LogicalFilter(subset=[rel#365:Subset#25.NONE.[]], condition=[>=(30, $0)])
EnumerableTableScan(subset=[rel#276:Subset#12.ENUMERABLE.[0]],
table=[[hr, m0]])
LogicalFilter(subset=[rel#365:Subset#25.NONE.[]], condition=[>=(30, $0)])
EnumerableTableScan(subset=[rel#276:Subset#12.ENUMERABLE.[0]], table=[[hr,
m0]])
EnumerableTableScan(subset=[rel#276:Subset#12.ENUMERABLE.[0]], table=[[hr,
m0]])
{noformat}
and so on.
> ProjectMergeRule is infinitely matched when is applied after
> ProjectReduceExpressionsRule
> -----------------------------------------------------------------------------------------
>
> Key: CALCITE-2223
> URL: https://issues.apache.org/jira/browse/CALCITE-2223
> Project: Calcite
> Issue Type: Bug
> Reporter: Volodymyr Vysotskyi
> Assignee: Julian Hyde
> Priority: Critical
> Attachments: heap_overview.png, provenance_contents.png
>
>
> For queries like this:
> {code:sql}
> select t1.f from (select cast(f as int) f, f from (select cast(f as int) f
> from (values('1')) t(f))) as t1
> {code}
> OOM is thrown when {{ProjectMergeRule}} is applied before applying
> {{ProjectReduceExpressionsRule}} in VolcanoPlanner.
> A simple test to reproduce this issue (in {{RelOptRulesTest}}):
> {code:java}
> @Test public void testOomProjectMergeRule() {
> RelBuilder relBuilder =
> RelBuilder.create(RelBuilderTest.config().build());
> RelNode relNode = relBuilder
> .values(new String[]{"f"}, "1")
> .project(
> relBuilder.alias(
> relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER),
> "f"))
> .project(
> relBuilder.alias(
> relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER),
> "f0"),
> relBuilder.alias(relBuilder.field(0), "f"))
> .project(
> relBuilder.alias(relBuilder.field(0), "f"))
> .build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> RuleSet ruleSet =
> RuleSets.ofList(
> ReduceExpressionsRule.PROJECT_INSTANCE,
> new ProjectMergeRuleWithLongerName(),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_VALUES_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> relNode.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.<RelOptMaterialization>of(),
> ImmutableList.<RelOptLattice>of());
> // check for output
> }
> /**
> * ProjectMergeRule inheritor which has
> * class name greater than ProjectReduceExpressionsRule class name
> (String.compareTo()).
> *
> * It is needed for RuleQueue.popMatch() method
> * to apply this rule before ProjectReduceExpressionsRule.
> */
> private static class ProjectMergeRuleWithLongerName extends
> ProjectMergeRule {
> public ProjectMergeRuleWithLongerName() {
> super(true, RelFactories.LOGICAL_BUILDER);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
