[
https://issues.apache.org/jira/browse/CALCITE-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16753370#comment-16753370
]
Volodymyr Vysotskyi commented on CALCITE-2223:
----------------------------------------------
These failures aren't expected. Do these tests pass when running separately? If
yes, it is possible that there are not enough resources, so please try to
increase the heap size:
{{export MAVEN_OPTS="-Xms2G -Xmx2G"}}
before executing {{mvn clean install}}. If it does not help, you may try to run
{{mvn clean install -DforkCount=1}}.
If this is env issue, please check {{etc/hosts}} that the hostname of the
machine with its correct IP are specified and associated. I'm not a MacOS user,
so I can't help here significantly.
Also, I have run tests using your branch, and the next test failed:
{noformat}
[ERROR] Failures:
[ERROR]
TestLimitWithExchanges.testPushLimitPastUnionExchange:56->testLimitHelper:142
Did not find expected pattern in plan: (?s)Limit\(offset=\[2\],
fetch=\[1\].*UnionExchange.*Limit\(fetch=\[3\]\).*Scan
00-00 Screen
00-01 Project(**=[$0])
00-02 SelectionVectorRemover
00-03 Limit(offset=[2], fetch=[1])
00-04 UnionExchange
01-01 Scan(table=[[dfs, multilevel/json]],
groupscan=[EasyGroupScan
[selectionRoot=file:/home/user515050/work/review/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.limit.TestLimitWithExchanges/root/multilevel/json,
numFiles=12, columns=[`**`],
files=[file:/home/user515050/work/review/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.limit.TestLimitWithExchanges/root/multilevel/json/1995/Q3/orders_95_q3.json,
file:/home/user515050/work/review/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.limit.TestLimitWithExchanges/root/multilevel/json/1995/Q1/orders_95_q1.json,
file:/home/user515050/work/review/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.limit.TestLimitWithExchanges/root/multilevel/json/1995/Q4/orders_95_q4.json,
file:/home/user515050/work/review/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.limit.TestLimitWithExchanges/root/multilevel/json/1995/Q2/orders_95_q2.json,
file:/home/user515050/work/review/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.limit.TestLimitWithExchanges/root/multilevel/json/1996/Q3/orders_96_q3.json,
file:/home/user515050/work/review/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.limit.TestLimitWithExchanges/root/multilevel/json/1996/Q1/orders_96_q1.json,
file:/home/user515050/work/review/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.limit.TestLimitWithExchanges/root/multilevel/json/1996/Q4/orders_96_q4.json,
file:/home/user515050/work/review/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.limit.TestLimitWithExchanges/root/multilevel/json/1996/Q2/orders_96_q2.json,
file:/home/user515050/work/review/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.limit.TestLimitWithExchanges/root/multilevel/json/1994/Q3/orders_94_q3.json,
file:/home/user515050/work/review/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.limit.TestLimitWithExchanges/root/multilevel/json/1994/Q1/orders_94_q1.json,
file:/home/user515050/work/review/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.limit.TestLimitWithExchanges/root/multilevel/json/1994/Q4/orders_94_q4.json,
file:/home/user515050/work/review/drill/exec/java-exec/target/org.apache.drill.exec.physical.impl.limit.TestLimitWithExchanges/root/multilevel/json/1994/Q2/orders_94_q2.json]]])
[INFO]
[ERROR] Tests run: 3739, Failures: 1, Errors: 0, Skipped: 153
{noformat}
With this change, {{LimitExchangeTransposeRule}} cannot be applied because of
ignoring {{UnionExchangePrel}} rel node as a part of the cycle.
Drill has a lot of interesting cases, in particular, tests with category
{{PlannerTest}}, but as it is expected, these tests mostly check the
correctness of drill-specific rules and use cases.
> ProjectMergeRule is infinitely matched when is applied after
> ProjectReduceExpressionsRule
> -----------------------------------------------------------------------------------------
>
> Key: CALCITE-2223
> URL: https://issues.apache.org/jira/browse/CALCITE-2223
> Project: Calcite
> Issue Type: Bug
> Reporter: Volodymyr Vysotskyi
> Assignee: Julian Hyde
> Priority: Critical
> Attachments: heap_overview.png, provenance_contents.png
>
>
> For queries like this:
> {code:sql}
> select t1.f from (select cast(f as int) f, f from (select cast(f as int) f
> from (values('1')) t(f))) as t1
> {code}
> OOM is thrown when {{ProjectMergeRule}} is applied before applying
> {{ProjectReduceExpressionsRule}} in VolcanoPlanner.
> A simple test to reproduce this issue (in {{RelOptRulesTest}}):
> {code:java}
> @Test public void testOomProjectMergeRule() {
> RelBuilder relBuilder =
> RelBuilder.create(RelBuilderTest.config().build());
> RelNode relNode = relBuilder
> .values(new String[]{"f"}, "1")
> .project(
> relBuilder.alias(
> relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER),
> "f"))
> .project(
> relBuilder.alias(
> relBuilder.cast(relBuilder.field(0), SqlTypeName.INTEGER),
> "f0"),
> relBuilder.alias(relBuilder.field(0), "f"))
> .project(
> relBuilder.alias(relBuilder.field(0), "f"))
> .build();
> RelOptPlanner planner = relNode.getCluster().getPlanner();
> RuleSet ruleSet =
> RuleSets.ofList(
> ReduceExpressionsRule.PROJECT_INSTANCE,
> new ProjectMergeRuleWithLongerName(),
> EnumerableRules.ENUMERABLE_PROJECT_RULE,
> EnumerableRules.ENUMERABLE_VALUES_RULE);
> Program program = Programs.of(ruleSet);
> RelTraitSet toTraits =
> relNode.getCluster().traitSet()
> .replace(0, EnumerableConvention.INSTANCE);
> RelNode output = program.run(planner, relNode, toTraits,
> ImmutableList.<RelOptMaterialization>of(),
> ImmutableList.<RelOptLattice>of());
> // check for output
> }
> /**
> * ProjectMergeRule inheritor which has
> * class name greater than ProjectReduceExpressionsRule class name
> (String.compareTo()).
> *
> * It is needed for RuleQueue.popMatch() method
> * to apply this rule before ProjectReduceExpressionsRule.
> */
> private static class ProjectMergeRuleWithLongerName extends
> ProjectMergeRule {
> public ProjectMergeRuleWithLongerName() {
> super(true, RelFactories.LOGICAL_BUILDER);
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
