[
https://issues.apache.org/jira/browse/CALCITE-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steven Talbot updated CALCITE-3831:
-----------------------------------
Description:
Note that this only happens when a rollup is required ("forceRollup" in the
terminology of "rewriteView").
Here's a test in MaterializationTest that shows the behavior:
{code:java}
@Test public void testAggregateMaterializationOnProjectWithDuplicates() {
checkMaterialize(
"select \"empid\" / 2, \"empid\", floor(cast('1997-01-20 12:34:56' as
timestamp) to month), count(*) + 1 as c, sum(\"empid\") as s\n"
+ "from \"emps\" group by \"empid\" / 2, \"empid\", floor(cast('1997-01-20
12:34:56' as timestamp) to month)",
"select e1, e2, f, sum(e1) as s from (select \"empid\" as e1, \"empid\" as e2,
floor(cast('1997-01-20 12:34:56' as timestamp) to year) as f\n"
+ "from \"emps\" ) t1 " +
"group by e1,e2,f");
}{code}
The query SQL uses that somewhat artificial subquery to force the SQL to rel
parser to create the project with duplicates for the test.
Obviously this is a silly SQL query to write, but in the process of building
more complex queries it's not absurd to end up with a project with duplicate
RexNodes.
Here's the top of the stack produced by that test (sorry my line number locally
are a bit different from current master):
{noformat}
Caused by: org.apache.calcite.util.mapping.Mappings$NoElementException: source
#0 has no target in mapping [size=3, sourceCount=4, targetCount=23,
elements=[1:1, 2:5, 3:4]]Caused by:
org.apache.calcite.util.mapping.Mappings$NoElementException: source #0 has no
target in mapping [size=3, sourceCount=4, targetCount=23, elements=[1:1, 2:5,
3:4]] at
org.apache.calcite.util.mapping.Mappings$AbstractMapping.getTarget(Mappings.java:879)
at
org.apache.calcite.rel.rules.AbstractMaterializedViewRule$MaterializedViewAggregateRule.rewriteView(AbstractMaterializedViewRule.java:1502)
at
org.apache.calcite.rel.rules.AbstractMaterializedViewRule.perform(AbstractMaterializedViewRule.java:521){noformat}
I tried to make a quick fix for this, since on the surface the problem in the
code seems simple, at the site of the exception
([https://github.com/apache/calcite/blob/134430e481bb5495d0852434041428104e29874e/core/src/main/java/org/apache/calcite/rel/rules/AbstractMaterializedViewRule.java#L1513])
we do a lookup in a mapping. The mapping comes about from inverting a mapping
that maps projects in the view to matching projects in the query (the relevant
line is
[https://github.com/apache/calcite/blob/134430e481bb5495d0852434041428104e29874e/core/src/main/java/org/apache/calcite/rel/rules/AbstractMaterializedViewRule.java#L1436]),
and so if two query projects match the same view project, they will overwrite
each other, and the lookup on the inverse mapping will fail.
Trying the quick fix of setting the inverse mapping at the same time we set the
rewriting mapping (i.e. `inverseMapping.set(i, k);` for the above line, and a
couple others) fixed the above exception, but led to a different exception. I
didn't spend too long digging in though, and I worry I was missing something.
was:
Note that this only happens when a rollup is required ("forceRollup" in the
terminology of "rewriteView").
Here's a test in MaterializationTest that shows the behavior:
{code:java}
@Test public void testAggregateMaterializationOnProjectWithDuplicates() {
checkMaterialize(
"select \"empid\" / 2, \"empid\", floor(cast('1997-01-20 12:34:56' as
timestamp) to month), count(*) + 1 as c, sum(\"empid\") as s\n"
+ "from \"emps\" group by \"empid\" / 2, \"empid\", floor(cast('1997-01-20
12:34:56' as timestamp) to month)",
"select e1, e2, f, sum(e1) as s from (select \"empid\" as e1, \"empid\" as e2,
floor(cast('1997-01-20 12:34:56' as timestamp) to year) as f\n"
+ "from \"emps\" ) t1 " +
"group by e1,e2,f");
}{code}
The query SQL uses that somewhat artificial subquery to force the SQL to rel
parser to create the project with duplicates for the test.
Obviously this is a silly SQL query to write, but in the process of building
more complex queries it's not absurd to end up with a project with duplicate
RexNodes.
Here's the top of the stack produced by that test:
{noformat}
Caused by: org.apache.calcite.util.mapping.Mappings$NoElementException: source
#0 has no target in mapping [size=3, sourceCount=4, targetCount=23,
elements=[1:1, 2:5, 3:4]]Caused by:
org.apache.calcite.util.mapping.Mappings$NoElementException: source #0 has no
target in mapping [size=3, sourceCount=4, targetCount=23, elements=[1:1, 2:5,
3:4]] at
org.apache.calcite.util.mapping.Mappings$AbstractMapping.getTarget(Mappings.java:879)
at
org.apache.calcite.rel.rules.AbstractMaterializedViewRule$MaterializedViewAggregateRule.rewriteView(AbstractMaterializedViewRule.java:1502)
at
org.apache.calcite.rel.rules.AbstractMaterializedViewRule.perform(AbstractMaterializedViewRule.java:521){noformat}
I tried to make a quick fix for this, since on the surface the problem in the
code seems simple, at [the site of the exception|#L1513]] we do a lookup in a
mapping. The mapping comes about from inverting a mapping that maps projects in
the view to matching projects in the query ([the relevant line is
here|[https://github.com/apache/calcite/blob/134430e481bb5495d0852434041428104e29874e/core/src/main/java/org/apache/calcite/rel/rules/AbstractMaterializedViewRule.java#L1436]]),
and so if two query projects match the same view project, they will overwrite
each other, and the lookup on the inverse mapping will fail.
Trying the quick fix of setting the inverse mapping at the same time we set the
rewriting mapping (i.e. `inverseMapping.set(i, k);` for the above line, and a
couple others) fixed the above exception, but led to a different exception. I
didn't spend too long digging in though, and I worry I was missing something.
> AbstractMaterializedViewRule throws exception with an Aggregate on a Project
> projecting duplicate RexNodes
> ----------------------------------------------------------------------------------------------------------
>
> Key: CALCITE-3831
> URL: https://issues.apache.org/jira/browse/CALCITE-3831
> Project: Calcite
> Issue Type: Bug
> Reporter: Steven Talbot
> Priority: Major
>
> Note that this only happens when a rollup is required ("forceRollup" in the
> terminology of "rewriteView").
> Here's a test in MaterializationTest that shows the behavior:
>
> {code:java}
> @Test public void testAggregateMaterializationOnProjectWithDuplicates() {
> checkMaterialize(
> "select \"empid\" / 2, \"empid\", floor(cast('1997-01-20 12:34:56' as
> timestamp) to month), count(*) + 1 as c, sum(\"empid\") as s\n"
> + "from \"emps\" group by \"empid\" / 2, \"empid\", floor(cast('1997-01-20
> 12:34:56' as timestamp) to month)",
> "select e1, e2, f, sum(e1) as s from (select \"empid\" as e1, \"empid\" as
> e2, floor(cast('1997-01-20 12:34:56' as timestamp) to year) as f\n"
> + "from \"emps\" ) t1 " +
> "group by e1,e2,f");
> }{code}
>
> The query SQL uses that somewhat artificial subquery to force the SQL to rel
> parser to create the project with duplicates for the test.
> Obviously this is a silly SQL query to write, but in the process of building
> more complex queries it's not absurd to end up with a project with duplicate
> RexNodes.
> Here's the top of the stack produced by that test (sorry my line number
> locally are a bit different from current master):
>
> {noformat}
> Caused by: org.apache.calcite.util.mapping.Mappings$NoElementException:
> source #0 has no target in mapping [size=3, sourceCount=4, targetCount=23,
> elements=[1:1, 2:5, 3:4]]Caused by:
> org.apache.calcite.util.mapping.Mappings$NoElementException: source #0 has no
> target in mapping [size=3, sourceCount=4, targetCount=23, elements=[1:1, 2:5,
> 3:4]] at
> org.apache.calcite.util.mapping.Mappings$AbstractMapping.getTarget(Mappings.java:879)
> at
> org.apache.calcite.rel.rules.AbstractMaterializedViewRule$MaterializedViewAggregateRule.rewriteView(AbstractMaterializedViewRule.java:1502)
> at
> org.apache.calcite.rel.rules.AbstractMaterializedViewRule.perform(AbstractMaterializedViewRule.java:521){noformat}
>
> I tried to make a quick fix for this, since on the surface the problem in the
> code seems simple, at the site of the exception
> ([https://github.com/apache/calcite/blob/134430e481bb5495d0852434041428104e29874e/core/src/main/java/org/apache/calcite/rel/rules/AbstractMaterializedViewRule.java#L1513])
> we do a lookup in a mapping. The mapping comes about from inverting a
> mapping that maps projects in the view to matching projects in the query (the
> relevant line is
> [https://github.com/apache/calcite/blob/134430e481bb5495d0852434041428104e29874e/core/src/main/java/org/apache/calcite/rel/rules/AbstractMaterializedViewRule.java#L1436]),
> and so if two query projects match the same view project, they will
> overwrite each other, and the lookup on the inverse mapping will fail.
>
> Trying the quick fix of setting the inverse mapping at the same time we set
> the rewriting mapping (i.e. `inverseMapping.set(i, k);` for the above line,
> and a couple others) fixed the above exception, but led to a different
> exception. I didn't spend too long digging in though, and I worry I was
> missing something.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
