[ https://issues.apache.org/jira/browse/FLINK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15731343#comment-15731343 ]
ASF GitHub Bot commented on FLINK-5226: --------------------------------------- Github user beyond1920 commented on a diff in the pull request: https://github.com/apache/flink/pull/2926#discussion_r91455098 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/plan/nodes/dataset/DataSetCalc.scala --- @@ -73,7 +75,11 @@ class DataSetCalc( val child = this.getInput val rowCnt = metadata.getRowCount(child) - val exprCnt = calcProgram.getExprCount + + // compute number of non-field access expressions (computations, conditions, etc.) + // we only want to account for computations, not for simple projections. + val exprCnt = calcProgram.getExprList.asScala.toList.count(!_.isInstanceOf[RexInputRef]) --- End diff -- maybe we could also exclude RexLiteral as well. > Eagerly project unused attributes > --------------------------------- > > Key: FLINK-5226 > URL: https://issues.apache.org/jira/browse/FLINK-5226 > Project: Flink > Issue Type: Improvement > Components: Table API & SQL > Affects Versions: 1.2.0 > Reporter: Fabian Hueske > Assignee: Fabian Hueske > > The optimizer does currently not eagerly remove unused attributes. > For example given a table {{tab5}} with five attributes {{a, b, c, d, e}}, > the following query > {code} > SELECT x.a, y.b FROM tab5 AS x, tab5 AS y WHERE x.a = y.a > {code} > would result in the non-optimized plan > {code} > LogicalProject(a=[$0], b=[$6]) > LogicalFilter(condition=[=($0, $5)]) > LogicalJoin(condition=[true], joinType=[inner]) > LogicalTableScan(table=[[tab5]]) > LogicalTableScan(table=[[tab5]]) > {code} > and the optimized plan: > {code} > DataSetCalc(select=[a, b0 AS b]) > DataSetJoin(where=[=(a, a0)], join=[a, b, c, d, e, a0, b0, c0, d0, e0], > joinType=[InnerJoin]) > DataSetScan(table=[[_DataSetTable_0]]) > DataSetScan(table=[[_DataSetTable_0]]) > {code} > This plan is inefficient because it joins all ten attributes of both tables > instead of eagerly projecting out all unused fields ({{x.b, x.c, x.d, x.e, > y.c, y.d, y.e}}). > Since this is one of the most common optimizations, I would assume that > Calcite provides some rules to extract eager projections. If this is the > case, the issue can be solved by adding such rules to {{FlinkRuleSets}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332)