[
https://issues.apache.org/jira/browse/FLINK-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234813#comment-15234813
]
Fabian Hueske commented on FLINK-3724:
--------------------------------------
This is another issue that requires special case handling for the efficient
type mode. I am starting to wonder whether the possible performance gains
justify the added code complexity in the program construction and code
generation code.
To resolve this concrete issue, I propose to not allow primitives as types in
Table programs but to wrap them in {{Tuple1}} instances. That way we can treat
them as composite types and have a common way to handle keys. The added
overhead should be insignificant. Serialization and comparisons performance
should not decrease (Tuple serializers do not add metadata) and {{Tuple1}}
instances should be reused by the generated code.
In the long run, we can also think about to only support {{Row}} and provide
nullable and non-nullable serializers and comparators for the same data type.
But this is a separate issue and should not be discussed here.
> testJoinPushThroughJoin fails for TableConfigMode=EFFICIENT
> -----------------------------------------------------------
>
> Key: FLINK-3724
> URL: https://issues.apache.org/jira/browse/FLINK-3724
> Project: Flink
> Issue Type: Bug
> Components: Table API, Tests
> Affects Versions: 1.1.0
> Reporter: Vasia Kalavri
>
> Efficient config mode in the Table API also makes the
> {{JoinITCase.testJoinPushThroughJoin}} test case fail, with the following
> error:
> {code}
> org.apache.flink.api.common.InvalidProgramException: Specifying keys via
> field positions is only valid for tuple data types. Type: Integer
> at
> org.apache.flink.api.common.operators.Keys$ExpressionKeys.<init>(Keys.java:217)
> at
> org.apache.flink.api.common.operators.Keys$ExpressionKeys.<init>(Keys.java:208)
> at
> org.apache.flink.api.java.operators.JoinOperator$JoinOperatorSets.where(JoinOperator.java:868)
> at
> org.apache.flink.api.table.plan.nodes.dataset.DataSetJoin.translateToPlan(DataSetJoin.scala:199)
> at
> org.apache.flink.api.table.plan.nodes.dataset.DataSetCalc.translateToPlan(DataSetCalc.scala:95)
> at
> org.apache.flink.api.table.plan.nodes.dataset.DataSetJoin.translateToPlan(DataSetJoin.scala:159)
> at
> org.apache.flink.api.table.plan.nodes.dataset.DataSetCalc.translateToPlan(DataSetCalc.scala:95)
> at
> org.apache.flink.api.java.table.JavaBatchTranslator.translate(JavaBatchTranslator.scala:90)
> at
> org.apache.flink.api.scala.table.ScalaBatchTranslator.translate(ScalaBatchTranslator.scala:51)
> at
> org.apache.flink.api.scala.table.TableConversions.toDataSet(TableConversions.scala:43)
> at
> org.apache.flink.api.scala.table.test.JoinITCase.testJoinPushThroughJoin(JoinITCase.scala:201)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
> at
> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
