flinksql关联hive维表java.lang.UnsupportedOperationException错误

2022-05-25 Thread hdxg1101300...@163.com 
你好：
我在使用flink1.12.4版本和hive1.1.0版本时遇到下面的错误：
   
场景时使用hive的catalog管理元数据，用flinksql把Kafka的数据注册成输入表，关联hive维表做数据拉宽；提交任务到yarn时遇到如下错误；
   sql如下：
   create view if not exists dwm_ai_robot_contact_view as select
CALLER,CALLEE,CUST_LEVEL,PRO_ID,REASON_CODE,INTENT_NAME,GEN_CENTER,BUS_PRO_ID 
from realtime_robot_contact table_a left join dc_dim.dim_province_code /*+ 
OPTIONS('lookup.join.cache.ttl'='12 h') */ for SYSTEM_TIME as of 
table_a.pro_time as dim on table_a.PRO_ID = dim.code；
错误信息：

   
 The program finished with the following exception:

org.apache.flink.client.program.ProgramInvocationException: The main method 
caused an error: class org.apache.calcite.sql.SqlSyntax$6: SPECIAL
at 
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
at 
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
at 
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
at 
org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:812)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:246)
at 
org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)
at 
org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
at 
org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
Caused by: java.lang.UnsupportedOperationException: class 
org.apache.calcite.sql.SqlSyntax$6: SPECIAL
at org.apache.calcite.util.Util.needToImplement(Util.java:1075)
at org.apache.calcite.sql.SqlSyntax$6.unparse(SqlSyntax.java:116)
at org.apache.calcite.sql.SqlOperator.unparse(SqlOperator.java:329)
at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:453)
at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:104)
at 
org.apache.calcite.sql.SqlSnapshot$SqlSnapshotOperator.unparse(SqlSnapshot.java:128)
at org.apache.calcite.sql.SqlSnapshot.unparse(SqlSnapshot.java:78)
at org.apache.calcite.sql.SqlAsOperator.unparse(SqlAsOperator.java:76)
at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:453)
at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:104)
at 
org.apache.calcite.sql.SqlJoin$SqlJoinOperator.unparse(SqlJoin.java:229)
at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:453)
at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:104)
at 
org.apache.calcite.sql.SqlSelectOperator.unparse(SqlSelectOperator.java:176)
at org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:453)
at org.apache.calcite.sql.SqlSelect.unparse(SqlSelect.java:246)
at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:154)
at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:176)
at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:185)
at 
org.apache.flink.table.planner.operations.SqlToOperationConverter.getQuotedSqlString(SqlToOperationConverter.java:984)
at 
org.apache.flink.table.planner.utils.Expander$Expanded.substitute(Expander.java:183)
at 
org.apache.flink.table.planner.operations.SqlToOperationConverter.convertViewQuery(SqlToOperationConverter.java:854)
at 
org.apache.flink.table.planner.operations.SqlToOperationConverter.convertCreateView(SqlToOperationConverter.java:823)
at 
org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:250)
at 
org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:101)
at 
org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:724)
at com.unicom.app.MultiSubmitSql.main(MultiSubmitSql.java:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
... 11 more

 这是错误信息。还希望帮忙解答。

在本地 idea上执行上述语句没有问题。


hdxg1101300...@163.com

Re: Re: flinksql关联hive维表java.lang.UnsupportedOperationException错误

2022-05-25 Thread hdxg1101300...@163.com 
flink-table-legacy 这个模块应该没用，但是我也没找到他在那，是一个独立的模块吗？



hdxg1101300...@163.com
 
发件人： Jingsong Li
发送时间： 2022-05-26 14:20
收件人： dev
主题： Re: flinksql关联hive维表java.lang.UnsupportedOperationException错误
如果flink-table-legacy包没用到，把它删掉即可修复你的问题
 
Best,
Jingsong
 
On Thu, May 26, 2022 at 2:16 PM hdxg1101300...@163.com <
hdxg1101300...@163.com> wrote:
 
> 你好：
> 我在使用flink1.12.4版本和hive1.1.0版本时遇到下面的错误：
>
>  
> 场景时使用hive的catalog管理元数据，用flinksql把Kafka的数据注册成输入表，关联hive维表做数据拉宽；提交任务到yarn时遇到如下错误；
>sql如下：
>create view if not exists dwm_ai_robot_contact_view as select
> CALLER,CALLEE,CUST_LEVEL,PRO_ID,REASON_CODE,INTENT_NAME,GEN_CENTER,BUS_PRO_ID
> from realtime_robot_contact table_a left join dc_dim.dim_province_code /*+
> OPTIONS('lookup.join.cache.ttl'='12 h') */ for SYSTEM_TIME as of
> table_a.pro_time as dim on table_a.PRO_ID = dim.code；
> 错误信息：
>
>
>  The program finished with the following exception:
>
> org.apache.flink.client.program.ProgramInvocationException: The main
> method caused an error: class org.apache.calcite.sql.SqlSyntax$6: SPECIAL
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
> at
> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:812)
> at
> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:246)
> at
> org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
> at
> org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at
> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
> Caused by: java.lang.UnsupportedOperationException: class
> org.apache.calcite.sql.SqlSyntax$6: SPECIAL
> at org.apache.calcite.util.Util.needToImplement(Util.java:1075)
> at org.apache.calcite.sql.SqlSyntax$6.unparse(SqlSyntax.java:116)
> at org.apache.calcite.sql.SqlOperator.unparse(SqlOperator.java:329)
> at
> org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:453)
> at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:104)
> at
> org.apache.calcite.sql.SqlSnapshot$SqlSnapshotOperator.unparse(SqlSnapshot.java:128)
> at org.apache.calcite.sql.SqlSnapshot.unparse(SqlSnapshot.java:78)
> at
> org.apache.calcite.sql.SqlAsOperator.unparse(SqlAsOperator.java:76)
> at
> org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:453)
> at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:104)
> at
> org.apache.calcite.sql.SqlJoin$SqlJoinOperator.unparse(SqlJoin.java:229)
> at
> org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:453)
> at org.apache.calcite.sql.SqlCall.unparse(SqlCall.java:104)
> at
> org.apache.calcite.sql.SqlSelectOperator.unparse(SqlSelectOperator.java:176)
> at
> org.apache.calcite.sql.SqlDialect.unparseCall(SqlDialect.java:453)
> at org.apache.calcite.sql.SqlSelect.unparse(SqlSelect.java:246)
> at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:154)
> at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:176)
> at org.apache.calcite.sql.SqlNode.toSqlString(SqlNode.java:185)
> at
> org.apache.flink.table.planner.operations.SqlToOperationConverter.getQuotedSqlString(SqlToOperationConverter.java:984)
> at
> org.apache.flink.table.planner.utils.Expander$Expanded.substitute(Expander.java:183)
> at
> org.apache.flink.table.planner.operations.SqlToOperationConverter.convertViewQuery(SqlToOperationConverter.java:854)
> at
> org.apache.flink.table.planner.operations.SqlToOperationConverter.convertCreateView(SqlToOperationConverter.java:823)
> at
> org.apache.flink.table.planner.operations.SqlToOperationConverter.convert(SqlToOperationConverter.java:250)
> at
> org.apache.flink.table.planner.delegation.ParserImpl.parse(ParserImpl.java:101)
> at
> org.apache.flink.table.api.internal.TableEnvironmentImpl.executeSql(TableEnvironmentImpl.java:724)
> at com.unico