您好:
我是西南财经大学一个在读博士,目前想使用PyFlink,但是遇到了一个问题,我的集群采用的CDH6.2.0,其中的Hadoop为3.0.0版本。当我在pytlink的程序里引入hadoop-common-3.0.0.jar(或hadoop-common-3.0.0-cdh6.2.0.jar)的时候,运行程序就会报错:
Caused by: java.lang.NoClassDefFoundError: Could not initialize class
org.apache.hadoop.security.UserGroupInformation
at
org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.security.User$SecureHadoopUser.<init>(User.java:300)
at
org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.security.User.getCurrent(User.java:173)
at
org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.security.UserProvider.getCurrent(UserProvider.java:187)
at
org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.AuthUtil.loginClient(AuthUtil.java:107)
at
org.apache.flink.hbase.shaded.org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:128)
at
org.apache.flink.connector.hbase.sink.HBaseSinkFunction.open(HBaseSinkFunction.java:116)
at
org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:34)
at
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:100)
at
org.apache.flink.table.runtime.operators.sink.SinkOperator.open(SinkOperator.java:58)
at
org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:107)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:700)
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:676)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:643)
at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:948)
at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:917)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:741)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:563)
at java.lang.Thread.run(Thread.java:750)
我看在github上采用的hadoop-common版本为2.8.5,是不支持3.0.0吗?还是什么原因呀,期望解答,万分感谢。