[
https://issues.apache.org/jira/browse/FLINK-8155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jiesen.li updated FLINK-8155:
-----------------------------
Description:
There was an error when I start flink 1.3.2 on yarn of hdp2.6,
but when I tried remove +YARN HA+,it works.
So seems flink-1.3.2 can not work on YARN HA.
{code:java}
[root@ambtest2st] /opt/flink-1.3.2/bin$ ./yarn-session.sh -d -n 4 -tm 2048
-s 4
2017-11-23 16:54:32,808 INFO org.apache.hadoop.service.AbstractService
- Service org.apache.hadoop.yarn.client.api.impl.YarnClientImpl
failed in state STARTED; cause: java.lang.RuntimeException:
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2186)
at
org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:160)
at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:93)
at
org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:174)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.getYarnClient(AbstractYarnClusterDescriptor.java:381)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:458)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:441)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:630)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:486)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:483)
at
org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:483)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2154)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2178)
... 17 more
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2060)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2152)
... 18 more
Error while deploying YARN cluster: Couldn't deploy Yarn cluster
java.lang.RuntimeException: Couldn't deploy Yarn cluster
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:443)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:630)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:486)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:483)
at
org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:483)
Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2186)
at
org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:160)
at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:93)
at
org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:174)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.getYarnClient(AbstractYarnClusterDescriptor.java:381)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:458)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:441)
... 9 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2154)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2178)
... 17 more
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2060)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2152)
... 18 more
{code}
was:
There was an error when I start flink 1.3.2 on yarn of hdp2.6,
but when I tried remove +YARN HA+,it works.
So seems flink-1.3.2 can not work on YARN HA.
{code:java}
2017-11-23 16:54:32,808 INFO org.apache.hadoop.service.AbstractService
- Service org.apache.hadoop.yarn.client.api.impl.YarnClientImpl
failed in state STARTED; cause: java.lang.RuntimeException:
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2186)
at
org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:160)
at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:93)
at
org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:174)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.getYarnClient(AbstractYarnClusterDescriptor.java:381)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:458)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:441)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:630)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:486)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:483)
at
org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:483)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2154)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2178)
... 17 more
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2060)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2152)
... 18 more
Error while deploying YARN cluster: Couldn't deploy Yarn cluster
java.lang.RuntimeException: Couldn't deploy Yarn cluster
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:443)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:630)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:486)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:483)
at
org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40)
at
org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:483)
Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2186)
at
org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:160)
at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:93)
at
org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
at
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:174)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.getYarnClient(AbstractYarnClusterDescriptor.java:381)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:458)
at
org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:441)
... 9 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2154)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2178)
... 17 more
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2060)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2152)
... 18 more
{code}
> Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not
> found
> -----------------------------------------------------------------------------------
>
> Key: FLINK-8155
> URL: https://issues.apache.org/jira/browse/FLINK-8155
> Project: Flink
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.3.2
> Environment: java:
> openjdk version "1.8.0_151"
> OpenJDK Runtime Environment (build 1.8.0_151-b12)
> OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)
> linux system:
> 3.10.0-514.21.2.el7.x86_64 #1 SMP Tue Jun 20 12:24:47 UTC 2017 x86_64 x86_64
> x86_64 GNU/Linux
> hadoop version:
> HDP-2.6.1.0(apache hadoop 2.7)
> Reporter: jiesen.li
>
> There was an error when I start flink 1.3.2 on yarn of hdp2.6,
> but when I tried remove +YARN HA+,it works.
> So seems flink-1.3.2 can not work on YARN HA.
> {code:java}
> [root@ambtest2st] /opt/flink-1.3.2/bin$ ./yarn-session.sh -d -n 4 -tm 2048
> -s 4
> 2017-11-23 16:54:32,808 INFO org.apache.hadoop.service.AbstractService
> - Service org.apache.hadoop.yarn.client.api.impl.YarnClientImpl
> failed in state STARTED; cause: java.lang.RuntimeException:
> java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2186)
> at
> org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:160)
> at
> org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:93)
> at
> org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:174)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.getYarnClient(AbstractYarnClusterDescriptor.java:381)
> at
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:458)
> at
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:441)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:630)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:486)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:483)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:483)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not
> found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2154)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2178)
> ... 17 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2060)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2152)
> ... 18 more
> Error while deploying YARN cluster: Couldn't deploy Yarn cluster
> java.lang.RuntimeException: Couldn't deploy Yarn cluster
> at
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:443)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:630)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:486)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:483)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40)
> at
> org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:483)
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2186)
> at
> org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:160)
> at
> org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:93)
> at
> org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:174)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.getYarnClient(AbstractYarnClusterDescriptor.java:381)
> at
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:458)
> at
> org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:441)
> ... 9 more
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not
> found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2154)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2178)
> ... 17 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2060)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2152)
> ... 18 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
