[
https://issues.apache.org/jira/browse/HIVE-14369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399006#comment-15399006
]
Xiaomeng Huang commented on HIVE-14369:
---------------------------------------
Hi [~ruili]
It can't work, too. I think I should set both hivecluster and sparkcluster in
hdfs-site.xml of sparkcluster.
> submit a task with hive on spark to other yarn cluster failed
> -------------------------------------------------------------
>
> Key: HIVE-14369
> URL: https://issues.apache.org/jira/browse/HIVE-14369
> Project: Hive
> Issue Type: Bug
> Reporter: Xiaomeng Huang
>
> In our environment, we have two hadoop clusters with HA, named hivecluster
> and sparkcluster.
> hivecluster is a HA hadoop cluster for hive, which has large hard disk.
> sparkcluster is a HA hadoop cluster for spark, which has large memory.
> e.g. below is a hdfs-site.xml of hivecluster:
> {code}
> <property>
> <name>dfs.ha.namenodes.hivecluster</name>
> <value>nn1,nn2</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.hivecluster.nn1</name>
> <value>10.17.21.32:9000</value>
> </property>
> <property>
> <name>dfs.namenode.rpc-address.hivecluster.nn2</name>
> <value>10.17.21.77:9000</value>
> </property>
> <property>
> <name>dfs.namenode.http-address.hivecluster.nn1</name>
> <value>10.17.21.32:50070</value>
> </property>
> <property>
> <name>dfs.namenode.http-address.hivecluster.nn2</name>
> <value>10.17.21.77:50070</value>
> </property>
> {code}
> Firstly, I created a hive table located as
> hdfs://hivecluster/hive/warehouse/xxx
> If I use hive on mr, it will run successfully.
> But if I use hive on spark to submit a task to yarn cluster of sparkcluster,
> it says:
> {code}
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.spark.SparkTask
> yarn日志显示：
> Diagnostics: java.lang.IllegalArgumentException:
> java.net.UnknownHostException: hivecluster
> Failing this attempt. Failing the application.
> {code}
> I didn't set host of hivecluster into hdfs-site.xml of sparkcluster
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
