Change your conf/spark-env.sh:
export HADOOP_CONF_DIR="/etc/hadoop/conf"export YARN_CONF_DIR="/etc/hadoop/conf"
Date: Thu, 28 Aug 2014 16:19:05 -0700
From: [email protected]
To: [email protected]
Subject: problem connection to hdfs on localhost from spark-shell
I have HDFS servers running locally and "hadoop dfs -ls /" are all running fine.
>From spark-shell I do this:
val lines = sc.textFile("hdfs:///test")
and I get this error message.
java.io.IOException: Failed on local exception: java.io.EOFException; Host
Details : local host is: "localhost.localdomain/127.0.0.1"; destination host
is: "localhost":9000;
I tried changing the contents of /etc/hosts to no avail. I also tried urls like
hdfs://localhost:9000/test and many other variants. Nothing worked. Also I see
a message (during spark-shell startup) that it has bound to a 192.168.x.x
address. Any help is appreciated.
-- Bharath
If you reply to this email, your message will be added to the
discussion below:
http://apache-spark-user-list.1001560.n3.nabble.com/problem-connection-to-hdfs-on-localhost-from-spark-shell-tp13074.html
To start a new topic under Apache Spark User List, email
[email protected]
To unsubscribe from Apache Spark User List, click here.
NAML
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/problem-connection-to-hdfs-on-localhost-from-spark-shell-tp13074p13101.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
