Thank You Jason,
Got the program working after setting
SPARK_WORKER_CORES
SPARK_WORKER_MEMORY
While running the program from eclipse, got strange ClassNotFoundException.
In JavaKMeans.java, ParsePoint is static inner class. When running the
program I got ClassNotFound for ParsePoint.
I have to set
String jars[] = {
"/home/surendran/workspace/SparkKMeans/target/SparkKMeans-0.0.1-SNAPSHOT.jar"
};
sparkConf.setJars(jars);
(or) to set SPARK_CLASSPATH to my jar
to get the program working.
What if I modify ParsePoint? I need to rebuild to bring the changes and
run from eclipse?
Regards,
Surendran
On Sunday 22 February 2015 07:55 PM, Jason Bell wrote:
If you would like a morr detailed walkthrough I wrote one recently.
https://dataissexy.wordpress.com/2015/02/03/apache-spark-standalone-clusters-bigdata-hadoop-spark/
Regards
Jason Bell
On 22 Feb 2015 14:16, "VISHNU SUBRAMANIAN"
<johnfedrickena...@gmail.com <mailto:johnfedrickena...@gmail.com>> wrote:
Try restarting your Spark cluster .
./sbin/stop-all.sh
./sbin/start-all.sh
Thanks,
Vishnu
On Sun, Feb 22, 2015 at 7:30 PM, Surendran Duraisamy
<2013ht12...@wilp.bits-pilani.ac.in
<mailto:2013ht12...@wilp.bits-pilani.ac.in>> wrote:
Hello All,
I am new to Apache Spark, I am trying to run JavaKMeans.java
from Spark Examples in my Ubuntu System.
I downloaded spark-1.2.1-bin-hadoop2.4.tgz
<http://www.apache.org/dyn/closer.cgi/spark/spark-1.2.1/spark-1.2.1-bin-hadoop2.4.tgz>
and started sbin/start-master.sh
After starting Spark and access http://localhost:8080/ to look
at the status of my Spark Instance, and it shows as follows.
* *URL:*spark://vm:7077
* *Workers:*0
* *Cores:*0 Total, 0 Used
* *Memory:*0.0 B Total, 0.0 B Used
* *Applications:*0 Running, 4 Completed
* *Drivers:*0 Running, 0 Completed
* *Status:*ALIVE
Number of Cores is 0 and Memory is 0.0B. I think because of
this I am getting following error when I try to run
JavaKMeans.java
"Initial job has not accepted any resources; check your
cluster UI to ensure that workers are registered and have
sufficient memory"
Am I missing any configuration before running
sbin/start-master.sh?
Regards,
Surendran
