Thank you for your advice - I really need this help and promise to post a blog entry once it works
I ran >bin\spark-class.cmd org.apache.spark.deploy.master.Master and this ran successfully and I got a web page at http://localhost:8080 this says Spark Master at spark://192.168.1.4:7077 *My machine could be called localhost, asterix or 192.169.1.4* Then in a separate cmd window I tryed >bin\spark-class.cmd org.apache.spark.deploy.worker.Worker spark:// 192.168.1.4:7077 14/08/22 11:52:42 INFO SecurityManager: Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 14/08/22 11:52:42 INFO SecurityManager: Changing view acls to: Steve 14/08/22 11:52:42 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(Steve) 14/08/22 11:52:43 INFO Slf4jLogger: Slf4jLogger started 14/08/22 11:52:43 INFO Remoting: Starting remoting 14/08/22 11:52:43 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkWorker@192.168.1.4:53068] 14/08/22 11:52:43 INFO Worker: Starting Spark worker 192.168.1.4:53068 with 6 cores, 14.9 GB RAM 14/08/22 11:52:43 INFO Worker: Spark home: e:\spark-1.0.2-bin-hadoop1\bin\.. 14/08/22 11:52:43 INFO WorkerWebUI: Started WorkerWebUI at http://192.168.1.4:8081 14/08/22 11:52:43 INFO Worker: Connecting to master spark://192.168.1.4:7077 ... 14/08/22 11:52:43 INFO Worker: Successfully registered with master spark:// 192.168.1.4:7077 Now I fire up IntelliJ and try to run JavaWordCount with the command line including -Dspark.master=spark://192.168.1.4:7077 And it says 14/08/22 11:52:58 INFO Worker: Asked to launch executor app-20140822115258-0002/0 for JavaWordCount 14/08/22 11:52:58 INFO ExecutorRunner: Launch command: "C:\Progra~1\Java\jdk1.6.0_25/bin/java" "-cp" "E:\ThirdParty\spark-1.0.2\bin\..\conf;E:\ThirdParty\spark-1.0.2\bin\..\assembly\target\scala-2.10\spark-assembly-1.0.2-hadoop1.0.3-mapr-3.0.3.jar;;E:\ThirdParty\spark-1.0.2\bin\..\lib_managed\jars\datanucleus-api-jdo-3.2.1.jar;E:\ThirdParty\spark-1.0.2\bin\..\lib_managed\jars\datanucleus-core-3.2.2.jar;E:\ThirdParty\spark-1.0.2\bin\..\lib_managed\jars\datanucleus-rdbms-3.2.1.jar" "-XX:MaxPermSize=128m" "-Xms512M" "-Xmx512M" "org.apache.spark.executor.CoarseGrainedExecutorBackend" "akka.tcp:// spark@192.168.1.4:53092/user/CoarseGrainedScheduler" "0" "192.168.1.4" "6" "akka.tcp://sparkWorker@192.168.1.4:53068/user/Worker" "app-20140822115258-0002" 14/08/22 11:53:01 INFO Worker: Asked to kill executor app-20140822115258-0002/0 14/08/22 11:53:01 INFO ExecutorRunner: Runner thread for executor app-20140822115258-0002/0 interrupted 14/08/22 11:53:01 INFO ExecutorRunner: Killing process! Exception in thread "ExecutorRunner for app-20140822115258-0002/0" java.lang.InterruptedException at java.lang.ProcessImpl.waitFor(Native Method) at org.apache.spark.deploy.worker.ExecutorRunner.org $apache$spark$deploy$worker$ExecutorRunner$$killProcess(ExecutorRunner.scala:80) at org.apache.spark.deploy.worker.ExecutorRunner.fetchAndRunExecutor(ExecutorRunner.scala:157) at org.apache.spark.deploy.worker.ExecutorRunner$$anon$1.run(ExecutorRunner.scala:58) 14/08/22 11:53:01 INFO LocalActorRef: Message .... Clearly I am making progress but now need to know how to launch - preferably in a debugger against the worker I have launched
