SPARK-2356 On Fri, Apr 3, 2015 at 1:01 PM, Ulanov, Alexander <alexander.ula...@hp.com> wrote: > Thanks! It worked. I've updated the PR and now start the test with working > directory in SPARK_HOME > > Now I get: > ERROR Shell: Failed to locate the winutils binary in the hadoop binary path > java.io.IOException: Could not locate executable null\bin\winutils.exe in the > Hadoop binaries. > > Any ideas? > > -----Original Message----- > From: Marcelo Vanzin [mailto:van...@cloudera.com] > Sent: Friday, April 03, 2015 12:52 PM > To: Ulanov, Alexander > Cc: dev@spark.apache.org > Subject: Re: Running LocalClusterSparkContext > > That looks like another bug on top of 6673; can you point that out in the PR > to make sure it's covered? > > In the meantime, could you try running the command from $SPARK_HOME instead > of from inside the mllib directory? > > On Fri, Apr 3, 2015 at 12:48 PM, Ulanov, Alexander <alexander.ula...@hp.com> > wrote: >> Hi Marcelo, >> >> Thank you for quick response! >> It seems that I get the issue 6673. If I set set SPARK_SCALA_VERSION=2.10 as >> suggested in https://issues.apache.org/jira/browse/SPARK-6673, I get instead: >> >> 15/04/03 12:46:24.510 ExecutorRunner for app-20150403124624-0000/0 >> ERROR ExecutorRunner: Error running executor >> java.lang.IllegalStateException: No assemblies found in >> 'C:\ulanov\dev\spark\mllib\.\assembly\target\scala-2.10'. >> >> -----Original Message----- >> From: Marcelo Vanzin [mailto:van...@cloudera.com] >> Sent: Friday, April 03, 2015 12:31 PM >> To: Ulanov, Alexander >> Cc: dev@spark.apache.org >> Subject: Re: Running LocalClusterSparkContext >> >> When was the last time you pulled? That should have been fixed as part of >> SPARK-6473. >> >> Notice latest master suffers from SPARK-6673 on Windows. >> >> On Fri, Apr 3, 2015 at 12:21 PM, Ulanov, Alexander <alexander.ula...@hp.com> >> wrote: >>> Hi, >>> >>> I am trying to execute unit tests with LocalClusterSparkContext on Windows >>> 7. I am getting a bunch of error in the log saying that: "Cannot find any >>> assembly build directories." Below is the part from the log where it >>> brakes. Could you suggest what's happening? In addition the application has >>> created a folder "app-20150403121631-0000" and is keeping to create empty >>> folders there named with numbers until I stop the application. >>> >>> >>> 15/04/03 12:16:31.239 sparkWorker2-akka.actor.default-dispatcher-3 INFO >>> Worker: Connecting to master >>> akka.tcp://sparkmas...@mynetwork.net:51990/user/Master... >>> 15/04/03 12:16:31.277 sparkDriver-akka.actor.default-dispatcher-2 INFO >>> AppClient$ClientActor: Connecting to master >>> akka.tcp://sparkmas...@mynetwork.net:51990/user/Master... >>> 15/04/03 12:16:31.549 sparkMaster-akka.actor.default-dispatcher-7 >>> INFO >>> Master: Registering worker Mynetwork.net:52006 with 1 cores, 512.0 MB >>> RAM >>> 15/04/03 12:16:31.556 sparkMaster-akka.actor.default-dispatcher-7 >>> INFO >>> Master: Registering worker Mynetwork.net:52020 with 1 cores, 512.0 MB >>> RAM >>> 15/04/03 12:16:31.561 sparkMaster-akka.actor.default-dispatcher-7 >>> INFO >>> Master: Registering app test-cluster >>> 15/04/03 12:16:31.568 sparkMaster-akka.actor.default-dispatcher-7 >>> INFO >>> Master: Registered app test-cluster with ID app-20150403121631-0000 >>> 15/04/03 12:16:31.573 sparkWorker1-akka.actor.default-dispatcher-3 >>> INFO Worker: Successfully registered with master >>> spark://Mynetwork.net:51990 >>> 15/04/03 12:16:31.588 sparkMaster-akka.actor.default-dispatcher-7 >>> INFO >>> Master: Launching executor app-20150403121631-0000/0 on worker >>> worker-20150403121631-Mynetwork.net-52020 >>> 15/04/03 12:16:31.589 sparkMaster-akka.actor.default-dispatcher-7 >>> INFO >>> Master: Launching executor app-20150403121631-0000/1 on worker >>> worker-20150403121631-Mynetwork.net-52006 >>> 15/04/03 12:16:31.590 sparkWorker2-akka.actor.default-dispatcher-3 >>> INFO Worker: Successfully registered with master >>> spark://Mynetwork.net:51990 >>> 15/04/03 12:16:31.595 sparkDriver-akka.actor.default-dispatcher-3 >>> INFO >>> SparkDeploySchedulerBackend: Connected to Spark cluster with app ID >>> app-20150403121631-0000 >>> 15/04/03 12:16:31.608 sparkWorker1-akka.actor.default-dispatcher-4 >>> INFO Worker: Asked to launch executor app-20150403121631-0000/1 for >>> test-cluster >>> 15/04/03 12:16:31.624 sparkWorker2-akka.actor.default-dispatcher-5 >>> INFO Worker: Asked to launch executor app-20150403121631-0000/0 for >>> test-cluster >>> 15/04/03 12:16:31.639 sparkDriver-akka.actor.default-dispatcher-3 >>> INFO >>> AppClient$ClientActor: Executor added: app-20150403121631-0000/0 on >>> worker-20150403121631-Mynetwork.net-52020 (Mynetwork.net:52020) with >>> 1 cores >>> 15/04/03 12:16:31.683 sparkDriver-akka.actor.default-dispatcher-3 >>> INFO >>> SparkDeploySchedulerBackend: Granted executor ID >>> app-20150403121631-0000/0 on hostPort Mynetwork.net:52020 with 1 >>> cores, 512.0 MB RAM >>> 15/04/03 12:16:31.683 sparkDriver-akka.actor.default-dispatcher-3 >>> INFO >>> AppClient$ClientActor: Executor added: app-20150403121631-0000/1 on >>> worker-20150403121631-Mynetwork.net-52006 (Mynetwork.net:52006) with >>> 1 cores >>> 15/04/03 12:16:31.684 sparkDriver-akka.actor.default-dispatcher-3 >>> INFO >>> SparkDeploySchedulerBackend: Granted executor ID >>> app-20150403121631-0000/1 on hostPort Mynetwork.net:52006 with 1 >>> cores, 512.0 MB RAM >>> 15/04/03 12:16:31.688 sparkDriver-akka.actor.default-dispatcher-3 >>> INFO >>> AppClient$ClientActor: Executor updated: app-20150403121631-0000/1 is >>> now LOADING >>> 15/04/03 12:16:31.688 sparkDriver-akka.actor.default-dispatcher-3 >>> INFO >>> AppClient$ClientActor: Executor updated: app-20150403121631-0000/0 is >>> now LOADING >>> 15/04/03 12:16:31.688 ExecutorRunner for app-20150403121631-0000/1 >>> ERROR ExecutorRunner: Error running executor >>> java.lang.IllegalStateException: Cannot find any assembly build directories. >>> at >>> org.apache.spark.launcher.CommandBuilderUtils.checkState(CommandBuilderUtils.java:228) >>> at >>> org.apache.spark.launcher.AbstractCommandBuilder.getScalaVersion(AbstractCommandBuilder.java:283) >>> at >>> org.apache.spark.launcher.AbstractCommandBuilder.buildClassPath(AbstractCommandBuilder.java:150) >>> at >>> org.apache.spark.launcher.AbstractCommandBuilder.buildJavaCommand(AbstractCommandBuilder.java:111) >>> at >>> org.apache.spark.launcher.WorkerCommandBuilder.buildCommand(WorkerCommandBuilder.scala:39) >>> at >>> org.apache.spark.launcher.WorkerCommandBuilder.buildCommand(WorkerCommandBuilder.scala:48) >>> at >>> org.apache.spark.deploy.worker.CommandUtils$.buildCommandSeq(CommandUtils.scala:61) >>> at >>> org.apache.spark.deploy.worker.CommandUtils$.buildProcessBuilder(CommandUtils.scala:49) >>> at >>> org.apache.spark.deploy.worker.ExecutorRunner.org$apache$spark$deploy$worker$ExecutorRunner$$fetchAndRunExecutor(ExecutorRunner.scala:132) >>> at >>> org.apache.spark.deploy.worker.ExecutorRunner$$anon$1.run(ExecutorRun >>> n >>> er.scala:68) >>> 15/04/03 12:16:31.712 sparkDriver-akka.actor.default-dispatcher-3 >>> INFO >>> AppClient$ClientActor: Executor updated: app-20150403121631-0000/0 is >>> now RUNNING >>> 15/04/03 12:16:31.725 sparkDriver-akka.actor.default-dispatcher-3 >>> INFO >>> AppClient$ClientActor: Executor updated: app-20150403121631-0000/1 is >>> now RUNNING >>> 15/04/03 12:16:31.726 sparkWorker1-akka.actor.default-dispatcher-4 INFO >>> Worker: Executor app-20150403121631-0000/1 finished with state FAILED >>> message java.lang.IllegalStateException: Cannot find any assembly build >>> directories. >>> 15/04/03 12:16:31.689 ExecutorRunner for app-20150403121631-0000/0 >>> ERROR ExecutorRunner: Error running executor >>> java.lang.IllegalStateException: Cannot find any assembly build directories. >>> >>> Best regards, Alexander >> >> >> >> -- >> Marcelo > > > > -- > Marcelo
-- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
