For local mode the job is running with the bits it was linked to. Make sure these are the same bits running on your cluster. Mahout links to Spark 1.1.0, is that what you are running on the cluster?
On Dec 21, 2014, at 9:06 PM, hlqv <[email protected]> wrote: @Pat Ferrel: Thank you so much. I restarted Hadoop and now it works fine for local mode (--master local[n]). I ran an example JavaWordCount with --master spark://localhost:7077 and got the correct result. But when I ran spark-itemsimilarity again with --master spark://localhost:7077 then I got the error in the log: ERROR SparkDeploySchedulerBackend: Application has been killed. Reason: All masters are unresponsive! Giving up It seems relating to the WARN log: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory I have looked the Spark UI, every thing look correct with two workers are available and 4GB RAM is usable. On 22 December 2014 at 06:11, Pat Ferrel <[email protected]> wrote: > Looks like it's refusing to initialize your spark context. Did you start > the master spark://localhost:7077 before running mahout? Can you see a UI > at localhost:8080? > > If no to any of those try --master local[n], which will run the job > locally not in a single machine cluster and will run much faster. Replace n > with the number of cores you want to allocate, like local[7]. > > You should use the Spark start scripts when using a single machine cluster. > > If none of this helps, are you able to run any of the Spark examples on > spark://localhost:7077? > > > On Dec 21, 2014, at 9:28 AM, hlqv <[email protected]> wrote: > > Hi everyone! > > In Mahout 1.0 Snapshot, I follow to this introduction > https://mahout.apache.org/users/recommender/intro-cooccurrence-spark.html > > and running the command below: > > mahout spark-itemsimilarity --input hdfs://localhost:9000/test/logs.csv > --output hdfs://localhost:9000/test/output/ --master spark://localhost:7077 > --filter1 purchase --filter2 view --itemIDColumn 2 --rowIDColumn 0 > --filterColumn 1 > > then I got the error: > > Exception in thread "main" com.typesafe.config.ConfigException$Missing: No > configuration setting found for key 'akka.event-handlers' > at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:115) > at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:136) > at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:142) > at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:150) > at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:155) > at com.typesafe.config.impl.SimpleConfig.getList(SimpleConfig.java:203) > at > > com.typesafe.config.impl.SimpleConfig.getHomogeneousUnwrappedList(SimpleConfig.java:260) > at > com.typesafe.config.impl.SimpleConfig.getStringList(SimpleConfig.java:318) > at akka.actor.ActorSystem$Settings.<init>(ActorSystem.scala:150) > at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:470) > at akka.actor.ActorSystem$.apply(ActorSystem.scala:111) > at akka.actor.ActorSystem$.apply(ActorSystem.scala:104) > at > > org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121) > at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:54) > at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53) > at > > org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1446) > at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141) > at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1442) > at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:56) > at org.apache.spark.SparkEnv$.create(SparkEnv.scala:153) > at org.apache.spark.SparkContext.<init>(SparkContext.scala:203) > at > > org.apache.mahout.sparkbindings.package$.mahoutSparkContext(package.scala:95) > at > > org.apache.mahout.drivers.MahoutSparkDriver.start(MahoutSparkDriver.scala:81) > at > > org.apache.mahout.drivers.ItemSimilarityDriver$.start(ItemSimilarityDriver.scala:128) > at > > org.apache.mahout.drivers.ItemSimilarityDriver$.process(ItemSimilarityDriver.scala:211) > at > > org.apache.mahout.drivers.ItemSimilarityDriver$$anonfun$main$1.apply(ItemSimilarityDriver.scala:116) > at > > org.apache.mahout.drivers.ItemSimilarityDriver$$anonfun$main$1.apply(ItemSimilarityDriver.scala:114) > at scala.Option.map(Option.scala:145) > at > > org.apache.mahout.drivers.ItemSimilarityDriver$.main(ItemSimilarityDriver.scala:114) > at > > org.apache.mahout.drivers.ItemSimilarityDriver.main(ItemSimilarityDriver.scala) > > I really don't know there is any mistake in my configuration, I installed > Hadoop with CDH4 and Spark with the version for CDH4 and also put a sample > file to HDFS. > > Thanks for your help > >
