Sure in local mode it works for me as well, the issue is that I run master only, I needed worker as well.
תודה רבה, משה בארי. 054-3133943 Email <moshe.be...@gmail.com> | linkedin <http://www.linkedin.com/in/mobee> On Mon, Sep 22, 2014 at 9:58 AM, Akhil Das-2 [via Apache Spark User List] < ml-node+s1001560n14785...@n3.nabble.com> wrote: > Hi Moshe, > > I ran the same code on my machine and it is working without any issues. > You can try running it in local mode and if that is working fine, then the > issue is with your configuration. > > [image: Inline image 1] > > Thanks > Best Regards > > On Sat, Sep 20, 2014 at 3:34 PM, Moshe Beeri <[hidden email] > <http://user/SendEmail.jtp?type=node&node=14785&i=0>> wrote: > >> Hi Sean, >> >> Thanks a lot for the answer , I loved your excellent book >> *Mahout in Action >> <http://www.amazon.com/Mahout-Action-Sean-Owen/dp/1935182684> *hope >> you'll keep on writing more books in the field of Big Data. >> The issue was with redundant Hadoop library, But now I am facing some >> other issue (see prev post in this thread) >> java.lang.ClassNotFoundException: com.example.scamel.Nizoz$$anonfun$3 >> >> But the class com.example.scamel.Nizoz (in fact Scala object) is the one >> under debugging. >> >> def main(args: Array[String]) { >> println(scala.tools.nsc.Properties.versionString) >> try { >> //Nizoz.connect >> val logFile = >> "/home/moshe/store/frameworks/spark-1.1.0-bin-hadoop1/README.md" // Should >> be some file on your system >> val conf = new SparkConf().setAppName("spark >> town").setMaster("spark://nash:7077"); //spark://master:7077 >> val sc = new SparkContext(conf) >> val logData = sc.textFile(logFile, 2).cache() >> *val numAs = logData.filter(line => line.contains("a")).count() >> // <- here is where the exception thrown * >> >> Do you have any idea whats wrong? >> Thanks, >> Moshe Beeri. >> >> >> ** >> >> >> תודה רבה, >> משה בארי. >> 054-3133943 >> [hidden email] <http://user/SendEmail.jtp?type=node&node=14731&i=0> | >> linkedin <http://www.linkedin.com/in/mobee> >> >> >> >> On Sat, Sep 20, 2014 at 12:02 PM, sowen [via Apache Spark User List] <[hidden >> email] <http://user/SendEmail.jtp?type=node&node=14731&i=1>> wrote: >> >>> Spark does not require Hadoop 2 or YARN. This looks like a problem with >>> the Hadoop installation as it is not funding native libraries it needs to >>> make some security related system call. Check the installation. >>> On Sep 20, 2014 9:13 AM, "Manu Suryavansh" <[hidden email] >>> <http://user/SendEmail.jtp?type=node&node=14724&i=0>> wrote: >>> >>>> Hi Moshe, >>>> >>>> Spark needs a Hadoop 2.x/YARN cluster. Other wise you can run it >>>> without hadoop in the stand alone mode. >>>> >>>> Manu >>>> >>>> >>>> >>>> On Sat, Sep 20, 2014 at 12:55 AM, Moshe Beeri <[hidden email] >>>> <http://user/SendEmail.jtp?type=node&node=14724&i=1>> wrote: >>>> >>>>> object Nizoz { >>>>> >>>>> def connect(): Unit = { >>>>> val conf = new SparkConf().setAppName("nizoz").setMaster("master"); >>>>> val spark = new SparkContext(conf) >>>>> val lines = >>>>> >>>>> spark.textFile("file:///home/moshe/store/frameworks/spark-1.1.0-bin-hadoop1/README.md") >>>>> val lineLengths = lines.map(s => s.length) >>>>> val totalLength = lineLengths.reduce((a, b) => a + b) >>>>> println("totalLength=" + totalLength) >>>>> >>>>> } >>>>> >>>>> def main(args: Array[String]) { >>>>> println(scala.tools.nsc.Properties.versionString) >>>>> try { >>>>> //Nizoz.connect >>>>> val logFile = >>>>> "/home/moshe/store/frameworks/spark-1.1.0-bin-hadoop1/README.md" // >>>>> Should >>>>> be some file on your system >>>>> val conf = new SparkConf().setAppName("Simple >>>>> Application").setMaster("spark://master:7077") >>>>> val sc = new SparkContext(conf) >>>>> val logData = sc.textFile(logFile, 2).cache() >>>>> val numAs = logData.filter(line => line.contains("a")).count() >>>>> val numBs = logData.filter(line => line.contains("b")).count() >>>>> println("Lines with a: %s, Lines with b: %s".format(numAs, >>>>> numBs)) >>>>> >>>>> } catch { >>>>> case e => { >>>>> println(e.getCause()) >>>>> println("stack:") >>>>> e.printStackTrace() >>>>> } >>>>> } >>>>> } >>>>> } >>>>> Runs with Scala 2.10.4 >>>>> The problem is this [vogue] exception: >>>>> >>>>> at com.example.scamel.Nizoz.main(Nizoz.scala) >>>>> Caused by: java.lang.RuntimeException: >>>>> java.lang.reflect.InvocationTargetException >>>>> at >>>>> >>>>> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131) >>>>> at org.apache.hadoop.security.Groups.<init>(Groups.java:64) >>>>> at >>>>> >>>>> org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:240) >>>>> ... >>>>> Caused by: java.lang.reflect.InvocationTargetException >>>>> at >>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >>>>> at >>>>> >>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) >>>>> at >>>>> >>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >>>>> ... >>>>> ... 10 more >>>>> Caused by: java.lang.UnsatisfiedLinkError: >>>>> org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V >>>>> at >>>>> org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native >>>>> Method) >>>>> at >>>>> >>>>> org.apache.hadoop.security.JniBasedUnixGroupsMapping.<clinit>(JniBasedUnixGroupsMapping.java:49) >>>>> >>>>> I have Hadoop 1.2.1 running on Ubuntu 14.04, the Scala console run as >>>>> expected. >>>>> >>>>> What am I doing wrong? >>>>> Any idea will be welcome >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> View this message in context: >>>>> http://apache-spark-user-list.1001560.n3.nabble.com/Fails-to-run-simple-Spark-Hello-World-scala-program-tp14718.html >>>>> Sent from the Apache Spark User List mailing list archive at >>>>> Nabble.com. >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [hidden email] >>>>> <http://user/SendEmail.jtp?type=node&node=14724&i=2> >>>>> For additional commands, e-mail: [hidden email] >>>>> <http://user/SendEmail.jtp?type=node&node=14724&i=3> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Manu Suryavansh >>>> >>> >>> >>> ------------------------------ >>> If you reply to this email, your message will be added to the >>> discussion below: >>> >>> http://apache-spark-user-list.1001560.n3.nabble.com/Fails-to-run-simple-Spark-Hello-World-scala-program-tp14718p14724.html >>> To unsubscribe from Fails to run simple Spark (Hello World) scala >>> program, click here. >>> NAML >>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >>> >> >> >> ------------------------------ >> View this message in context: Re: Fails to run simple Spark (Hello >> World) scala program >> <http://apache-spark-user-list.1001560.n3.nabble.com/Fails-to-run-simple-Spark-Hello-World-scala-program-tp14718p14731.html> >> >> Sent from the Apache Spark User List mailing list archive >> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com. >> > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-spark-user-list.1001560.n3.nabble.com/Fails-to-run-simple-Spark-Hello-World-scala-program-tp14718p14785.html > To unsubscribe from Fails to run simple Spark (Hello World) scala > program, click here > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=14718&code=bW9zaGUuYmVlcmlAZ21haWwuY29tfDE0NzE4fDE0NzUwMDQ2Ng==> > . > NAML > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Fails-to-run-simple-Spark-Hello-World-scala-program-tp14718p14907.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
