nopes. eclipse.
On Fri, Jun 17, 2016 at 3:58 PM, Siva A <siva9940261...@gmail.com> wrote: > If you are running from IDE, Are you using Intellij? > > On Fri, Jun 17, 2016 at 3:20 PM, Siva A <siva9940261...@gmail.com> wrote: > >> Can you try to package as a jar and run using spark-submit >> >> Siva >> >> On Fri, Jun 17, 2016 at 3:17 PM, VG <vlin...@gmail.com> wrote: >> >>> I am trying to run from IDE and everything else is working fine. >>> I added spark-xml jar and now I ended up into this dependency >>> >>> 6/06/17 15:15:57 INFO BlockManagerMaster: Registered BlockManager >>> Exception in thread "main" *java.lang.NoClassDefFoundError: >>> scala/collection/GenTraversableOnce$class* >>> at >>> org.apache.spark.sql.execution.datasources.CaseInsensitiveMap.<init>(ddl.scala:150) >>> at >>> org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:154) >>> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:119) >>> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:109) >>> at org.ariba.spark.PostsProcessing.main(PostsProcessing.java:19) >>> Caused by:* java.lang.ClassNotFoundException: >>> scala.collection.GenTraversableOnce$class* >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> ... 5 more >>> 16/06/17 15:15:58 INFO SparkContext: Invoking stop() from shutdown hook >>> >>> >>> >>> On Fri, Jun 17, 2016 at 2:59 PM, Marco Mistroni <mmistr...@gmail.com> >>> wrote: >>> >>>> So you are using spark-submit or spark-shell? >>>> >>>> you will need to launch either by passing --packages option (like in >>>> the example below for spark-csv). you will need to iknow >>>> >>>> --packages com.databricks:spark-xml_<scala.version>:<package version> >>>> >>>> hth >>>> >>>> >>>> >>>> On Fri, Jun 17, 2016 at 10:20 AM, VG <vlin...@gmail.com> wrote: >>>> >>>>> Apologies for that. >>>>> I am trying to use spark-xml to load data of a xml file. >>>>> >>>>> here is the exception >>>>> >>>>> 16/06/17 14:49:04 INFO BlockManagerMaster: Registered BlockManager >>>>> Exception in thread "main" java.lang.ClassNotFoundException: Failed to >>>>> find data source: org.apache.spark.xml. Please find packages at >>>>> http://spark-packages.org >>>>> at >>>>> org.apache.spark.sql.execution.datasources.ResolvedDataSource$.lookupDataSource(ResolvedDataSource.scala:77) >>>>> at >>>>> org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:102) >>>>> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:119) >>>>> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:109) >>>>> at org.ariba.spark.PostsProcessing.main(PostsProcessing.java:19) >>>>> Caused by: java.lang.ClassNotFoundException: >>>>> org.apache.spark.xml.DefaultSource >>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>> at >>>>> org.apache.spark.sql.execution.datasources.ResolvedDataSource$$anonfun$4$$anonfun$apply$1.apply(ResolvedDataSource.scala:62) >>>>> at >>>>> org.apache.spark.sql.execution.datasources.ResolvedDataSource$$anonfun$4$$anonfun$apply$1.apply(ResolvedDataSource.scala:62) >>>>> at scala.util.Try$.apply(Try.scala:192) >>>>> at >>>>> org.apache.spark.sql.execution.datasources.ResolvedDataSource$$anonfun$4.apply(ResolvedDataSource.scala:62) >>>>> at >>>>> org.apache.spark.sql.execution.datasources.ResolvedDataSource$$anonfun$4.apply(ResolvedDataSource.scala:62) >>>>> at scala.util.Try.orElse(Try.scala:84) >>>>> at >>>>> org.apache.spark.sql.execution.datasources.ResolvedDataSource$.lookupDataSource(ResolvedDataSource.scala:62) >>>>> ... 4 more >>>>> >>>>> Code >>>>> SQLContext sqlContext = new SQLContext(sc); >>>>> DataFrame df = sqlContext.read() >>>>> .format("org.apache.spark.xml") >>>>> .option("rowTag", "row") >>>>> .load("A.xml"); >>>>> >>>>> Any suggestions please .. >>>>> >>>>> >>>>> >>>>> >>>>> On Fri, Jun 17, 2016 at 2:42 PM, Marco Mistroni <mmistr...@gmail.com> >>>>> wrote: >>>>> >>>>>> too little info >>>>>> it'll help if you can post the exception and show your sbt file (if >>>>>> you are using sbt), and provide minimal details on what you are doing >>>>>> kr >>>>>> >>>>>> On Fri, Jun 17, 2016 at 10:08 AM, VG <vlin...@gmail.com> wrote: >>>>>> >>>>>>> Failed to find data source: com.databricks.spark.xml >>>>>>> >>>>>>> Any suggestions to resolve this >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
