It's the import statement Olivier showed that makes the method available. Note that you can also use `sc.createDataFrame(myRDD)`, without the need for the import statement. I personally prefer this approach.
Dean Wampler, Ph.D. Author: Programming Scala, 2nd Edition <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) Typesafe <http://typesafe.com> @deanwampler <http://twitter.com/deanwampler> http://polyglotprogramming.com On Tue, May 12, 2015 at 9:33 AM, Olivier Girardot <ssab...@gmail.com> wrote: > you need to instantiate a SQLContext : > val sc : SparkContext = ... > val sqlContext = new SQLContext(sc) > import sqlContext.implicits._ > > Le mar. 12 mai 2015 à 12:29, SLiZn Liu <sliznmail...@gmail.com> a écrit : > >> I added `libraryDependencies += "org.apache.spark" % "spark-sql_2.11" % >> "1.3.1"` to `build.sbt` but the error remains. Do I need to import modules >> other than `import org.apache.spark.sql.{ Row, SQLContext }`? >> >> On Tue, May 12, 2015 at 5:56 PM Olivier Girardot <ssab...@gmail.com> >> wrote: >> >>> toDF is part of spark SQL so you need Spark SQL dependency + import >>> sqlContext.implicits._ to get the toDF method. >>> >>> Regards, >>> >>> Olivier. >>> >>> Le mar. 12 mai 2015 à 11:36, SLiZn Liu <sliznmail...@gmail.com> a >>> écrit : >>> >>>> Hi User Group, >>>> >>>> I’m trying to reproduce the example on Spark SQL Programming Guide >>>> <https://spark.apache.org/docs/latest/sql-programming-guide.html#inferring-the-schema-using-reflection>, >>>> and got a compile error when packaging with sbt: >>>> >>>> [error] myfile.scala:30: value toDF is not a member of >>>> org.apache.spark.rdd.RDD[Person] >>>> [error] val people = >>>> sc.textFile("examples/src/main/resources/people.txt").map(_.split(",")).map(p >>>> => Person(p(0), p(1).trim.toInt)).toDF() >>>> [error] >>>> ^ >>>> [error] one error found >>>> [error] (compile:compileIncremental) Compilation failed >>>> [error] Total time: 3 s, completed May 12, 2015 4:11:53 PM >>>> >>>> I double checked my code includes import sqlContext.implicits._ after >>>> reading this post >>>> <https://mail-archives.apache.org/mod_mbox/spark-user/201503.mbox/%3c1426522113299-22083.p...@n3.nabble.com%3E> >>>> on spark mailing list, even tried to use toDF("col1", "col2") >>>> suggested by Xiangrui Meng in that post and got the same error. >>>> >>>> The Spark version is specified in build.sbt file as follows: >>>> >>>> scalaVersion := "2.11.6" >>>> libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "1.3.1" % >>>> "provided" >>>> libraryDependencies += "org.apache.spark" % "spark-mllib_2.11" % "1.3.1" >>>> >>>> Anyone have ideas the cause of this error? >>>> >>>> REGARDS, >>>> Todd Leo >>>> >>>> >>>
