After this line:
val sc = new SparkContext(conf)
You need to add this line:
import sc.implicits._ //this is used to implicitly convert an RDD to a
DataFrame.
Hope this helps
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/RDD-to-DataFrame-for-using-
Please remember to copy the user list next time. I might not be able
to respond quickly. There are many others who can help or who can
benefit from the discussion. Thanks! -Xiangrui
On Tue, Mar 17, 2015 at 12:04 PM, Jay Katukuri wrote:
> Great Xiangrui. It works now.
>
> Sorry that I needed to bu
Please check this section in the user guide:
http://spark.apache.org/docs/latest/sql-programming-guide.html#inferring-the-schema-using-reflection
You need `import sqlContext.implicits._` to use `toDF()`.
-Xiangrui
On Mon, Mar 16, 2015 at 2:34 PM, Jay Katukuri wrote:
> Hi Xiangrui,
> Thanks a lo
Try this:
val ratings = purchase.map { line =>
line.split(',') match { case Array(user, item, rate) =>
(user.toInt, item.toInt, rate.toFloat)
}.toDF("user", "item", "rate")
Doc for DataFrames:
http://spark.apache.org/docs/latest/sql-programming-guide.html
-Xiangrui
On Mon, Mar 16, 2015 at 9
Hi all,
I am trying to use the new ALS implementation under
org.apache.spark.ml.recommendation.ALS.
The new method to invoke for training seems to be override def fit(dataset:
DataFrame, paramMap: ParamMap): ALSModel.
How do I create a dataframe object from ratings data set that is on hdfs ?
