You can use JavaPairRDD which has: override def wrapRDD(rdd: RDD[(K, V)]): JavaPairRDD[K, V] = JavaPairRDD.fromRDD(rdd)
Cheers On Thu, Feb 12, 2015 at 7:36 AM, Vladimir Protsenko <[email protected]> wrote: > Hi. I am stuck with how to save file to hdfs from spark. > > I have written MyOutputFormat extends FileOutputFormat<String, MyObject>, > then in spark calling this: > > rddres.saveAsHadoopFile[MyOutputFormat]("hdfs://localhost/output") or > rddres.saveAsHadoopFile("hdfs://localhost/output", classOf[String], > classOf[MyObject], > classOf[MyOutputFormat]) > > where rddres is RDD[(String, MyObject)] from up of transformation pipeline. > > Compilation error is: /value saveAsHadoopFile is not a member of > org.apache.spark.rdd.RDD[(String, vlpr.MyObject)]/. > > Could someone give me insights on what could be done here to make it > working? Why it is not a member? Because of wrong types? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/saveAsHadoopFile-is-not-a-member-of-RDD-String-MyObject-tp21627.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
