ibm.com/person/us-wtan
>
>
>
> From:Matei Zaharia
> To: user@spark.apache.org,
> Date:06/04/2014 04:28 PM
> Subject:Re: reuse hadoop code in Spark
>
>
>
> Yes, you can write some glue in Spark to call these. Some functions to look
://researcher.ibm.com/person/us-wtan
From: Matei Zaharia
To: user@spark.apache.org,
Date: 06/04/2014 04:28 PM
Subject:Re: reuse hadoop code in Spark
Yes, you can write some glue in Spark to call these. Some functions to
look at:
- SparkContext.hadoopRDD lets you create an input RDD from
Yes, you can write some glue in Spark to call these. Some functions to look at:
- SparkContext.hadoopRDD lets you create an input RDD from an existing JobConf
configured by Hadoop (including InputFormat, paths, etc)
- RDD.mapPartitions lets you operate in all the values on one partition (block)
