Hey Rishitesh,
Thats perfect thanks so much! Dont know why i didnt think of using
mapPartitions like this
Thanks,
Jem
On Fri, Aug 28, 2015 at 10:35 AM Rishitesh Mishra
wrote:
> Hi Jem,
> A simple way to get this is to use MapPartitionedRDD. Please see the below
> code. For this you need to kn
Hi Jem,
A simple way to get this is to use MapPartitionedRDD. Please see the below
code. For this you need to know your parent RDD's partition numbers that
you want to exclude. One drawback here is the new RDD will also invoke
similar number of tasks as parent RDDs as both the RDDs have same numbe
