Did you try coalesce? It doesn't shuffle the data around. Thanks Best Regards
On Wed, Oct 21, 2015 at 10:27 AM, shahid <[email protected]> wrote: > Hi > > I have a large partition(data skewed) i need to split it to no. of > partitions, repartitioning causes lot of shuffle. Can we do that..? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Can-we-split-partition-tp25151.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
