Alex,
If not, you can try using the functions coalesce(n) or repartition(n).
As per the API, coalesce will not make a shuffle but repartition will.
Regards.
2015-10-16 0:52 GMT+01:00 Mohammed Guller :
> You may find the spark.sql.shuffle.partitions property useful. The default
> value is 200.
You may find the spark.sql.shuffle.partitions property useful. The default
value is 200.
Mohammed
From: Alex Nastetsky [mailto:alex.nastet...@vervemobile.com]
Sent: Wednesday, October 14, 2015 8:14 PM
To: user
Subject: dataframes and numPartitions
A lot of RDD methods take a numPartitions param
