Ah sorry I miss read your question. In pyspark it looks like you just need to instantiate the Partitioner class with numPartitions and partitionFunc.
On Tue, Sep 1, 2015 at 11:13 AM shahid ashraf <[email protected]> wrote: > Hi > > I did not get this, e.g if i need to create a custom partitioner like > range partitioner. > > On Tue, Sep 1, 2015 at 3:22 PM, Jem Tucker <[email protected]> wrote: > >> Hi, >> >> You just need to extend Partitioner and override the numPartitions and >> getPartition methods, see below >> >> class MyPartitioner extends partitioner { >> def numPartitions: Int = // Return the number of partitions >> def getPartition(key Any): Int = // Return the partition for a given key >> } >> >> On Tue, Sep 1, 2015 at 10:15 AM shahid qadri <[email protected]> >> wrote: >> >>> Hi Sparkians >>> >>> How can we create a customer partition in pyspark >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> > > > -- > with Regards > Shahid Ashraf >
