Re: Equally weighted partitions in Spark

+ 1 )) } override def equals(other: Any): Boolean = other match { case r: EqualWeightPartitioner[_,_] => r.bucketToPartitionMap.sameElements(bucketToPartitionMap) case _ => false } } -- View this message in context: http://apache-spark-user-list.1001560.n3.na

Re: Equally weighted partitions in Spark

ion(key: Any): Int would >>> a) get the weight and then find the bucket >>> b) assign a random partition from the list of partitions associated >>> with >>> each bucket >>> >>> Just wanted to know if someone else had come across this issue before and >>> there was a better way of doing this. >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-spark-user-list.1001560.n3.nabble.com/Equally-weighted-partitions-in-Spark-tp5171p5212.html >>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>> >> >> >

Re: Equally weighted partitions in Spark

cket >> 4) getPartition(key: Any): Int would >> a) get the weight and then find the bucket >> b) assign a random partition from the list of partitions associated with >> each bucket >> >> Just wanted to know if someone else had come across this issue before and

Re: Equally weighted partitions in Spark

ist of partitions associated with > each bucket > > Just wanted to know if someone else had come across this issue before and > there was a better way of doing this. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Equall

Re: Equally weighted partitions in Spark

-weighted-partitions-in-Spark-tp5171p5212.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Equally weighted partitions in Spark

ese complex instruments evenly, the > overall job times would greatly reduce. > > > Deenar > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Equally-weighted-partitions-in-Spark-tp5171p5208.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Re: Equally weighted partitions in Spark

evenly, the overall job times would greatly reduce. Deenar -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Equally-weighted-partitions-in-Spark-tp5171p5208.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Equally weighted partitions in Spark

> > > Regards > Deenar > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Equally-weighted-partitions-in-Spark-tp5171.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Equally weighted partitions in Spark

buted. Has anyone created such a partitioner before? Regards Deenar -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Equally-weighted-partitions-in-Spark-tp5171.html Sent from the Apache Spark User List mailing list archive at Nabble.com.