Re: When to use CombineByKey vs reduceByKey?

2014-06-12 Thread Diana Hu
iners, mergeValue, etc) and return that instead of > allocating a new object. So it should work with mutable objects — please > post what problems you had with that. reduceByKey actually also allows this > if your types are the same. > > Matei > > > On Jun 11, 2014, at 3:21 PM,

When to use CombineByKey vs reduceByKey?

2014-06-11 Thread Diana Hu
Hello all, I've seen some performance improvements using combineByKey as opposed to reduceByKey or a groupByKey+map function. I have a couple questions. it'd be great if any one can provide some light into this. 1) When should I use combineByKey vs reduceByKey? 2) Do the containers need to be im