Can't you extend a class in place of object which can be generic ?
class GenericAccumulator[B] extends AccumulatorParam[Seq[B]] {
}
On Wed, Oct 1, 2014 at 3:38 AM, Johan Stenberg <[email protected]>
wrote:
> Just realized that, of course, objects can't be generic, but how do I
> create a generic AccumulatorParam?
>
> 2014-10-01 12:33 GMT+02:00 Johan Stenberg <[email protected]>:
>
>> Hi,
>>
>> I have a problem with using accumulators in Spark. As seen on the Spark
>> website, if you want custom accumulators you can simply extend (with an
>> object) the AccumulatorParam trait. The problem is that I need to make that
>> object generic, such as this:
>>
>> object SeqAccumulatorParam[B] extends AccumulatorParam[Seq[B]] {
>>
>> override def zero(initialValue: Seq[B]): Seq[B] = Seq[B]()
>>
>> override def addInPlace(s1: Seq[B], s2: Seq[B]): Seq[B] = s1 ++ s2
>>
>> }
>>
>> But this gives me a compile error because objects can't use generic
>> parameters. My situation doesn't really allow me to define a
>> SeqAccumulatorParam for each given type since that would lead to a lot of
>> ugly code duplication.
>>
>> I have an alternative method, just placing all of the results in an RDD
>> and then later iterating over them with an accumulator, defined for that
>> single type, but this would be much nicer.
>>
>> My question is: is there any other way to create accumulators or some
>> magic for making generics and objects work?
>>
>> Cheers,
>>
>> Johan
>>
>
>
