Re: Getting around Serializability issues for types not in my control

2015-03-23 Thread Adelbert Chang
at 1:09 PM, Cody Koeninger wrote: > Have you tried instantiating the instance inside the closure, rather than > outside of it? > > If that works, you may need to switch to use mapPartition / > foreachPartition for efficiency reasons. > > > On Mon, Mar 23, 2015 at 3:03 PM

Re: Getting around Serializability issues for types not in my control

2015-03-23 Thread Adelbert Chang
Is there no way to pull out the bits of the instance I want before I sent it through the closure for aggregate? I did try pulling things out, along the lines of def foo[G[_], B](blah: Blah)(implicit G: Applicative[G]) = { val lift: B => G[RDD[B]] = b => G.point(sparkContext.parallelize(List(b)))