Re: [re-cont] map and flatMap

On Wed, Mar 12, 2014 at 3:06 PM, andy petrella wrote: > Folks, > > I want just to pint something out... > I didn't had time yet to sort it out and to think enough to give valuable > strict explanation of -- event though, intuitively I feel they are a lot > ===> need spark people or time to move fo

[spark-streaming] Is this LocalInputDStream useful to someone?

Hi guys, In my recent blog post (http://mandubian.com/2014/03/08/zpark-ml-nio-1/), I needed to have an InputDStream helper looking like NetworkInputDStream to be able to push my data into DStream in an async way. But I didn't want the remoting aspect as my data source runs locally and nowhere else

Re: Making RDDs Covariant

Hi, Covariance always seems like a good idea at first but you must be really careful as it always has unexpected consequences... >From my experience, covariance often becomes a pain when dealing with serialization/deserialization (I've experienced a few cases while developing play-json & datomisca

Re: Making RDDs Covariant

On Sat, Mar 22, 2014 at 3:45 PM, Michael Armbrust wrote: > > > > From my experience, covariance often becomes a pain when dealing with > > serialization/deserialization (I've experienced a few cases while > > developing play-json & datomisca). > > Moreover, if you have implicits, variance often be

Re: Making RDDs Covariant

On Sat, Mar 22, 2014 at 8:38 PM, David Hall wrote: > On Sat, Mar 22, 2014 at 8:59 AM, Pascal Voitot Dev < > pascal.voitot@gmail.com> wrote: > > > The problem I was talking about is when you try to use typeclass > converters > > and make them contravari

Re: Making RDDs Covariant

I think we are going to be okay here since, as Koert > said, the current serializers use runtime type information. We could also > keep at ClassTag around for the original type when the RDD was created. > Good things to be aware of though. > > Michael > > On Sat, Mar 22, 201

Re: Making RDDs Covariant

rmation like that and stop too soon :) You know that I've been struggling for some time playing with spark & scalaz-stream and I'm curious ;) So without covariance I cannot for now define a generic noop Sink. > > My0.02c > Andy > > Sent from Tab, sorry for the typos... &

Re: [spark-streaming] Is this LocalInputDStream useful to someone?

Hi guys, Nobody saw/answered my question so I resend it ;) Thanks Pascal On Thu, Mar 20, 2014 at 9:17 AM, Pascal Voitot Dev < pascal.voitot@gmail.com> wrote: > Hi guys, > > In my recent blog post (http://mandubian.com/2014/03/08/zpark-ml-nio-1/), > I needed to have an I