its definitely unfortunate that the current naming breaks scala's for
comprehension
On Sat, Mar 15, 2014 at 2:15 PM, andy petrella wrote:
> [Thanks a *lot* for your answers!]
>
> That's CoOl, a possible example would be to simply write a
> for-comprehension that would do this:
> >
> > val allEve
[Thanks a *lot* for your answers!]
That's CoOl, a possible example would be to simply write a
for-comprehension that would do this:
>
> val allEvents = for {
> deviceId <- rddFromHdfsOfDeviceId
> deviceEvent <- rddFromHdfsOfDeviceEvent(deviceId)
> } deviceEvent
> val hist = computeHistOf(
just going head first without any thinking, it changed flatMap to
flatMapData and added a flatMap. for FlatMappedRDD my compute is:
firstParent[T].iterator(split, context).flatMap(f andThen (_.compute(split,
context)))
scala> val x = sc.parallelize(1 to 100)
scala> x.flatMap _
res0: (Int => org.
MappedRDD does:
firstParent[T].iterator(split, context).map(f)
and FlatMappedRDD:
firstParent[T].iterator(split, context).flatMap(f)
do yeah seems like its a map or flatMap over the iterator inside, not the
RDD itself, sort of...
On Sat, Mar 15, 2014 at 9:08 AM, andy petrella wrote:
> Yep,
> R
Yep,
Regarding flatMap and an implicit parameter might work like in scala's
future for instance:
https://github.com/scala/scala/blob/master/src/library/scala/concurrent/Future.scala#L246
Dunno, still waiting for some insights from the team ^^
andy
On Wed, Mar 12, 2014 at 3:23 PM, Pascal Voitot D
On Wed, Mar 12, 2014 at 3:06 PM, andy petrella wrote:
> Folks,
>
> I want just to pint something out...
> I didn't had time yet to sort it out and to think enough to give valuable
> strict explanation of -- event though, intuitively I feel they are a lot
> ===> need spark people or time to move fo
Folks,
I want just to pint something out...
I didn't had time yet to sort it out and to think enough to give valuable
strict explanation of -- event though, intuitively I feel they are a lot
===> need spark people or time to move forward.
But here is the thing regarding *flatMap*.
Actually, it lo
