The stream receiver seems to leverage actor receivers
http://spark.apache.org/docs/0.8.1/streaming-custom-receivers.html
But spark system doesnt lend itself to a messaging kind of a structure..
more of a DAG kind
Just curious are you looking for the actor subsystem to act on messages or
just l
Yeah that's the one we discussed...sorry I pointed to a different one that
I was reading...
On Wed, Aug 20, 2014 at 3:28 PM, DB Tsai wrote:
> To be specific, I was discussing this PR with Debasish which reduces
> lots of issues when sending big objects to executors without using
> broadcast exp
To be specific, I was discussing this PR with Debasish which reduces
lots of issues when sending big objects to executors without using
broadcast explicitly.
Broadcast RDD object once per TaskSet (instead of sending it for every task)
https://issues.apache.org/jira/browse/SPARK-2521
Sincerely,
D
Hi Patrick,
Last few days I came across some bugs which got exposed due to ALS runs on
large scale data...although it was not related to the akka changes but
during the debug I found across some akka related changes that might have
an impact of overall performance...one example is the following:
Hey Deb,
Can you be specific what changes you are mentioning? We have not, to my
knowledge, made major architectural changes around akka use.
I think in general we don't want people to be using Spark's actor system
directly - it is an internal communication component in Spark and could
e.g. be re
