All,
We have an use case in which 2 spark streaming jobs in same EMR cluster.
I am thinking of allowing multiple streaming contexts and run them as 2
separate spark-submit with wait for app completion set to false.
With this, the failure detection and monitoring seems obscure and doesn't
seem to
> I know that the spark-submit script has a "--deploy-mode cluster" option.
> Does this mean that the receiver will be managed on the cluster?
>
> Thanks
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Spa
pt has a "--deploy-mode cluster" option.
> Does this mean that the receiver will be managed on the cluster?
> Thanks
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-in-Production-tp20644p20662.html
> Sent from the
I know that the spark-submit script has a "--deploy-mode cluster" option.
Does this mean that the receiver will be managed on the cluster?
Thanks
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-in-Production-tp20644p20662.html
Sen
Run Spark Cluster managed my Apache Mesos. Mesos can run in high-availability
mode, in which multiple Mesos masters run simultaneously.
-
Software Developer
SigmoidAnalytics, Bangalore
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-in
of streaming fault-tolerance semantics.
http://people.apache.org/~tdas/spark-1.2-temp/
On Thu, Dec 11, 2014 at 4:03 PM, twizansk wrote:
> Hi,
>
> I'm looking for resources and examples for the deployment of spark streaming
> in production. Specifically, I would like to know how
Hi,
I'm looking for resources and examples for the deployment of spark streaming
in production. Specifically, I would like to know how high availability and
fault tolerance of receivers is typically achieved.
The workers are managed by the spark framework and are therefore fault
tol
